java:ant fails with WindowsError [Error 193]

[rkdecampo@…]

I recently upgraded to Bitten 0.6b2 and found that my previously working builds are now broken. It appears that Bitten is having difficulty executing ANT on Windows:

[ERROR   ] Error executing 
['S:\\Freeware\\ANT\\xyzzy\\bin\\ant', '-noinput', 
'-listener', 'org.apache.tools.ant.XmlLogger', 
'-Dant.XmlLogger.stylesheet.uri', '""', '-DXmlLogger.file', 
'c:\\docume~1\\admini~1.pro\\locals~1\\temp\\ant_logo0pyvr.xml', 
'-buildfile', 
'c:\\docume~1\\admini~1.pro\\locals~1\\temp\\bittencq2i3t\\build_Main_149\\xyzzy.xml', 'all']: 
WindowsError [Error 193] %1 is not a valid Win32 application

The previously working version had not defined a build INI file. In an attempt to get the new version to work I have defined one and specified a [ant] and [java] section but this does not help.

[osimons]

Could you run the build slave with -v option (verbose logging), and include the other related debug messages that should appear right before it fails.

[anonymous]

Here you go, but I am not sure they will help much:

[INFO    ] Slave launched at 2009-09-14 16:48:20
[DEBUG   ] Authentication not provided. Attempting to execute build anonymously.

[DEBUG   ] Configured packages: {'ant': {'home': 'S:\\Freeware\\ANT\\xyzzy'}, 'java': {'home': 'C:\\jdk1.6.0_01'}}
[DEBUG   ] Sending slave configuration: <slave version="2" name="devsrv01">
<platform processor=""/><os version="2000" family="nt">Windows</os>
<package home="S:\Freeware\ANT\xyzzy" name="ant"/>
<package home="C:\jdk1.6.0_01" name="java"/></slave>
[DEBUG   ] Sending POST request to 'http://devsrv01/trac/xyzzy/builds'
[INFO    ] Build pending at http://devsrv01/trac/xyzzy/builds/149
[DEBUG   ] Sending GET request to 'http://devsrv01/trac/xyzzy/builds/149'

[DEBUG   ] Running build in directory c:\docume~1\admini~1.pro\locals~1\temp\bit
teng8bqpv\build_Main_149

I snipped a checkout step here which worked fine.

[DEBUG   ] Sending POST request to 'http://devsrv01/trac/xyzzy/builds/149
/steps/'
[INFO    ] Executing build step 'build'
[DEBUG   ] Executing <function ant at 0x00CF4930> with arguments: 
{'file_': 'xyzzy.xml', 'target': 'all'}
[ERROR   ] Error executing ['S:\\Freeware\\ANT\\xyzzy\\bin\\ant', '-noinput', 
'-listener', 'org.apache.tools.ant.XmlLogger', 
'-Dant.XmlLogger.stylesheet.uri', '""', '-DXmlLogger.file', 
'c:\\docume~1\\admini~1.pro\\locals~1\\temp\\ant_logignxo6.xml', 
'-buildfile', 
'c:\\docume~1\\admini~1.pro\\locals~1\\temp\\bitteng8bqpv\\build_Main_149\\xyzzy.xml', 'all']: WindowsError [Error 193] %1
 is not a valid Win32 application
[ERROR   ] Build step 'build' failed
[DEBUG   ] Sending POST request to 'http://devsrv01/trac/xyzzy/builds/149/steps/'
[WARNING ] Stopping build due to failure

It appears to me that whatever process creation code is being used is expecting an executable and not a batch file. E.g., svn works fine, but ant.bat is not executed.

[osimons]

Replying to anonymous:

It appears to me that whatever process creation code is being used is expecting an executable and not a batch file. E.g., svn works fine, but ant.bat is not executed.

The way commands are executed in subprocesses has been reworked recently, and one thing I did was drop execution through shell on Windows. That requires scripts and shell builtins to be called as something like cmd.exe /C executable args.

Could you try this patch, and if it works I'll see how to integrate this:

bitten/build/javatools.py

@@ -60 +60 @@
     if target:
         args.append(target)
 
+    # If Windows, execute through shell to interpret the shell script
+    if os.name == 'nt':
+        args = ['/C', executable] + [arg.strip('"') for arg in args]
+        executable = os.environ['COMSPEC']
+
     cmdline = CommandLine(executable, args, cwd=ctxt.basedir)
     for out, err in cmdline.execute():
         if out is not None:

[me@…]

I'm having a similar problem since Bitten 0.6b2; on windows only, I am getting:

[INFO    ] Executing build step 'compile'
[DEBUG   ] Executing <function ant at 0x019614B0> with arguments: {'args': '-v -
d -Dname=scan_test-pc', 'file_': 'build.xml', 'target': 'integration-build'}
[ERROR   ] Error executing ['ant', '-v', '-d', '-Dname=scan_test-pc', '-noinput'
, '-listener', 'org.apache.tools.ant.XmlLogger', '-Dant.XmlLogger.stylesheet.uri
', '""', '-DXmlLogger.file', 'c:\\users\\scan_t~1\\appdata\\local\\temp\\ant_log
all3ij.xml', '-buildfile', 'c:\\users\\scan_t~1\\appdata\\local\\temp\\bittenp62
_o-\\build_Branch_2334\\build.xml', 'integration-build']: WindowsError [Error 2]
 The system cannot find the file specified
[ERROR   ] Build step 'compile' failed
[DEBUG   ] Sending POST request to 'https://dev.nolij.com/trac/NolijWeb/builds/2
334/steps/'
[WARNING ] Stopping build due to failure
[DEBUG   ] Removing build directory c:\users\scan_t~1\appdata\local\temp\bittenp
62_o-\build_Branch_2334

[osimons]

Replying to me@…:

I'm having a similar problem since Bitten 0.6b2; on windows only, I am getting:

Could you try to patch the slave with patch from comment:3?

[rkdecampo@…]

Replying to osimons:

Could you try this patch, and if it works I'll see how to integrate this:

Yes, it is working with the patch applied, thanks.

[anonymous]

Replying to osimons:

Replying to me@…:

I'm having a similar problem since Bitten 0.6b2; on windows only, I am getting:

Could you try to patch the slave with patch from comment:3?

This patch did not completely solve it for me; I had to also add an additional line to accomodate some missing quoting. Am I really the only person who has this problem? I would think this quoting would break every java builder in windows(?)

Index: build/javatools.py =================================================================== --- build/javatools.py (revision 768) +++ build/javatools.py (working copy) @@ -51,7 +51,7 @@

else:

args = []

args += ['-noinput', '-listener', 'org.apache.tools.ant.Xml Logger?',

• '-Dant.Xml Logger?.stylesheet.uri', '""',

+ '-Dant.Xml Logger?.stylesheet.uri=\'\,

'-DXmlLogger.file', logfile.name]

if file_:

args += ['-buildfile', ctxt.resolve(file_)]

@@ -60,6 +60,11 @@

if target:

args.append(target)

+ # If Windows, execute through shell to interpret the shell script + if os.name == 'nt': + args = ['/C', executable] + [arg.strip('"') for arg in args] + executable = os.environCOMSPEC? +

cmdline = Command Line?(executable, args, cwd=ctxt.basedir) for out, err in cmdline.execute():

if out is not None:

[osimons]

Patch from comment:7 for clarity:

build/javatools.py

@@ -51 +51 @@
     else:
         args = []
     args += ['-noinput', '-listener', 'org.apache.tools.ant.XmlLogger',
-             '-Dant.XmlLogger.stylesheet.uri', '""',
+             '-Dant.XmlLogger.stylesheet.uri=\'\'',
              '-DXmlLogger.file', logfile.name]
     if file_:
         args += ['-buildfile', ctxt.resolve(file_)]
@@ -60 +60 @@
     if target:
         args.append(target)
 
+       # If Windows, execute through shell to interpret the shell script
+       if os.name == 'nt':
+               args = ['/C', executable] + [arg.strip('"') for arg in args]
+               executable = os.environ['COMSPEC']
+
     cmdline = CommandLine(executable, args, cwd=ctxt.basedir)
     for out, err in cmdline.execute():
         if out is not None:

As an alternative, could you perhaps alo drop this part in my patch:

 + [arg.strip('"') for arg in args]

and replace it with:

+ args

(ie. remove the part where i strip quotes...)

It needs to execute through the Windows shell, and with cmd.exe the quoting of arguments are different from normal.

Yet another alternative may be just to do the ant agument as '' (ie. emtpy string) as i think that will also be quoted through the shell.

Does any of the alterantives also work?

[osimons]

Adding a shell argument for CommandLine for executing Windows batch files and builtins.

[osimons]

I'm now thinking that perhaps something like attachment:t449-commandline_nt_shell-r768.diff​ is the right way forward. Unlike on *nix platforms, ant.bat needs a shell somehow. Instead of doing cmd.exe /C <script> <args> ourselves we should just use the shell-handling of subprocess.Popen().

This patch adds the same behavior for any <sh:exec file="mycript.bat"> commands on Windows. These currently do not work and needs fixing anyway.

Patch also adds a note in documentation about quoting and Windows command shell: It may well pass on arguments to scripts and builtins with different quoting than what was actually input.

[osimons]

Last patch committed in [784:785] - this should fix this issue, and generally make script execution on Windows work again.