perl: system() fails on sh scripts in scalar mode

[dmik]

The system() call fails with SYS1041 (The name NNN is not recognized...) when passed a scalar argument like this and the command to execute is a SH script:

system("aclocal --version");
if ( $? == -1 )
{
  print "command failed: $!\n";
}
else
{
  printf "command exited with value %d", $? >> 8;
}

This must be because according to the ​docs scalar arguments are executed with the shell command which in case of OS/2 seems to be cmd.exe /c.

Interesting is that if you use the list form instead (system("aclocal", "--version")), all works. Perhaps, because LIBC exec() is used in this case which can properly recognize SH scripts and use SH.EXE for them.

This needs to be fixed as it obviously breaks many things (like automake).

[dmik]

A possible fix is to grab the LIBC logic of detecting the current shell (see ​http://trac.netlabs.org/libc/ticket/287 for details) and depending on it use either -c or /c.

[dmik]

I can't build perl so far. Various issues. Both from SRPM and from SVN.

The SVN build fails because it can't properly generate regnodes.h for some reason.

The RPM build fails because Extutils/MakeMaker.pm is not at the right place and I see no code that would put it there. I see however that a file lib/buildcustomize.pl is generated and this file contains the right INC statements that would make MakeMaker.pm available for miniperl.exe. There is also an option to Configure (usesitecustomize, defines a macro USE_SITECUSTOMIZE) that enables buildcustomize.pl processing, but it is not defined by the build process. This all looks completely weird to me.

[dmik]

Okay, I found the reason. USE_SITECUSTOMIZE is set automatically whenever PERL_IS_MINIPERL is set. The problem was that the OS/2 makefile generator (os2/Makefile.SHs) is outdated and didn't pass PERL_IS_MINIPERL to the miniperl build command. The fix is attached below.

I didn't commit it to SVN yet because SVN is somehow out of sync with the source RPM (i.e. files os2/os2.sym and os2/os2add.sym are missing from SVN for some reason, there are some more). First, I want it to get back in sync. Yuri, can you comment on this?

[dmik]

Going further. The title of the defect is not entirely correct. It fails not all SH scripts but only on some. In short, perl tries to guess if the first argument to system() is an executable or a script. If it thinks the file is a script, it tries to process the #! magic which tells which script interpreter to use on its own. If you specify a full path to a normal OS/2 EXE there, all works as expected. However, if what you give after #! is a symlink to an exe, perl fails to recognize it as an exe and falls bak to a default case under which it assumes some unknown command or script is given and calls the system shell passing it everything in a hope that the system shell will figure that out.

So there are two separate problems:

1. file_type() falis to recognize symlinks to exes as exes.
2. perl assumes that the system shell is always an OS/2 CMD.EXE compatible shell and hence uses the CMD /C syntax to start unknown commands and there is no way to override it.

Fixing any of the above would fix the particular problem I have with aclocal from autoconf but I think we should fix both.

In general, this part of the code is very dirty (many warnings marking dead code and such), overcomplicated and I really don't like that perl tries to be smarter than LIBC (which also has shell and script launcher detection which usually works OK). But it's too much work to do a full cleanup so I will only fix the mentioned problem for now.

[dmik]

Yuri has added missing files in r741 and r742. Now, together with the committed miniperl-build-fix.diff in r743, I can successfully build perl from SVN trunk.

In r744 I also fixed problem 1 from comment:4.

[dmik]

Regarding problem 2. Actually, perl already recognizes the EXECSHELL variable which you can use to specify the default shell for unknown scripts instead of OS2_SHELL. It didn't help me before because it only allows to specify the shell name and not the argument used to issue a command (like /C). However, the real problem with aclocal was not using CMD.EXE instead of SH.EXE but the fact that #!perl inside this script referred to a symlink on my machine (which points to perl.exe) and prior r744 understanding symlinks at this place was broken. Now it's fixed and it doesn't seem to be necessary to support EMXSHELL/SHELL/whatever with the -c option instead of OS2_SHELL since all known scripts will work well now (given that the shell executable is correctly specified in #! and exists). If it's not the case, then passing the script to EXECSHELL directly (i.e. w/o the -c option) will work: the specified shell is supposed to read it as a script and execute it (which of course means that the "unknown" script and the shell in EXECSHELL match).

[dmik]

I also found by accident and fixed in r745 another quite a nasty bug related to shell script execution from perl. If the shell specified in #! did't point to an existing file perl would endlessly try to search for it until killed. I'm sure I saw this behavior before.

[dmik]

This seems to be done. I or Yuri will release a new perl RPM soon.