Make spawnvpe support extension-less executables

[dmik]

The current version of spawnpe() unconditionally adds an .exe extension to the program name if it appears to have no extension and then performs a search along PATH. As a result, if there is a program that has no extension (e.g. a bash/perl script without the .sh extension which is very common), such a program will never be found by this function.

Currently, this limitation may be worked around by creating a symlink to the program that has .exe on the end — this symlink will make spanwpe() happy. However, this workaround has several disadvantages:

1. It requires an additional file (FS pollution and such).
2. It fulls CMD.EXE which is unable to process symlinks and will just fail.

[dmik]

A better solution is to first search for a program as is (w/o touching its name or extension) and only add .exe if such an attempt fails. Attached is a patch that performs that (untested).

[dmik]

BTW, there are a few apps that involve the symlink workaround. Once the above patch is applied, these symlinks may be dropped. One such application is git, see ​http://trac.netlabs.org/ports/ticket/35).

[KO Myung-Hun]

spawnvpe() can launch a script without a shell such as cmd.exe or sh.exe ?

[dmik]

Not clear what exactly you mean. spawnvpe() supports the #! interperter notation. So if you write #! /path/myproc at the top of your script, it will try to locate /path/myproc, then myproc along PATH and then use what it finds (or report an error if it doesn't find anything). If there is no #! line at the top, it will fail. Note that the check for .cmd./.bat/.rex extensions comes before the #! check and if it succeeds, COMSPEC/OS2_SHELL/cmd.exe is always used to run the script. And also note that even before that a check for the EXE signature is done and on success the file is passed directly to DosExecPgm w/o using any interpreter.

[KO Myung-Hun]

I've found the reason why I failed. It's so funny. There was a directory named 'sh' in a working directory. So spawnvpe() failed.

Oweing to this, I've concluded that kLIBC have to respect a path of a interpreter. Currently, kLIBC searches a interpreter like this.

1. interpreter_name in PATH
2. interpreter_name + .exe in PATH

But, it should be like this.

1. interpreter_path
2. interpreter_path + .exe
3. interpreter_name in PATH
4. interpreter_name + .exe in PATH

For examples, /bin/sh should be searced like this.

1. /bin/sh
2. /bin/sh.exe
3. sh in PATH
4. sh.exe in PATH

[dmik]

You are not fully right. kLIBC currently searches it like this:

1. interpreter_path
3. interpreter_name in PATH
4. interpreter_name + .exe in PATH

So what it doesn't do is adding .exe to the full path (i.e. /bin/sh.exe won't be tried in such case, given your example). But it doesn't look like a real problem to me. If you do it for yourself, it's not a problem to put .exe to the script on your own. If you distribute things, it's bad practice to rely on the hard coded path anyway (except for "absolute" paths in the UNIXROOT environment but that ones usually provide symlinks of .exe to non-.exe or vice versa).

It may be added for the sake of completeness though. If you wish you may supply a patch.

[KO Myung-Hun]

Good point. But adding .exe implicitly is important on OS/2. None of OS/2 users expect that they fail to execute the executables just because they use a pathname without .exe extension. And I don't use #! unixy shell personally. Instead I use .cmd + extproc combination. So I don't care about a path problem in my own usages. But many ported unixy shells do not know of .exe. It's crazy to modify all them one by one to avoid this problem. In addition, using symlinks to avoid this is bad idea. I don't understand why you suggest this appraoch even though you already know the disadvantage of this as described in this ticket explanation. Just adding .exe implicitly is better and cleaner way than using symlinks.

FYI, I've found that this ticket was a duplicate of ticket #180.

[dmik]

I didn't say that symlinks are bad idea per se. This ticket is about spavnvPe not checking for file existence before adding .exe to it, it's completely different from the topic you raised in comment:5. It's actually off-topic here at all, since it's__spawnve that does the interpreter magic recognition. And It first checks for file existence and only probes .exe when nothing found.

Symlinks are good as a temporary workaround though. When you can't change the software and just need to make it work etc.

Getting back to your concern. You may have something like /usr/bin/perl in a Unix script and in the current state of the UNIXROOT support in LIBC it won't work at all in many cases (like when you are on a different drive or your UNIXROOT is not at root). The only way to make it work now is to add a directory with Perl to PATH (in this case it will be picked up regardless of if it ends with .exe or not) — and since /usr/bin is always in PATH, everything that expects shells in this directory will work regardless of the extension (through the PATH search mechanism).

But as I said, strictly speaking you are right and step 2. from your list is clearly missing here. Imagine that e.g. #285 is fixed and some script contains something very rare like /usr/libexec/myapp/myshell (which is normally not in PATH) — in this case, the script will fail if there is only $UNIXROOT/usr/libexec/myapp/myshell.exe and no extension-less symlink. So I recommend that you create a new ticket for this issue and provide a patch. This will significantly increase chances it gets fixed in the next release.

Yes, #180 looks like the same as this one. Thanks for finding it. I will close this ticket then.

[KO Myung-Hun]

Yes, our dicussion for adding .exe to interpreter_path was off-topic. As your suggestion, I opened a new ticket(#308) and attached a patch.

Thanks.

[dmik]

Great, thanks. BTW I closed #180 in favor of this ticket as this one provides a patch and more background information.

[bird]

Too much noise here, and the original report has precedence. Closing as duplicate of #180.

[dmik]

Okay, and doing a general fix (as in #180) is surely better. Thanks for this and (on this occasion) for all other recent fixes.