Custom Query (344 matches)

At least with subprocess.Popen() setting BEGINLIBPATH with either os.putenv or passing an env dictionary with this variable set has no effect. It used to work and could be broken by the latest subprocess changes (like r1281).

While using subprocess.PIPE works per se (i.e. you may read from this pipe later on in python code), chaining several processes together doesn't.

Consider this snippet:

import os, subprocess

os.putenv ('LANG', 'en_US')

cmd1 = ['cat', 't.py']
cmd2 = ['grep', 'cmd']

proc1 = subprocess.Popen (cmd1, stdout = subprocess.PIPE)
proc2 = subprocess.Popen (cmd2, stdin = proc1.stdout)

rc = proc2.wait ()
print '*** proc2 rc', rc

rc = proc1.wait ()
print '*** proc1 rc', rc

On OS/2 (python 2.7.6-18.oc00) it gives the following output:

grep: (standard input): Operation not supported on socket
*** proc2 rc 2
*** proc1 rc 0

Expected result (tested on macOS):

cmd1 = ['cat', 't.py']
cmd2 = ['grep', 'cmd']
proc1 = subprocess.Popen (cmd1, stdout = subprocess.PIPE)
proc2 = subprocess.Popen (cmd2, stdin = proc1.stdout)
*** proc2 rc 0
*** proc1 rc 0

I.e. the child process fails to read from its stdin (which is a pipe connected to the parent's stdout) on OS/2.

Note that if you replace cat t.py | grep cmd with e.g. rpm2cpio MYRPM | cpio -idm, the error will be as follows:

2 blocks
cpio: error closing archive: Bad file number

meaning that cpio worked per se (files unpacked correctly) but failed to close the input stream.

Also note that both examples work smoothly as shell commands from under dash or ash.

It looks like we're using pipes (or socket pairs?) incorrectly in Python.

In #275 subprocess.Popen was switched from fork to spawn. However, the new implementation is not thread safe. In order to implement piping of stdin/out/err in the started child the dup() technique is used where the parent stdin/out/err are temporarily associated with pipe handles and then reverted back after the child inherits them.

This is not thread safe as another parent thread might want to use stdin/out/err while they are redirected and the results may be quite unpredictable. One example is ​https://github.com/bitwiseworks/mozilla-os2/issues/247#issuecomment-346183854.

This needs to be fixed.