Changes between Initial Version and Version 1 of Ticket #275, comment 1


Ignore:
Timestamp:
Oct 31, 2017, 7:21:32 PM (7 years ago)
Author:
dmik
Comment:

Legend:

Unmodified
Added
Removed
Modified

  • Ticket #275, comment 1

    initial v1  
    1 The primary reason for this change is waf (a python based make framerowh) which is used in Samba 4. And in the current implementation Python fails to start a child process after creating a certain number of internal build task objects (here it starts failing with 1500 such objects). Note that fork itself works well, ie a new child process gets created as a copy of the parent Python process. However an attempt to start a right executable (gcc.exe in this case) within this forked process with exec results in DosExecPgm returning ERROR_BAD_ENFIRONMENT. I have never seen this error and don’t know what it means exactly but my guess is that cloning the parent Python process data pages in the forked child somehow exhausts some system resources and/or breaks some segment layout needed for DosExecPgm when there is too many data (too many Python objects) to clone.
     1The primary reason for this change is waf (a python based make framework) which is used in Samba 4. And in the current implementation Python fails to start a child process after creating a certain number of internal build task objects (here it starts failing with 1500 of such objects). Note that fork() itself works well, ie a new child process gets created as a copy of the parent Python process. However an attempt to start a right executable (gcc.exe in this case) within this forked process with exec results in DosExecPgm returning ERROR_BAD_ENVIRONMENT. I have never seen this error and don’t know what it means exactly but my guess is that cloning the parent Python process data pages in the forked child somehow exhausts some system resources and/or breaks some segment layout needed for DosExecPgm when there is too many data (too many Python objects) to clone.
    22
    3 Note that changing it to spawn will let other use cases benefit as well, both in terms of speed (cloning process data requires some time)  and memory footprint (a clean process will use much less memory than a fully cloned Python process with a lot of objects).. Mozilla which now uses Python as a make framework is one example. Currently it’s quite slow and memory consuming.
     3Note that changing it to spawn() will let other use cases benefit as well, both in terms of speed (cloning process data requires some time)  and memory footprint (a clean process will use much less memory than a fully cloned Python process with a lot of objects). Mozilla, which now uses Python as a make framework, is one example. Currently, it’s quite slow and memory consuming.