libc: fork() failed when using dlopen()

[froloff]

If process load module dynamically (dlopen) after that fork will fail.

I examine the situation a bit and divide it into 3 cases

1. process dlopen() module then forking. Child process failed during initial fork processing.
2. process dlopen() module, dlclose() module then forking. Parent process trapped during fork processing.
3. process dlopen() module, dlclose() module, dlopen() module again then forking. Parent process goes endless loop in forkParValidateModules() during fork processing.

I'll send testcase and libc logs via email.

Note. I didn't find code deregistering module from modules chain - conj procedure to __libc_ForkRegisterModule().

[bird]

I've analysed and fixed the problems.

The first problem (dlopen(); fork();) fails because of recursive execution of the initial fork buffer in forkChlDoFork(). The recursive call occurs when process the DosLoadModuleEx record for the loaded dll. The fix is simply to advance the fork stage before processing the buffer.

The second problem and third problem are cause by missing fork module deregistration when a DLL is unloaded. The fix is to implement this deregistration. This fix will require relinking of DLLs as it includes dll0.s changes.

There is a forth problem here, the sequence fork(); waitpid(); dlopen(); dlclose(); fork(); waitpid(); fails on the last waitpid. This is covered by ticket #114.

[bird]

(In [2786]) Fixed problems with fork() and module loading/unloading. Fixes #76. Fixed atexit() and on_exit() problem with callbacks in unloaded DLLs. Fixes #103.

[bird]

(In [2787]) Fixed problems with fork() and module loading/unloading. Fixes #76. Fixed atexit() and on_exit() problem with callbacks in unloaded DLLs. Fixes #103.