Kernel threads implementation (assertion failed in vboxdrv.sys)

[Valery V. Sedletski]

When testing a new VBox build on an Intel Core i3 machine, I got a trap 3 when booting the support driver (no such problems on Athlon64 and Core2Duo machines). It is probably, a failed assertion.

Trying to determine the assertion address, I got the cause of this problem known. The failed assertion is in Runtime\common\misc\thread.cpp, in RTThreadCreate(), the AssertReleaseRC(rc) at the end of the function. Here, rc == -12 == VERR_NOT_IMPLEMENTED is returned, hence the failed assertion. The error code was returned from rtThreadNativeCreate(). This function is unimplemented, and returns the above return code.

The kernel thread is VBoxTscThread, and could be disabled with undefining SUPDRV_USE_TSC_DELTA_THREAD. The disabling of this thread is not fatal, and everything seems to work without it. Also, I saw the messages in VM logs like this:

GIM: Warning!!! Host TSC is unstable. The guest may behave unpredictably with a paravirtualized clock.

The problem is that Runtime\r0drv\os2\thread2.cpp API's are unimplemented, and contain dummies. And the problem is that the kernel threads in OS/2 are a problem, because no preempting in kernel. The kernel multitasking in OS/2 kernel is cooperative one.

So, the question remains to be open, how to implement/emulate kernel threads in OS/2

1) I heard from Pavel Shtemenko that JFS uses some kind of a lazy writer thread in ring0. So, we may check JFS sources, what is used to start such threads. Also, there's some number of 'threads' in kernel, like sysinit, ager, etc.

2) OS4User suggested to try using context hooks to emulate kernel threads.

Context hook is created once with DevHlp_AllocateCtxHook, and can be "armed" multiple times with DevHlp_ArmCtxHook. The hook is executed in task time in the first available context (ring0). AllocateCtxHook returns a hook handle, which can be used until it is freed with DevHlp_FreeCtxHook.

PS: here Ctx hooks also seem to be used for kernel threads: ​http://trac.netlabs.org/ahci/browser/trunk/src/os2ahci/ctxhook.c

[erdmann]

I found this, the attachment to the ticket shows how OS/2 kernel threads can directly be created (without the use of some Ring 3 daemon program calling into the device driver and the thread being blocked in the device driver) by using the VDH routine "VDHCreateThread":

​http://trac.netlabs.org/uniaud/ticket/78

Additionally I found this:
​http://www.yqcomputer.com/515_1075_1.htm

<quote>
The usage is:

BOOL _Pascal NEAR VDHCreateThread(PTID,PFN);
VOID _Pascal NEAR VDHExitThread(ULONG);

There was a short discussion about VDHCreateThread in OS/2 Device Driver
Programming at yahoo:

​http://www.yqcomputer.com/

In my LXAPI32.SYS device driver I've implemented "linux kernel threads"
with this function. But you have to be carefull. If a thread is started
by an normal applications request (open, ioctl, etc.) the new created
thread belongs to the application. It hangs on exit until the thread ends.
A workaround is to create a "real kernel thread" as a worker thread at
device driver INIT time. This thread waits for a specific condition (for
example a semaphore). If a kernel thread should be started at
applications request it posts the semaphore. Then the worker thread
starts an other thread.
LXAPI32.SYS works this way.

</quote>

[Valery V. Sedletski]

2Lars: Oh, this is a new fresh idea. So, VDD's can create threads? -- I know that I can import VDD helpers into a driver of IFS. But you say it is started at init time in LXAPI32.SYS. -- But init routine runs at ring3, and VDH are intended for ring0. So, how is a worker thread started then (from ring3)? And it may, probably, prevent the sysinit process to terminate, until the worker thread terminates (you said that application may block on exit until the thread ends)
And, it is needed to be created not by application request, but by the driver request -- in context of vboxdrv.sys or vboxguest.sys. So, could I just call VDHCreateThread in a driver, and create a thread? Which context will the created thread belong? I suspect that it will be a normal ring3 thread in context of an application which calls the VDD...

[erdmann]

Sorry for not answering any earlier:
1) LXAPI32.SYS is a BASEDEV= type driver. A BASEDEV= type driver initializes at Ring0. Only DEVICE= type drivers initialize at Ring3.
2) VDDs also initialize at Ring0. As such, if you invoke VDHCreateThread from a BASEDEV= driver it will work at any time.
3) Only an application calling into a driver which in turn starts a VDHCreateThread will block until that newly started thread terminates. That's because in this case the newly created thread runs in the context of the application calling into the driver. If the driver itself (from INIT or INIT_COMPLETE) starts a thread, it will run in the context of the kernel. You will have noticed that the "sysinit" process (which in this sense is the kernel context) will NEVER terminate. It will continue to exist until you shut down the system. Therefore there is no problem if you start a thread in the context of sysinit.
4) Yes you can call VDHCreateThread in a driver. You just need to be aware of one thing:
VDHCreateThread needs to be called from a 32-bit code segment and it needs DS and ES being set to the flat Ring0 data selector (DOS32FLATDS) when it is invoked. Nonetheless, a VDD and therefore also VDHCreateThread expects to use a 16-bit stack segment. However, DEVICE= and BASEDEV= type drivers will already execute with a 16-bit stack being active. Therefore, everything is just fine from a stack point of view.
5) The routine invoked by VDHCreateThread needs to follow the FAR32 PASCAL calling convention. In particular this means that it has to return with a FAR32 return !! It would also need to remove all parameters from the stack on return, however this is irrelevant in this case as the thread function takes no argument.
You also need to be aware that when the thread routine is invoked, a 16-bit stack will be active. Therefore, if you intend to execute/call 32-bit code in that thread you need to thunk to a 32-bit stack (via KernThunkStackTo32) at the beginning and thunk back to 16-bit (via KernThunkStackTo16) at the end of the thread routine.