Trap 000e when exiting VBox on IBM's kernels

[Valery V. Sedletski]

When exiting VBox (either, VirtualBox.exe or VBoxSDL.exe) on IBM's kernels, I get a trap:

Trap 14 (0EH) - Page Fault 0000, Not Present, Read Access, Supervisor
eax=fe3c2320 ebx=fd39dce0 ecx=00000271 edx=cb4a1900 esi=00000000 edi=f8c98c82
eip=fff3fbe9 esp=0000529a ebp=000052be iopl=0 rf -- -- nv up ei pl zr na pe nc
cs=0178 ss=1530 ds=0170 es=0170 fs=0000 gs=0000 cr2=fd39e1c2 cr3=00225000 p=00
0178:fff3fbe9 66833c4b00     cmp   word ptr [ebx+ecx*2],+00 ds:fd39e1c2=invalid

0178:fff3fbe9 66833c4b00     cmp       word ptr [ebx+ecx*2],+00
0178:fff3fbee 7515           jnz       fff3fc05
0178:fff3fbf0 8b5f1c         mov       ebx,dword ptr [edi+1c]
0178:fff3fbf3 8bf1           mov       esi,ecx
0178:fff3fbf5 53             push      ebx
0178:fff3fbf6 50             push      eax
0178:fff3fbf7 56             push      esi
0178:fff3fbf8 e813ebffff     call      fff3e710
0178:fff3fbfd 8bce           mov       ecx,esi
0178:fff3fbff 8bf0           mov       esi,eax
0178:fff3fc01 85f6           test      esi,esi
0178:fff3fc03 751b           jnz       fff3fc20
0178:fff656c2 f8c98c82 000000b5 fff3ee6b 00000000 _ldrFreeMTE + 284
0178:fff4bb02 000bc640 fe3c2228 000000b5 00008000 _SEMApiAddMuxWaitSem + d4
0178:fff4c284 f8c98d46 f8c98d62 00000000 e5d470a0 _SEMApiCreateMuxWaitSem + a0
0178:fff664ea 00000018 f8c98d62 00000000 e5d470a0 _ldrLoadImports + ed
0178:fff95ead bc640000 00000000 00000018 fb800c18 _TKScanTasks + 1cb
0178:fff960ae fd39f194 ffffffff 00000000 00000001 _TKUnlinkTask + f4
0178:ffeffa67 fe3c2228 0000ab0d 00000158 a35bb644 _dhw_Register + 3eb
0178:fff70859 00000286 fe30c77c fe30c77c fff33f5a _LDRFreeModule + 2a9
0178:fff0a609 00000000 00000000 00000003 fe30c77c _VDMINT10IssueRequest + 50d
0178:fff33d7c f8c98f48 fff870c6 00000001 00000000 _THEGlobal_get + 4c3
0178:fff3fb3c os2krnl:HIGH32CODE:g_GetPICRegs_Mask + c9
0178:fff3fcc0 SIGPOSTPENDINGHOOK - bb
##

There's no such trap on OS/4 kernels. As we see here, the trap occurs on freeing MTE's for VBox binaries. This seems to be because of loading the binaries to high memory (the binaries were processed with highmem.exe, and it maybe, because of this)

[diver]

btw to which IBM kernel do you refer? 14.106 or older?

[Valery V. Sedletski]

2diver: 14.104_SMP. But I will test it with 14.106 too. I just haven't installed it on that machine.

So far I played with highmem.exe utility, and see that some DLL's really won't be loaded to high memory entirely. Some of them need to be loaded its code (not data) only to high mem. I got the following table:

vnccli0    b
vncsrv0    b
vboxxpc    b
vboxsis    b
vboxsfld   b
vboxsclp   b
vboxhlp    b
vboxhchn   b
vboxdnd    b
vboxdbg    b
vboxdbgd   b
vboxcapi   b
vboxauts   b
vboxauth   b

vboxsvcm   b
vboxipcc   b
vboxc      b

vboxdtc    b
vboxdtrc   b

vboxvnc    b
vboxvncm   b

pthr01     b
png1616    b
mmap       b
libxml2    b
libvpx2    b
jpeg       b
expat7     b
curl7      b
crypto10   b
crypt32    b
aio        b

ssl10      c
stdcpp6    c
urpo       c
z          c

vboxdd     c
vboxdd2    c
vboxddu    c
vboxrem    c
vboxrm32   c
vboxrm64   c
vboxrt     c
vboxvmm    c
vboxxcom   c

here (c) means "code only", and (b) means "both data and code".

So, If I mark the above DLL's segments as described, I don't get the subject trap. But this is just in case SMP is disabled in BIOS. If it is not disabled, I still get the trap mentioned above. Not sure about how are high memory issues and SMP are connected.

[Valery V. Sedletski]

Upd: I tried IBM kernel 14.106_SMP (with or without PSD) and now see that this trap is gone (like with OpenOffice). But I get much errors like

libc: could not register program in shared memory

so, I need to reboot constantly, and it is unusable. OS/4 kernels don't cause this error. I run vboxsdl.exe once and then already see this. After this error all kLibc processes won't start until reboot.

[diver]

this is what I also saw. The smaller VAL is the better it runs. I have no idea really why this is the case. As I have completely no idea how VAL works at all.

[Valery V. Sedletski]

2diver: Not so simple. VAL is the address, above which the kernel memory resides. Lower than it -- application area. So, lower VAL means less memory for applications and more -- for kernel and drivers. There needs to be some balance between those. So, the problems with "libc: could not register program in shared memory" could be because of a shortage in kernel memory. So, we need to decrease VAL a bit (but not too low -- somewhere between 2048 and 3072)

[Valery V. Sedletski]

*because of shortage in application memory, I mean