Build JDK

[dmik]

This ticket is only to track the JDK build (make jdk), no hotspot, corba, jaxp and jaxws.

The JDK build within this ticket may not include the GUI facilities yet.

[dmik]

With the release build, I get a C heap corruption here, at the very end of the java invocation. The debug build shows endless attempts to call HeapWalk? (probably it tries to debug this heap corruption on its own).

[dmik]

It seems that I bumped into the first problem with the bootstrap JDK version: JAXB1.0 is needed for building JDK 6 but it is missing from JDK 1.5.0 we use for bootstrapping. I will either have to find a separate JAXB1.0 distribution for JDK 5 or try to build JAXP from JDK 6 first and use it instead.

[dmik]

Found JAXB 2.1 as ​https://jaxb.dev.java.net/2.1/JAXB2_20061211.jar. Run it with java -jar (it will create a directory structure jaxb-ri-20061211) and copy all jaxb-ri-20061211/lib/*.jar to the e.g. JDK 1.5's jre/lib/ext subdirectory. The build now continues.

[dmik]

Now I get the famous "Record too long" message from emxomfar when linking fdlibm.lib. I will have to fix it now.

[dmik]

Ahh, there is nothing wrong with emxomfar actually! The .obj files were in aout format instead of OMF (due to a missing flag which I fixed in r75).

BTW, Silvan, could you resend me an object file that was causing the same problem for you back then? I could recheck it now while my hands are on it.

[dmik]

The windows code uses LIBC wide character alternatives like _wgetdcwd() _wfullpath() all over the place, so I think it makes sense to provide their implementations in a central place once in Odin instead of porting each individual call. There is a MSVCRT emulation library in Odin and it contains implementations of those functions, but it doesn't build by default. I will see if it's alive at all.

[dmik]

Created ​http://svn.netlabs.org/odin32/ticket/8 for this.

[dmik]

Resolved the problem by creating MINIVCRT.LIB in Odin that provides necessary functions. Going further.

[dmik]

As of r86 (and the corresponding Odin32 revision) java.dll and several other libraries are now built.

[dmik]

jil.dll is now built.

[dmik]

Built java.exe and javaw.exe. They successfully load JVM.DLL and a couple of other Java DLLs but then terminate with:

Could not create the Java virtual machine.

Probably, some other essential module is still missing. Investigating.

[dmik]

The previous problem is solved (we were missing a few exports and some other bits like the correct version information in JVM). I also ported a few more dlls (like ZIP.DLL) and went further. Now it dies with the assertion:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error (d:/Coding/javaos2/openjdk/hotspot/src/share/vm/oops/methodOop.hpp:631), pid=44285, tid=2902261764
#  Error: assert(is_native(),"must be native")

It apparently tries to call some native method but screws up. Some more bits must be missing (there is a bunch of small sub-components to port yet).

[dmik]

We've got the first serious issue: net.dll that (among other things) provides native method implementations for the java.net.NetworkInterface? class, uses things not available in Odin32 (this includes iphlpapi.dll and the registry-based approach to enumerate network interfaces). I guess we will have to provide a native OS/2 implementation for NetworkInterface? and some related classes. In the mean time, I will provide a dummy implementation to let us move further. Created #14 for that.

[dmik]

It doesn't actually make much sense to continue w/o the network support so I will switch to #14 now.

[dmik]

Done #14 and also nio.dll.

[dmik]

Done management.dll.

[dmik]

And instrument.dll.

[dmik]

To debug the JVM, I need to make the debug build (this requires some tailoring of global makefiles of both hotspot and jvm).

[dmik]

Argh. JDK make system is incredibly, incredibly stupid. It's painfully illogical, complicated, does not support building of separate sub-components and takes ages to do anything.

[dmik]

In fact, dealing with it (and with the process of building in general) takes 70% of my time on this project -- during this time I either sit and wait or dig into the make files searching for one small damn thing.

[dmik]

Done with the debug build. Let's see what's really wrong with the JVM when we attempt to start it.

[dmik]

GetModuleInformation?() from Open32 (wgss50.dll) does not seem to work correctly: it returns strange numbers as the base DLL address and zero as the image size, for all loaded DLLs. The Java code relies on that -- in particular, it uses this information to detect which DLL the given address in the process space belongs to. It seems that I will have to rewrite it using the native OS/2 calls...

[dmik]

To say the truth, GetModuleInformation?() is actually implemented in Odin itself (wprocess.cpp) but it somehow doesn't provide the correct information (for example, it uses a module handle as a base address of the DLL which doesn't correspond to the reality). The reason for that as I guess is that native OS/2 LX DLLs that participate in Odin (which Odin-based Win32 applications look at as Win32 DLLs) consist of multiple objects of different size and these objects are not necessarily loaded one by one into a contiguous block of memory. For this reason, the OS/2 DLL memory layout cannot be mapped 1-to-1 to the Win32 scheme where the DLL occupies a single block of memory (starting at MODULEINFO::lpBaseDll and ending at MODULEINFO::lpBaseDll + MODULEINFO::SizeOfImage? as I understand it).

[dmik]

Reimplemented the respective Java calls using OS/2 native API for that sort of things in r113. This required adding these APIs to Odin OS/2 API wrapper libraries.

Debugging the JVM crash further.

[dmik]

Finally, I found and fixed the problem with exceptions (and related crashes) puzzling me within the last few days. The fix is in Odin (see r21433). We are now really close to make it work.

[dmik]

Odin's r21434 actually.

[dmik]

BTW, there is a real mess with underscores for public functions in Windows compilers, see ​http://wyw.dcweb.cn/stdcall.htm. On OS/2, in GCC, all those functions are always exported with underscores regardless of the calling convention and declspec(). For this reason, we have to manually remove the underscores (and in some cases, the @ suffix) using the DEF file (or, to be exact, a special rule in the Makefile that creates the DEF file). Just a note for whom it may concern.

P.S. Probably, the ideal solution would be to patch the EMX target in GCC so that it emits export names in a way fully compatible with MSVC depending on how stdcall and declspec(dllexport) are used (which are originally MSVC things anyway) but that's too much work.

[dmik]

After r119, java -version works, finally. As well as many console classes which this command line option involves.

[dmik]

In r124, fixed the crash that happened in the release build of JVM due to too the extensive optimization.

[dmik]

An interesting problem has arisen. On Windows, once you load a DLL by full path (e.g. D:\DIR\MYLIB.DLL) it becomes accessible by name (i.e. just MYLIB) for the current process which means that if another DLL you load imports something from MYLIB (import table always uses the short name) it will successfully find the already loaded copy and use it. While sounds logical and expectable, it's not the case on OS/2.

In case of Java (which loads JVM and many other its DLLs by the full path that it detects at runtime), we have the following: VERIFY.DLL gets loaded successfully, but JAVA.DLL can't be loaded because of VERIFY which can't be found in LIBPATH or BEGINLIBPATH (remember, the library itself was loaded using the full path)...

I don't see any generic solution we could implement in Odin in order to solve it once and for all so we will have to play with the in-place BEGINLIBPATH setting in the required places of the Java sources. I already did that once for JVM, the same has to be done for VERIFY now. There may be more such cases.

[dmik]

The problem is fixed in r134.

[dmik]

Debug version works. Release version with asserts works. Production release version crashes. F*ck.

[dmik]

Many applications require various bits from the security packages, including ZeroToaster? so they refuse to work with the current pre-alpha build. The lack of these security classes also affects the cooked JRE and JDK images -- even javac.exe refuses to work when run from the JDK image's directory tree.

The generated exceptions look like that (this is from ZeroToaster?):

Exception in thread "main" java.lang.ExceptionInInitializerError
        at java.util.jar.JarFile.initializeVerifier(JarFile.java:331)
        at java.util.jar.JarFile.getInputStream(JarFile.java:403)
        at sun.misc.URLClassPath$JarLoader$2.getInputStream(URLClassPath.java:706)
        at sun.misc.Resource.cachedInputStream(Resource.java:77)
        at sun.misc.Resource.getByteBuffer(Resource.java:160)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:266)
        at java.net.URLClassLoader.access$000(URLClassLoader.java:73)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:212)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
Caused by: java.lang.RuntimeException: Sun provider not found
        at sun.security.jca.Providers.getSunProvider(Providers.java:104)
        at sun.security.util.ManifestEntryVerifier.<clinit>(ManifestEntryVerifier.java:47)
        ... 13 more
Caused by: java.lang.ClassNotFoundException: sun/security/provider/Sun
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:186)
        at sun.security.jca.Providers.getSunProvider(Providers.java:97)
        ... 14 more
Could not find the main class: net.zerotoaster.mta.main.ZeroToaster. Program will exit.

The key class which is missing in all these cases is sun.security.provider.Sun. I will try to provide the security support (e.g. ticket 17) before closing the Core milestone. If I fail, I will search for an application that doesn't need these classes for the first OpenJDK/2 pre-pre-alpha demo (e.g. HelloWorld?.java :).

[dmik]

Fixed the production release crash in r145; that was a tough one to find too.

[dmik]

After brushing it up some more, I think we are ready to close the Core milestone. Not many real applications will work due to the temporarily disabled security stuff, but I think it's enough for the pre-pre-alpha.

Attached is the screenshot of the execution of the following Java program with OpenJDK/2:

class test extends Object
{
    public static void main (String args[])
    {
        System.getProperties().list (System.out);
    }
}