Java Plugin for mozilla

[Silvan Scherrer]

we also need a plugin for mozilla/firefox

[Yoda_Java6]

Is it really the case, that Java 1.6 GA will not include a Moz plugin ?

This ticket says that it is scheduled for 'Enhanced' milestone,
and Enhanced milestone still seems to not scheduled at all.

I think everyone would expect/hope, that Moz plugin would be
part of the GA

[Silvan Scherrer]

of course we try to bring the plugin also for GA. but we will not hold back a GA because this feature is not done.

[Yoda_Java6]

(seems I've never got email about your reply, diver - strange).

In my world, Java plugin is most like the most used JAVA situation for
all eCS users - it will most likely only be a minority, that actually runs
real java apps stand alone.

Coming out with Java 6 without a JAVA plugin for MOZ apps, looks
like half an app to me - not a GA product.

At the same time, OJI plugin API in MOZ apps is gone !

Today it requires a new API, which doesn't exist in eCS ports of MOZ

Has this situation been fully dealt with ?

[dmik]

The good news is that the ​IcedTea plugin (OSS Java plugin implementation) is now the default one in Ubuntu and it seems to work fine in many cases. At least, the standard Java applet demos work and the applets from this collection ​http://www.falstad.com/mathphysics.html that I tried work too.

So, porting IcedTea to OS/2 will be the first thing that we will try.

[dmik]

The sources are checked in. Sorting out dependencies (mainly mozilla and glib).

The code uses pthreads, I think I will port it to native OS/2 threads (the code is quite simple).

The code also uses GTK (probably, for the settings dialog), I have to think on what to do with that. Maybe replace with Qt.

[dmik]

The IcedTea plugin requires a ​patch to OpenJDK6/7 that makes sun.applet.AppletViewerPanel public. I've applied this patch in r355.

[dmik]

I found that the RPM build of glib lacks some functionality. In particular, the following functions are missing:

• g_spawn_async
• g_io_channel_new_file

Also, for some strange reason clock_gettime() is defined by kLIBC headers but its implementation is actually missing.

This needs to be sorted out.

Besides that, the plugin C++ code is ported to OS/2 now.

[dmik]

Okay, the fixed build of glib is done (see ​http://svn.netlabs.org/ports/ticket/14 for more info). Now all glib functions can be resolved.

The next is to fix some more bits in the plug-in itself.

[dmik]

The plug-in DLL now builds successfully. I will need to add some resource strings so that it's recognized by Firefox (will borrow this from Flash) and also build the Java part of the plugin (should be easy). Now I will be able to try it.

[dmik]

I found that some Java classes (accompanying the plugin) need to be ported to OS/2 (WinOS2), e.g. sun.appliet.PluginViewer. Currently, this class extends sun.awt.X11.XEmbeddedFrame while this class is obviously missing from our JDK build. The correct class is sun.awt.windows.WEmbeddedFrame.

[dmik]

A bunch of improvements:

• Made Java code build nicely depending on the platform.
• Made the plug-in to be recognized by Firefox (comment:13).
• Fixed logging.

Now I see that g_spawn_async() (from glib) doesn't actually work. Need to debug it.

[dmik]

I've added the native g_spawn implementation to glib but now faced another problem: due to some changes in libc 0.6.5, spawn() now tries to append some extra "kLIBC" argument to the executable and this breaks java.exe (which doesn't expect this). Needs to be investigated.

[dmik]

Okay, the problem will be seen in all WinOS2 apps using the GetCommandLine() API because it bypasses LIBC. I need to think on how to solve that.

[dmik]

Now I've run into some other problem: Firefox crashes in GLIB2 if the process can be successfully started. Debugging.

[dmik]

Ok, problem solved. It was actually two problems (glib, see r 499 in Unix Ports) and the plugin itself (see r366).

[dmik]

Debugging glib pipes now. They don't seem to work as desgined. The icedtea plugin uses them for communicating with the java process it starts.

[dmik]

The plugin uses Unix pipes which are special objects in the file system to communicate to the Java process. Since this concept doesn't exist in OS/2, we will have to use a different approach. We will have to change the Java code as well (it currently assumes pipe file names on input and uses them in the java.io.File constructor but this is meaningless on OS/2).

Named pipes are currently created with mkfifo() which is a no-op in kLIBC currently.

[dmik]

A short report. I decided to use unnamed pipes and pass file descriptors to the child instead of file (pipe) names. This required some minor changes in both the C and Java part of the plugin. I faced various issues in glib (in particular incorrect SIGPIPE processing) which I had also to fix.

Now I'm having some issues with pthread_cancel()/pthread_join(). The latter hangs even if the former was called before it. Investigating.

[dmik]

Turned out that the whole cancellation machinery (cancellation points, cleanup routines, etc.) is missing from the OS/2 implementation of pthreads by Yuri Dario that we have. Since implementing it all would require a bit of effort, I went another way: I changed the plugin code so that it doesn't need pthread_cancel() to stop worker threads. See r369.

[dmik]

Finally, got the Java part of the plugin running.

Now I have some problems with path separators. It expects Unix paths and fails. Fixing.

[dmik]

The separator issues (and some other) are fixed in r372.

Now I got problems with reading from pipes in the Java code. Looks like the file descriptor doesn't reach the Java stream read implementation. Will debug that.

[dmik]

I found why the file descriptor doesn't reach the implementation. On Windows, there is a mess inside the java.io.FileDescriptor implementation since it encapsulates both the file handle (a HANDLE, returned by e.g. the CreateFile() Windows API) and the file descriptor (returned by the LIBC open() API). And it seems that FileInputStream/FileOutputStream implementations only use the HANDLE version on Windows. While in order to implement pipe I/O in the plugin we need to operate on file descriptors.

I need to modify the Java code so that it can use both the HANDLE and LIBC file descriptors.

[dmik]

Turned out that the simplest way is to modify the plugin code. However, still some problems with pipes. I need to write a simple Java test case as the code in case of the plugin is too complex.

[dmik]

To clarify my last comment, not plugin code but Java code. I.e. use different functions to read/write depending on whether it is given a file descriptor or a HANDLE. It seems to work but for pipes it returns a failure when reading saying it's a bad descriptor.

[dmik]

Glib does something with file descriptors and that's why they don't work (a raw spawn call works fine in the testcase). Investigating.

[dmik]

r373 contains the fixed Java code that works with pipe descriptors.

Now the communication between the plugin and the stated JVM process works. The plugin runs much farther but I get a Java exception inside sun.awt.windows.WComponentPeer.checkCreation() when it attempts to create an embedded frame inside the Mozilla window. Perhaps that's because the plugin getsthe OS/2 HWND while the Java code expects the Windows (Odin) HWND. We need to proxy it. AFAIR, Odin already contains the necessary code.

[dmik]

I did proxying (r377) but having some redraw problems (clipping seems to be wrong so that I don't get anything). Investigating.

[dmik]

I also get an "Read error (Lock violation)" IOException when reading some property files. Another Unix vs DOS file IO issue perhaps. Will check that too as it seems to break the applet initialization process.

[dmik]

Found another problem. The plugin downloads .jar files to the cache and then uses them during execution. For some reason, some .jar files can't be downloaded (I get zero length files). Probably another file issue.

[dmik]

JFTR. Downloading hangs because some socket operation is hanging indefinitely. It only happens whithin the Mozilla environment, e.g. if I try the same code from a simple Java testcase it works. Working on this problem ATM.

[dmik]

Found the cause. The hung socket was a false path. The real reason is GLIB again: the plugin uses g_io_add_watch() to watch for the data from the appletviewer (Java part) in the input pipe but currently we use the Unix implementation which uses poll()/select() for watching. This, in turn, requires to run the main GLIB event loop in order to get the notifications processed. However, Mozilla on OS/2 does not run the GLIB event loop (as opposed to the Linux version) and therefore nobody processes the notifications so far.

We will have to use the PM message loop it seems.

[dmik]

A proper way would be to fix glib so that it integrates select() and the PM message queue under the hood but it's too much for a quick fix. I'm trying a different approach to save time: run the glib main event loop on an additional thread. I faced the known issue that select() in kLIBC only works for sockets, basically. And if there are also normal file handles in the array, select() fails with EINVAL. I need to find out what else the plugin is trying to select() on.

[dmik]

Found and fixes the select() failure: it was pipe() again (in glib itself). Changed it to socket_pair() too (r502 in Unix ports).

[dmik]

I added some code to GLib that performs integration of the GLib and PM message queues and I finally got the Java plugin working (see the attachment). Hooray!

[dmik]

First screen shot ever of th Java 6 Firefox plugin (IcedTea) for OS/2 in action

[dmik]

Committed GLib fixes in r 503 (Unix ports). The relevant plugin changes are to be committed later as they need some brush up.

[dmik]

The plugin changes are in r383.

The next step is to fix the problem with terminating the Java process started on behalf of the plugin. It now remains running after Firefox terminates.

What we also need to fix is the version string our Java reports. As you see on the screen shot, the test applet cannot properly detect the Update number.

[dmik]

The JAVA process hang is fixed in r384.

Now I will check #161. And we should also do #167 within GA3.

[Yoda_Java6]

I think #133 is the same issue as #161.
Both problems started with GCC builds of Odin.

[dmik]

Got a small problem with file locking in Java. For some reason if a region of the file is locked using java.file.nio.channels.FileChannel.lock(), it turns out to be impossible to write to that file from the SAME process (which is a nonsense). I will try to solve that now and if it requires too much, I will disable file locking in the plugin code.

[dmik]

The problem with locking is that it seems to cause the plugin to screw up sometimes due to a huge amount of assertions during read/write of cache data files (Lock violation exception).

[dmik]

in r392, I changed the logic of defining the default paths for the plugin. Now, if ICEDTEA_WEB_JRE is not set in the environment, JAVA.EXE is searched in PATH. If ICEDTEA_WEB_DATA is not set, the netx.jar and plugin.jar are searched in a subdirectory "npicedt" located in the same directory where NPICEDT.DLL is loaded by Firefox from. This makes it possible to set up the plugin w/o the need to set any additional environment variables.

Later, when RPM packages are ready, we will change the default paths to predefined locations.

[dmik]

The problem with the Java version number from comment:41 is fixed in r393. The new version number scheme is:

java.version (normal version): 1.6.0_BB
java.runtime.version (full version): 1.6.0_BB-bNN[-word]

where BB is the upstream source bundle release/build (e.g. the latest one is 25, this number is also used as the marketing Update number if I got it right), NN is our serial build number (to distinguish several OS/2 releases of the same upstream release) and word is some symbolic name for our build, e.g. GA3. So for the GA3 milestone it will become:

java.version (normal version): 1.6.0_25
java.runtime.version (full version): 1.6.0_25-b01-GA3

Note that this is a small ambiguity since the OpenJDK site calls its release numbers "builds" (and shortens it to b25 if the build number is 25) which interferes with our serial build numbers. But this is how these numbers are designed to be used (my understanding after checking what Java reports on Linux and Mac OS) so I'm going to do the same on OS/2.

[dmik]

Hmm, there is one more additional problem. With my fixes to GLib needed for the plugin, yum install something crashes with the following:

(process:1405): GLib-CRITICAL **: g_timer_stop: assertion `timer != NULL' failed

(process:1405): GLib-CRITICAL **: g_timer_destroy: assertion `timer != NULL' failed
Traceback (most recent call last):
  File "C:\USR\BIN\YUM", line 29, in <module>
    yummain.user_main(sys.argv[1:], exit_code=True)
  File "/@unixroot/usr/share/yum-cli/yummain.py", line 254, in user_main
    errcode = main(args)
  File "/@unixroot/usr/share/yum-cli/yummain.py", line 109, in main
    result, resultmsgs = base.doCommands()
  File "/@unixroot/usr/share/yum-cli/cli.py", line 352, in doCommands
    self._getTs(needTsRemove)
  File "/@unixroot/usr/lib/python2.6/site-packages/yum/depsolve.py", line 99, in _getTs
    self._getTsInfo(remove_only)
  File "/@unixroot/usr/lib/python2.6/site-packages/yum/depsolve.py", line 110, in _getTsInfo
    pkgSack = self.pkgSack
  File "/@unixroot/usr/lib/python2.6/site-packages/yum/__init__.py", line 777, in <lambda>
    pkgSack = property(fget=lambda self: self._getSacks(),
  File "/@unixroot/usr/lib/python2.6/site-packages/yum/__init__.py", line 567, in _getSacks
    self.repos.populateSack(which=repos)
  File "/@unixroot/usr/lib/python2.6/site-packages/yum/repos.py", line 279, in populateSack
    sack.populate(repo, mdtype, callback, cacheonly)
  File "/@unixroot/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 187, in populate
    dobj = repo_cache_function(xml, csum)
  File "/@unixroot/usr/lib/python2.6/site-packages/sqlitecachec.py", line 46, in getPrimary
    self.repoid))
TypeError: Can not open SQL database: unable to open database file

This needs to be fixed too...

[dmik]

BTW, in r395 I made setting BEGINLIBPATH (or even LIBPATH) for Java not necessary at all: in order to use Java you only need to make sure that the original directory structure is preserved (e.g. Java DLLs are in bin/ and the JVM DLL is in bin/<type>/).

This is achieved by performing the necessary BEGINLIBPATH manipulation from the JVM.DLL's _DLL_InitTerm() procedure -- since JVM.DLL is the first DLL loaded (dynamically) when starting any Java task, it covers all the cases.

This is very handy and e.g. removes the requirement of the correct BEGINLIBPATH setting when using Java from other aplications (like OpenOffice).

[dmik]

Regarding the locking problem, I created #169 for it. For now, I simply disable the existing locking scheme because properly solving it would take some time while for beta testing it's not very important (chances are low that any conflict occurs unless you make a heavy use of the plugin with a bunch of browser windows running java applets in parallel). See r396.

[dmik]

This is considered to be done. All further problems should be reported as separate tickets with the component field set to "icedtea-web".