Today I installed Mozilla on my girlfriend’s Windows box. What a
pleasant experience that was — congratulations Mozilla hackers.
She’s already using OpenOffice, and now she will be using free
software for email and browsing. Free software has come a long way
since those days ten years ago when a Tcl-based browser seemed like a
good idea and we were all waiting for XWord to work.

Refactoring

Today’s gcjx refactoring is like the work I had done on my
foundation: necessary, but in the end there’s just the same old dirt
showing.

Last night I spent some time making the libgcj bytecode verifier
a
bit more generic. This is still preliminary, but basically the
idea is to make it so that the verifier can be plugged into other
environments. We plan to put it in the gcj front end as well, but
other VM implementors might want to write the necessary glue code to
make it work for them — this is a bit ugly, but not really very
difficult.

Sometimes you just need a short break to reset your mind a bit.
The other day I was struggling with one particular gcjx bug. I came
back to it tonight, after a grueling day hacking the Eclipse 3 build
system, and it was suddenly trivial.

So, good news. gcjx can now once again parse all of Classpath,
only this time, it is handling inner and anonymous classes more
correctly (still not perfectly…), and it survives through code
generation:

$ find /tmp/gcjx-out -name '*.class' -print | wc -l
   2391

This weekend I finished enough of anonymous and inner class
handling to check it in. gcjx is now back to generating code. There
are some more fixes to come in this area. Recently I also added a few
warnings; usually this is really simple in the new framework.

Bryce had a nice idea, which was to replace gcjh with a new gcjx
back end. This was inspired by the gcjh Miranda
method bug, which is particularly ugly because it will require a
major infrastructural improvement to gcjh. So, I started writing the
JNI header back end, which predictably is really simple. gcjx holds
the promise of being able to write class files, header files, and an
object file all from a single compilation; strange but also
convenient.

Today I helped Anthony debug a libgcj problem over irc. I sure
wish there were an easy way to hook an irc session up to an
already-running gdb so we could interactively share the debugger.
Anyone have a cool hack for this? Preferably one that has some sense
built into it so that not just anybody can hijack the debugger…

GDirect

I spent a little time yesterday working on an easy way to create
native methods for Java. I call it “GDirect”. The idea is pretty
simple, basically have a package that creates native method
implementations on the fly, based on declarations in your code. So,
e.g., you would write something like this:

public class example {
  static {
    // Second argument is base name of shared library.
    GDirect.link (example.class, "m");
  }

  public static native double atan (double x);
}

In this example, example.atan will call the C math
library function atan, nothing more is needed.

Issues

There are a few issues with this approach. One is pointers, but
with gcj we can just represent these with RawData.
Another is structures, which I’m just punting on. A third issue is
type mismatch — if the Java int and the C
int are not the same, you need translation at runtime.
So far I’m not handling this, but it is actually pretty easy to deal
with. There are some other type-mapping issues to resolve as well,
I’ll look at those more a bit later.

Since this uses JNI under the hood, it is pretty slow. With the
upcoming binary compatibility ABI in gcj, though, we can do better
there. In some situations we can completely eliminate the need for a
runtime stub and just have direct calls to the C functions from
Java.

Opportunities

This code can be made completely portable across JVMs without much
effort. It works by creating new functions on the fly with libffi,
and it registers these functions with the JVM via the JNI
RegisterNatives call. For gcj, we’ll have some CNI
equivalent of this in the not-too-distant future, so we’ll be able to
make GDirect both portable and more efficient when running on our
implementation.

libffi’s closure API is what makes this possible. It works on
most platforms, though not all yet. I think a similar technique could
be used to make bridges in other places, e.g. interfacing Java to
Mozilla or to OOo.

Another random idea is to make it simpler to wrap libraries that
already have an object-oriented interface. One way we could do this
would be to tell GDirect about a field which holds a native pointer.
Then we would treat non-static native methods specially, looking up
the native pointer at invocation time and passing it to the underlying
C API. Combined with a simple facility to rewrite method names when
linking, this would make it very easy to create and maintain, say, a
Gtk wrapper library. For instance, something like
gtk_window_set_type_hint would become:

public class GtkWindow extends ... {
  // This would actually be in a superclass somewhere.
  private RawData nativeInfo;

  static {
    GDirect.link (GtkWindow.class, "gtk", "nativeInfo",
                  GtkRewriter.getInstance ());
  }

  public native void setTypeHint (int hint);
}

Here, GtkRewriter would be a special class that would know how to
transform “GtkWindow.setTypeHint” into “gtk_window_set_type_hint”.

Tom Fitzsimmons has modified JHBuild to
build
gcj. This is pretty cool, check it out. I’m hoping to convince
him to add more packages, like GNU Crypto, to the
mix.

gcc

The tree-ssa branch was merged into
mainline today. Congratulations are in order to Diego and
everyone else who worked on this, it’s been a long time coming. Not
to be ungrateful, but now someone should write that VRP pass… 🙂

I’m still working on making anonymous, inner, and local classes
behave properly. It’s been slow going, but mostly because it is
spring and things are more social now.

Nightly builds

I finally investigated a couple nightly build problems. Now the
API comparison pages
should be working again. I still haven’t investigated the GNU Crypto
check failure or the Kawa build failure; maybe later.

At long last, a movie where the climactic scene is literally dog
versus vampire. Who writes this stuff?

I spent today reading a few threads on JavaLobby. What a sad waste of
time that was! We’ve finally found a place where the typical comment
level is even lower than on SlashDot.

As DV
points out, it sucks that some
folks don’t get it. Even, it turns out, folks who are ostensibly
in our own community.

One question that comes up from time to time is “what good is an
open Java?”. The answer to that is so obvious that we generally just
skip over it, assuming everybody has been following along the last
ten years. Apparently, they haven’t.

So, the story is, any single entity that controls access to the
source will be driven by the incentives that affect them. For
instance in the software market you often see companies moving more
toward enterprise customers, leaving the lower end to be filled by
others. The general idea here, btw, is true regardless of whether we
are talking about open or closed source — for open source the
incentives differ (and of course the consequences differ too, as we’ll
see).

What this implies is that, if you aren’t along the “incentive
path” of the gatekeeping organization, you are in trouble. This is
one reason that open source is useful — it is a way for users to
reduce their risk when adopting technologies, since they have some
ability to solve the problems they face in their “less interesting”
environment.

Another way to be screwed by non-free software is to be working on
an area that is “anti-strategic”. In this scenario, the gatekeeper
decides that supporting your situation is actively against their
interests. Then you are just completely out of luck.

Now, for Java and Linux the way this plays out is that Linux’s
higher rate of change occasionally undermines JVMs. Linux vendors
typically have a hard time shipping a solid JVM on all their supported
platforms in a timely way, especially if there have been changes in
the kernel or libc — floating stacks, SELinux, NPTL, or even more
mundane things.

Considerations like this are what drove us to actually implement
gcj-eclipse (as opposed to just dreaming about it and hacking on it in
my spare time). There simply wasn’t a way to get a working JVM on all
our target platforms on our schedule.

Contrast this with, say, any other piece of the entire operating
system. With free software, you run your own life. If you want to
build an operating system that works on S/390, or PPC64, or whatever,
you can port anything you think is important to it. It might not be
easy, etc, but at least you have the option. Case in point: for the
JVM we had to punt, but for Eclipse we put in the work to make a
64-bit port. So, there’s your value right there.

So far we haven’t really run into the “anti-strategic” scenario in
the Java-on-Linux thing, though I’ve seen various folks predict its
arrival on the assumption that Sun will turn to the IP protection
racket as their technology business slides. If that happens it will
be a nice bit of punctuation to the whole free software saga — sort
of the moral at the end explaining why tying your fate to an
organization whose needs, eventually, diverge from your own.

Dalibor
has some nice things to say on the doomsday scenario of Sun moving
into the protection racket, and some of the goofier aspects of the
SCSL and TCK licenses.

Specific Response

I read through Fleury’s comments again. I’ve debunked most of
this stuff already, I won’t bother doing it again.

gcjx

gcjx development
has been going a bit slow lately. I’m working on fixing anonymous
classes and the capture of final local variables, it is sort of slow
going.

Eclipse 3

I’ve been looking at Eclipse 3.0M8 lately. It seems pretty good
— faster, more eye candy (actually a minus for me, a plus for some I
guess), some handy new compiler knobs. Also it has a built-in code
formatting feature, which can mostly handle GNU Classpath’s style.
I’ll probably check in the style file at some point, meanwhile email
me if you want a copy.

There’s more noise from Gosling
about open-sourcing Java. You can also read about it on sys-con.com
and, of course, SlashDot.

Gosling usually dissembles a little when talking on this subject,
and some things are usually unclear, so I’m going to go point by
point. Also on SlashDot we see the usual misperceptions, but since
they are very common in the Java development community I’ll take them
on as well.

Why does IBM send open letters?

I don’t know why they send them. Gosling’s claim is that this is
“kindergarten-ish”, but we don’t really know that IBM hasn’t already
tried direct talks with no resolution. Also take a look at IBM’s
comments on most of the JSRs they participate in…

One might also reasonably ask: why doesn’t IBM just write a free
Java themselves? They already have a lot of the parts: two excellent
compilers (jikes and the one in Eclipse), the JikesRVM virtual
machine, contributions to Classpath, ICU, plus lots of other bits they
hold more closely (the J9 VM). The answer to this question can only
be that they’ve signed something with Sun preventing them from doing
so. The reality is that IBM could easily write the missing parts of
Classpath, if it liked.

BTW, for those following along, I never did get a reply from Bob
Sutor. I’m not really surprised, but it is a bit disappointing.

Doesn’t IBM already have the source?

Gosling is at his most deceptive here: “Which means that about the
only thing that they could get from liberalization is to be able to
skip testing.” To hear him say it, the JVM licensing is just like
open source, except for a testing requirement. If only this were
true. The truth is, the TCK costs money. It has never been available
to any free software implementation under an acceptable license.

As I’ve said many times before: it would be great if Sun opened
their implementation. That would be really wonderful (even though it
would probably kill literally years of my work :-). However, it
isn’t strictly needed. Instead we really just need three things to
make a Free Java:

• Access to the TCK on acceptable terms
• Involvement in the JCP without contamination or other
  unacceptable constraints
• Easing of the requirements on subsetting

Hasn’t IBM helped with Java already?

Well, sure they have. Gosling makes a big deal out of the
Swing/SWT thing, but I see that as just a technical dispute, not a
major political one. It’s a mistake to assume any company like IBM
acts with a single mind; I’m sure even in the Eclipse project there
are dissenting voices. For instance, the VEP plugin works with Swing.
And, as for IBM championing Swing and then going with SWT, which looks
more AWT-ish (but isn’t), well, we all make mistakes. So what?

In the bigger picture, I don’t care why IBM wants Java to be free
software. Their motivations are their own. Mine are the same ones
that have been my passion for nearly 14 years now: the ability to fix
bugs that affect me, to add features I think are needed, to work on
projects in cooperation with like-minded comrades, to avoid being
locked in to any one company’s view of the world. Personally I find
APIs wonderful, but even good documentation is no substitute for an
occasional (contamination-free) look underneath the hood.

If you look around at the commercialization of free software, one
thing you’ll see is that different companies come along and try to own
some critical piece of infrastructure. They want to impose a private
tax on free software, setting themselves up in a privileged position.
To my mind this is antithetical to the goals of the movement as a
whole. The point is to avoid these taxes — the infrastructure must
be free. We have to fight these wherever we find them; unfortunately
a fair part of the free Java world has fallen for Sun’s bait.

Will IBM open DB2 and WebSphere?

Gosling throws this in, in an attempt to change the subject. We
aren’t talking about DB2 or WebSphere, we’re talking about Java. For
all we know, IBM doesn’t even own all the IP in these and couldn’t
free them if it wanted to. Heck, companies have lost the
source to their programs before, maybe that happened and
they’re just too embarrassed 🙂

Developers are afraid that Java will fork

This is one of the big ones, the real reasons that we can’t blow
off and that aren’t mere spy-vs-spy stuff.

What does it mean to fork Java? Or to fork anything? I know this
sounds metaphysical and sort of stupid, but I’m serious. Suppose we
all take the Java source and just go crazy, adding features, removing
APIs, all those bad things. Well, Sun — or, hopefully, some more
neutral entity like a nonprofit set up for that purpose — will still
control the branding. Remember the flap about “gcc 2.96”? I still hear
about that from time to time (and I don’t recall ever once hearing
anything nice). That was all a branding problem, not a forking
problem per se. There’s a strong sentiment in the community that the
upstream project controls the brand, and you violate this at your
peril.

Second, the truth is, Java has already suffered most of the bad
effects of a fork. While Sun has in general done an excellent job of
keeping Java compatible, there are still places where it is not.
There is valid bytecode that will verify in some releases but not
others. All releases have bugs of various flavors. There are APIs
that never quite worked, or that are underspecified. There are even
a couple cases where Sun has violated binary compatibility by
changing constants or interfaces. Java programmers already have to
pick specific target versions and test on multiple platforms.

(Don’t get me wrong. In this area Java’s promises are, generally,
on target. When developing libgcj, we’ve noticed that most pure Java
changes are much simpler than changing target library code in other
languages, simply because the portability story is so much simpler.
You just don’t have to do as much work, and the idea of an
autoconf-like tool for Java is delightfully absurd.)

Third, another truth in the free world is that the upstream
projects usually have significant control, and the Linux vendors
generally have a strong commitment to compatibility. Sure, you do get
some horse-race effects where one vendor or another will try to put
out a release with some feature or kernel version earlier than anybody
else. However, this isn’t a problem — this is a strength! For
instance, if Red Hat puts out NPTL, or SELinux, or whatever, this
gives the community a platform on which to test and modify their
applications in advance of general acceptance. And if a feature turns
out to be a bad idea, it just disappears.

For something like Java, everybody already recognizes that one of
its strengths is its cross-platform API compatibility. It would be
dumb to hurt that, sort of like adding a gratuitously incompatible
change to Perl or Python. Why would anyone do that? The whole point
of shipping the thing is to enable your customers to run the code they
already have, or code that they get elsewhere off the net.

So, I think an open Java would in fact not fork in any
significantly harmful way. I expect we would see some minor
implementation divergences, perhaps process ports of different quality
across distributions, different default UI themes, etc. That is,
mostly trivia, and the non-trivial would be merged upstream gracefully
(assuming upstream is in fact a graceful place with which to merge —
process matters).

Maybe Sun is worried about malicious entities out there, i.e.,
Microsoft. That’s sort of an absurd worry at this point, seeing that
MS now has .NET. (Sun could have plenty of other worries too; they
don’t share them with me 🙂

Patents

To my mind, the various tos and fros about Java are not very
important. It is a bit entertaining to see Sun put out Open Source
feelers occasionally, then watch everyone get riled up over nothing.
Whee! The various Free Java efforts will get there eventually — it
might not seem like it right now, but that’s how free software works.
It never seems real until suddenly it is better than what you’re
selling.

The real danger we’re facing comes from patents. Microsoft, IBM,
and seemingly everybody else is patenting things as fast as
possible. This is a huge hole for our community, and I’m really
concerned that in a few years we’ll be left unable to ship anything
meaningful. It seems that most new technologies come with a raft of
bad patents attached.

The solution? I think we need a Free Patent Foundation, that
helps free software developers file patents, and then uses them for
the benefit of the whole community. The goal would be to get a broad
enough range of patents that we could force cross-licensing with
everybody, sort of eliminating patents as a relevant force.

That’s the dream anyway. I have no clue how to set this up.