Archive for the ‘software’ Category

Frysk in the Present and Future

I read Federico’s post about finding poll() wakeups and got really excited about the idea of using frysk to do this.

Unfortunately I looked and frysk doesn’t yet have symbol table lookups, so there’s really no good way to set a breakpoint yet, or to fetch the local callback variables.

What a bummer. frysk offers a couple nice things for this problem: it is very easy to trace many programs (including through forks and multi-threaded programs) at once; this is one of its biggest advantages over gdb for this problem. It is also very easy to write a little custom code to pick exactly what you’re interested in — e.g., any program that calls poll more than 5 times per second — as if you have a scriptable strace.

I didn’t want to leave empty-handed though. So, I refactored ftrace a little and wrote my first actually-functioning frysk-using jython script:

import sys
import frysk
import java

class fdtracer(frysk.ftrace.SyscallHandler):
def handle(self, task, syscall, what):
frysk.ftrace.Util.printStackTrace(java.lang.System.out, task)

tracer = frysk.ftrace.Ftrace()
tracer.setExitHandler(fdtracer())
tracer.addTracePid(int(sys.argv[1]))
tracer.trace()

This will trace a process (identified by pid) and print a stack trace every time a system call in the process exits. (Bah, can’t figure out how to make this blog software not delete leading spaces. So much for posting python code, dammit.)

This is silly of course. Still, writing something like the bad close catcher is an obvious extension of this. And Federico’s case is not really all that far off… I want that to be a nice little script that will automatically find gnome-session, trace all its descendants, notice over-eager polling, and finally print the callback functions that are triggered.

Auto-tools Slides

I’ve uploaded the slides from my auto-tools talk at NCLUG last night. I had a great time up there, as always, though I missed Bob Proulx this time.

I thought the talk went well. It was a gentle intro to the auto-tools: what they do, how to read the common bits, where to get more information. If you want a really thorough tutorial I recommend reading Alexandre’s 550 slides.

Eclipse and the Web

Andrew’s post about hacking the spec file editor using eclipse made me remember something I wish Eclipse could do: integrate better with free software projects.

I mean, it isn’t terrible that you have to download a team spec file, or whatever. But, it seems like things could be much better in this day and age.

For instance, simply clicking on a team project set link ought to pull up the project in Eclipse. That way every project on sourceforge could have a big “Hack Me” button.

I looked and I see I’ve written about this before. I’m persistent, or obsessive, or something.

compiz

I tried compiz on my FC6t3 box today. Fun stuff! I think my favorite effect is the deformation that happens when moving windows around… don’t tell my boss but I spent a couple hours today just dragging a terminal window hither and yon. Surely, I thought, the window would tear down the center if I moved it hard enough. But in the end desktop physics proves deceptive — or windows are tougher than they look.

These effects are a great drug. Every release now seems to have some new GUI gadget that flashes or strobes or vibrates. I suspect that those of us who work on free software every day are becoming increasingly jaded, requiring ever-stronger fixes to achive the same “gee whiz high”.

Fedora Core 23 is going to be truly amazing. Windows will emerge from photorealistic 3-D forests, crouching and scowling. We will use Tarzan avatars to bend them to our wills. As their tails lash we will see progress indications, new blog posts, and stock quotes flicker into being and then disappear in the motion.

Subsequent releases will feature themes downloadable from motion picture and television studios. Angelina or Spongebob will dig through layers of rendered mud — mud which on inspection will be seen to have been constructed from the contents of our home directories — to find shining diamonds. We will peer into the diamonds and, deep within their artificial luminescence, we will behold a galaxy filled with Emacs lisp, each cons a star.

Playing with frysk

I read a thread on the valgrind list about tracking bogus close calls, and I immediately thought that this would be a good job for frysk.

So, I gave it a try. It was absurdly easy to write…

First, frysk has included ftrace in its tree for a while — this is a very simple frysk-based analog to strace. I started by taking the ftrace sources and finding the spot where a system call is printed. Since ftrace.java is all of 200 lines of code, this took about 20 seconds.

Then I changed this code to look for a call to close (another 20 seconds) and in particular a failing call (slightly longer than 20 seconds, due to a frysk oddity — more objective observers would probably note that I hadn’t read the sources…).

Well, that’s it. Unfortunately it still doesn’t work — it turns out that libunwind still can’t properly make stack traces of other processes. I understand this is under development.

That’s disappointing of course. But today, contra both my mood of late and my general disposition, I’m more inclined to focus on the positive. It is very cool that this sort of hacking is so easy to do. In the future I think it will be even simpler: not only will the various bugs be sorted out, but you’ll be able to write little scripts to drive the frysk libraries — using jython, or groovy, or kawa, or whatever suits your fancy.

This ties into a couple of things I think are interesting.

One is that, in Java, most programs double as libraries as a consequence of the design of the language and runtime. No special care is needed to make this work; for very badly written programs you can simply load them via a new class loader and let them operate in their own universe. This in turn makes it much simpler to treat existing java programs as part of a toolbox that you can assemble as you like.

And, frysk is (or for the more cautious, will be) a very useful part of this toolbox — useful in a “java-y” way, in a “script-y” way, and with various front ends, useful in a “unix-y” way, I think. Essentially it enables us to write non-trivial ad hoc debugging and tracing scripts.

JNotify

I heard about JNotify via the debian-java list today, so I downloaded it and gave it a whirl. While some of the code is a bit yucky (direct calls to syscall? Who does that?), it works quite nicely. And, in true Java style, it makes an effort to be cross-platform; a Windows port is included.

I don’t have any immediate use for this in my own hacking; too bad.

Anyway… I read about inotify a bit more today and I was pleased to discover that a watch on a directory can also yield notification about changes to files in that directory. This means that contrary to what I had heard (irc being what it is), this feature would definitely be usable for speeding up common operations in monotone. In fact, it’s a wonder that nobody has done this already.

Another area worth exploring is changing make to use inotify, to avoid all those pesky stats.

Gnome Art

Is it appropriate to say that one has “discovered” a web site? A random aside.

Yesterday I discovered art.gnome.org. Some nice background images are there, and I immediately wondered why this isn’t directly integrated into the Gnome background chooser.

Of course, once I thought this I raced forward to thinking: Gnome really ought to interact with gnome.org in many more ways, perhaps also with a “branding component” (a la Eclipse) so that distros can route users via their sites first.

What’s good for the goose

I’ve been thinking more about the opening of Java.

It is interesting how all the attention has been focused on Sun opening their implementation. For the most part this makes sense, of course — Sun is the center of the Java universe, and the first decision to open has to come from them.

But… seeing as IBM pushed so hard to get Sun to make Java open source, I think IBM not only ought to commit to opening their VM as well, but also do so on the same schedule, and with an eye toward having a single open source JVM project which spans all architectures.

Open Source Java

You’ve probably already read about Sun’s continuing effort to open source Java. And, before I get too far into what I have to say — my preliminary reaction is that this is awesome.

Sun has even made a couple nice ways to provide feedback.

I think quite frequently about the properties that a free Java implementation must have; this, after all, is the reason for the existence of Classpath and gcj. I’m going to make a stab at writing them all down here, in the hopes that this helps the efforts at Sun.

License

What license should be used?

I don’t have a particular answer to that. However, I do have a few requirements to place on the result.

First, the license absolutely must be open in the DSFG sense. This probably goes without saying, but I figured I’d be explicit.

Second, ideally the license would be GPL compatible. It doesn’t make sense to me to limit innovation by closing off the JDK from mixing with an entire class of software. Also, a move like this would be an important victory in the battle against “license silos” and for license harmonization — important because, let’s face it, the opening of the JDK will be a truly significant open source event, the most important one in years.

If GPL compatibility somehow cannot be achieved, LGPL compatibility is still required. The reason for this is that a reasonable chunk of the platform (e.g., all the gnome-ish GUI libraries) are LGPL. This is a much lower bar to meet, I’m sure there won’t be a problem here.

Likewise ASL compatibility is a must.

Geir claims that the GPL would be a bad choice. From my perspective it would not be. True, Classpath isn’t pure GPL — but that is largely an accident of history, based on the embedded systems origin of gcj. My impression now is that with the advent of embeddded Linux, the embedded world has largely crossed the GPL barrier. Beyond that, my focus these days is really on the free operating systems like Linux; GPL would present no problems for, say, Fedora.

One important consideration in the license is what to do about patents. This isn’t really my area; I think patents are pretty bad and so I have a huge blank spot in my brain about them, a spot which is filled with a mixture of denial, anxiety, and loathing. The key is to make sure that the result is acceptable to the various players — corporations and also nonprofits like Debian or Fedora. This shouldn’t be hard, plenty of other organizations already have done it.

Compatibility

Sun worries about compatibility. I hope those fears can slowly be put to rest. At least here in the free world, we recognize that compatibility is very important. In fact, we put an absurd amount of effort into guaranteeing compatibility in Classpath. The whole point, for us, is to make it possible to run the enormous body of existing Java code on free OSs.

Ideally the TCK would also be opened as part of Java. I picture Fedora’s Sun-based Java RPM working much the same way as the GCC RPM: we would run the TCK as part of the build process, and die if it does not pass. This would provide critical quality assurance.

Note that rules requiring testing of binaries by some 3rd party won’t work too well. For Fedora and the like, the ability to build from source is crucial. This is especially true as the kernel and glibc hackers continue to innovate — over the years changes in these areas have caused problems for the various proprietary JVMs. (And, supporting this whole opening process: had the JVMs been opened, we would simply have fixed them. But due to their closed nature we either required workarounds, or simply lived with the brokenness.)

Sharing

We’ve heard that not all of the code will be available under an open-source license. It would be great to get more details on what, precisely, won’t be available.

The sooner we get feedback on this, the better. This seems like a great area for collaboration to occur. I can think of a couple ways to make it happen.

First, if the problem areas are already implemented in Classpath, we could open a discussion on contributing them. RMS would most likely have to be involved — but, really, his reputation is much worse than the reality. I’d support cooperation along these lines (assuming of course that the baseline is met properly: an acceptable license, etc). There are probably a few interesting things in Classpath that would benefit the JDK.

Second, if the areas in question aren’t covered by Classpath, maybe we could put a little focus on them in order to have the code available when the JDK is opened. This needn’t be a Classpath-driven effort — I’m sure many people out there would be interested in this same result.

Governance

There are plenty of well-run free software projects out there. Choosing a functioning governance model isn’t that big of a deal any more — heck, folks at Sun can just walk across campus and ask people working on any of their other free software projects. I’m just not concerned about this; the biggest thing is to have a committment to success, so that if something isn’t working, you change it.

That said, there are a couple things worth noting.

The JCP could be more open. Both Dalibor and Geir have covered this point well. Listen to them!

I’ve read the occasional blog entry worrying about simply opening the process to anybody. I agree with Simon here — anarchy is no way to run an open source project. I can’t think of a single successful one that is run that way; instead, developers have to earn their stripes by a process of patch submission and review. I don’t think Java should be any different; if anything, the barrier to entry ought to be a bit higher than ordinary, since Java is a large, mature, and (presumably 🙂 stable code base. We certainly vet people before giving them access to Classpath, and we also watch them more carefully just after they’ve been given commit rights.

Final Advice

You’ll probably see a lot of random, uninformed, and hostile commentary on slashdot and javalobby. At least, that’s been my experience historically — “open source java” is a big red flag for the bulls. My advice is, ignore the crap and concentrate on pushing forward.

Fedora Core 6 (test 2)

Last week I installed FC6 test 2 on my laptop. I did an upgrade (using CDs) from my FC5 install.

The install itself went very well. Anaconda weirdly showed strange characters while in text mode, but this was ignorable.

FC6 is shipping a new Eclipse — 3.2. This works pretty well; I found a bug or two and reported them, but nothing too serious. So far I haven’t explored many of the new features, but I did notice that I can now background “cvs diff” operations. I’ve been wanting that for a while…

The notification area on the panel is now properly translucent. For some reason its opacity really bothered me, and this is one of the few Gnome bugs I CCd myself on.

I installed yum-updatesd so that I could run puplet. So far it seems nice, though I have to say I preferred the old RHN icon (I’m probably the only person who liked that :-). But, icons always change; I’m very happy to have the functionality. I’m not too good at remembering to periodically “yum update”, you see.

In somewhat related news, I’ve been trying the Gnome sticky notes applet for managing my to-do list. We’ll see how that works out; over the years I’ve tried many different programs in this space, with little success.

Overall FC6t2 looks quite good to me. I’m constantly amazed that this whole process (not Fedora, but the entire free software setup) produces good results — and yet it does, year after year.