I was reading about PEG recently, and thinking “that is pretty interesting” — and of course it turns out that there is an Emacs implementation.

It is a bit odd how primitive parsing support is in Emacs. It is one of those mysteries, like how window configuration and manipulation support can be so weak. Peculiar.

CEDET includes a parser generator, called wisent. That is long overdue… though even it is a bit odd, apparently preferring a yacc-ish input syntax. I don’t know about you, but when I have a lisp system just sitting there, I reflexively reach for sexp-based formats. Well, ok, it is a port of bison. But still.

I did a little parser hacking in gdb recently. In gdb, if you complete an expression involving a field lookup, it will currently print every matching symbol in your program — when all you really wanted was the completion of a field name. This is what I set out to fix.

My first idea was: hey, the parser knows what tokens are valid. I can just ask it! But, I don’t think there’s a way to do that with bison parsers. At least, no documented way — boo. And anyway, as it turns out, this is not what you want.

For instance, consider the simple case of “p pointer->field“. This is syntactically valid as-is, so the parser would indicate that the desired completions are whatever can come next — say, an operator. But if the cursor is just after the “d”, you want to continue completing on the field name. So, you have to differentiate this case based on whitespace.

I ended up hacking the lexer as well as the parser. The lexer can now return a special COMPLETE token, which it does depending on the previous tokens and the presence or absence of whitespace. I also added some new productions like:

expression: expression '.' name COMPLETE
expression: expression '.' COMPLETE

From here it is pretty simple to solve the rest of the problem.

I don’t remember reading about this anywhere, but I’m sure it has been done before. I thought it was a pretty fun hack 🙂 – I love problems that start with the user experience and end up someplace much deeper.

An awesome color theme for Emacs.

Steve Yegge wrote a new javascript mode for Emacs. I’m interested in modes that push Emacs a little and do real parsing rather than the typical (and goofy) parse-partial-sexp stuff; this one falls into that category. Semantic also does this, though more generically and perhaps with more sophistication; I think semantic parsers are born incremental. I hope to push this mode into ELPA as soon as he responds to my patch…

While once again waiting for Gnus to contact a news server, I thought: I’ll never be able to move my RSS reading into Gnus, because the delays will skyrocket. Sure, there’s nntp//rss, but that means configuring a separate program and keeping it running — and I’ve heard that this program can have a memory footprint as big as Emacs itself. (As an aside: remember the old days when Emacs was routinely the program with the largest footprint on your desktop? For me it is never number one, and sometimes even slips to 3 or 4.)

Maybe some savior will come along and make Gnus fetch RSS feeds in the background, using a process filter. I assume, without looking, that retro-fitting this into Gnus would be very hard. For new Emacs code, though, this is the way to go; you can set things up so that most mode-specific operations report “working…” back to the user when background operations are happening — while still letting the user switch buffers and work on other things. For instance, nowadays vc-annotate works this way, which is very nice, since annotate is fairly slow in most version control systems.

Even better would be to make Emacs capable of multi-threading. Most people arrive at this idea eventually. Unfortunately, I think it is just not possible; partly due to bad language choices: dynamic scope is very handy, but having only dynamic scope is terrible; but also partly due to consequent design choices for the rest of Emacs: buffers are big global objects, maintaining compatibility for the enormous body of existing lisp is crucial, and auditing even the built-in body of elisp is, in difficulty, somewhere between daunting and impossible.

A few weeks ago I heard a funny idea in this area. Instead of trying to handle multi-threading, how about old fashioned multi-process support, with some kind of message passing? Emacs could fork(), and then the child could wander off with its own copy of everything; and then the subprocess could send up messages and data which would be integrated into the Emacs event loop. This is basically the same idea as process filters, only with the benefit that the process could be expressed in the same lisp form as the handler, and the subprocess would have access to all the relevant lisp state.

Naturally, most of these messages would just be elisp; but perhaps it would be worthwhile to add a way to transfer the contents of a buffer wholesale.

One thing I like about Emacs is that when I ask for information, the default case is that it is displayed in a section (which Emacs calls a “window”) of the window (which Emacs calls a “frame”) that I’m already focused on. What’s nice about this is that if I ask for, say, help on a symbol in C mode, the information will show up, but I don’t have to look away, or worry about a new window stealing the focus, or move the mouse around. Also, I can easily dismiss the display without moving my hands from the keyboard.

Do you know about help on symbol in C mode? I didn’t know about this until last week. If you are in a C mode buffer, you can type C-h S, aka info-lookup-symbol, to look up help for a given C symbol. This works by digging through the libc info pages… this isn’t perfect, but it is pretty handy. (And, yes, I’m one of the few people who really loves info and uses it every day. I wish all the documentation on my system were available in this format — plus one or two others, yet to be invented, which would allow better IDE-ish features in Emacs.)

On a related note, a couple days Kevin Ryde sent me the latest version of gtk-look.el for uploading into ELPA. I’ve been meaning to try this for a while, so I finally installed it. gtk-lookup-symbol is similar to info-lookup-symbol, except it uses the devhelp documentation libraries as its source. Great stuff!

Naturally, I could just use devhelp, which is a very nice tool. Unfortunately, it is a very nice tool with one major drawback — it makes a new window, with all the attendant problems. And, by default, gtk-look suffers from this problem as well; the docs show up in firefox.

Luckily, Emacs is absurdly configurable, and this problem is solved by installing the w3m-el package (on Fedora) and configuring browse-url-browser-function to display devhelp files in w3m:

(setq browse-url-browser-function

    '(("file:.*/usr/local/share/gtk-doc/html" . w3m-browse-url)

     ("file:.*/usr/share/gtk-doc/html" . w3m-browse-url)

     ("." . browse-url-firefox)))

With this I can easily view the help for gtk, glib, etc, functions in the Emacs-y way that I prefer. If this sounds good to you, you’re only seconds away from installing it via ELPA.

Now what I need is c-lookup-symbol-dwim, which searches all the databases. This shouldn’t be too hard…

Tom Fitzsimmons showed me Conkeror while I was up in Toronto this past week. It is chrome for firefox that acts like Emacs. It features Emacs-style key bindings, including M-x and prefix arguments, numbered links, and buffer-style browsing. Developer quote from the web site: “To make sure Conkeror remains pure, I do not own a mouse.”

I’ve been using this on my laptop for the last few days. It is a bit unpolished. And, I find I do still miss tabbed browsing — but hardly anything else. It is extremely simple to install… so come on, you know you want to.

I love hearing the odd details of how other people work. Usually I’m hoping to learn something that will help me be more productive, and, if not, to hear about something delightfully awful, convoluted, and obscure.

Talking to Roland at JavaOne yielded a nice mix of both. One interesting thing is that he uses full-text search on his source and basically (AIUI) doesn’t run grep from inside Emacs. I, on the other hand, was a long-time igrep user, switching to rgrep (new in Emacs 22) recently.

I decided to give Roland’s approach a try.

First I took a look at namazu, since I knew it had some sort of Emacs integration, and because it is packaged in Fedora. Unfortunately it doesn’t seem to understand how to index source code.

So then I tried GNU idutils. I had heard of this years ago, and even tried it once or twice, but somehow it didn’t end up in my repertoire. Anyway, it turns out that this is a pretty nice tool. It isn’t packaged for Fedora, but building it only took a couple minutes. The Emacs support code needed some tweaks (email me if you want this), but works ok enough out of the box.

The result is pretty nice. For small projects it may not matter much, but searches over the code I’m indexing (GCC) are much, much faster. This is especially nice since the work I’m doing on GCC at the moment requires repeated searches for all the users of various global variables.

Idutils could use a little love — right now there’s no way to incrementally update the index, which is a pain. If it affects me enough I suppose I’ll set up a cron job or something using inotify. This is just gloss, though. The important thing is that it has removed a pointless wait from my process.

This weekend I installed EMMS, the Emacs multimedia system. I also packaged it for ELPA, but I haven’t uploaded it yet — expect it soon.

EMMS is fairly primitive, but that doesn’t matter to me. I barely use any features of existing music players, I typically just play a lot of songs in a row and ignore the UI. So why move this into Emacs? Just so I can start and stop music without moving my hands to the mouse. At this point I’m almost completely living in Emacs, to the extent that I’ve started trying to migrate all my shell use there as well (changing a 15 year habit of having a couple terminals open — not easy).

I did hook EMMS to my hacked zenity (aka Emacs notification area) code. This, I suppose, goes against the “mouse-free” approach. But first, these notification area hacks are fun (and now more than half of the notification icons on my desktop are Emacs-related — haha), and second sometimes my hand is already on the mouse, and it is sometimes nice to be able to pause that way as well.

Emacs 22 is officially released, in case you didn’t hear. It has an absurd number of new features and new packages — a couple IRC clients, a new gdb UI, Tramp (a way to access files via ssh or — very useful — sudo), Calc (by the legendary Dave Gillespie), and a bunch more.

I haven’t published my “project.el” (per-project settings done in an easy way) yet, but that will come soon. I also started hacking on a keyring for Emacs, to save all those pesky Tramp and ange-ftp passwords. The base code works but I need to wire it in to all the places that ask for a password — I suppose this should really go into the Emacs core. ELPA-wise, work continues and I upload a couple packages a week. I started looking at uploading DVC — I think I want to start getting larger, more useful packages into the repository.