In the previous installment we found out how to write new gdb commands in Python.  Now we’ll see how to write new functions, so your Python code can be called during expression evaluation.  This will make gdb’s command language even more powerful.

A longstanding oddity of gdb is that its command language is unreflective.  That is, there is a lot of information that gdb has that is extremely difficult to manipulate programmatically.  You can often accomplish seemingly impossible tasks if you are willing to do something very ugly: for instance, you can use “set logging” to write things to a file, then use “shell” to run sed or something over the file to produce a second file in gdb syntax, then “source” the result.  This is a pain, not to mention slow.  The Python interface removes this hackery and replaces it with what you really want anyway: an ordinary programming language with a rich set of libraries.

If you use gdb regularly, then you already know about convenience variables.  These are special variables whose names start with a dollar sign, that can be used to name values.  They are part of gdb’s state, but you can freely use them in expressions:

(gdb) set var $conv = 23
(gdb) call function ($conv)

On the python branch, we introduce convenience functions.  These use a similar syntax (in fact the same syntax: they are actually implemented as convenience variables whose type is “internal function”), and when invoked simply run a bit of Python code.  As you might expect after reading the previous post, a new function is implemented by subclassing gdb.Function.  Our example this time is the one that got me hacking on python-gdb in the first place: checking the name of the caller.

import gdb
class CallerIs (gdb.Function):
    """Return True if the calling function's name is equal to a string.
This function takes one or two arguments.
The first argument is the name of a function; if the calling function's
name is equal to this argument, this function returns True.
The optional second argument tells this function how many stack frames
to traverse to find the calling function.  The default is 1."""

    def __init__ (self):
        super (CallerIs, self).__init__ ("caller_is")

    def invoke (self, name, nframes = 1):
        frame = gdb.get_current_frame ()
        while nframes > 0:
            frame = frame.get_prev ()
            nframes = nframes - 1
        return frame.get_name () == name.string ()

CallerIs()

Let’s walk through this.

The Python docstring is used as the help string in gdb.  Convenience function documentation is shown by typing “help function” in the CLI.

Function‘s initializer takes one argument: the name of the function.  We chose “caller_is“, so the user will see this as “$caller_is“.

When the function is invoked, gdb calls the invoke method with the arguments that the user supplied.  These will all be instances of the gdb.Value class.  A Value is how a value from the debuggee (called the “inferior” in gdb terminology) is represented.  Value provides all the useful methods you might expect, many in a nicely pythonic form.  For example, a Value representing a integer can be added to Python integers, or converted to one using the built-in int function. (See the manual for a full rundown of Value.)

gdb just tries to pass all the arguments the user supplied down to the invoke method.  That is, gdb doesn’t care about the function arity — so you can do fun things, like implement varargs functions, or, as you see above, use Python’s default value feature.

This invoke method takes a function name and an optional number of frames.  Then it sees if the Nth frame’s function name matches the argument — if so, it returns true, otherwise it returns false.  You can use this to good effect in a breakpoint condition, to break only if your function was called from a certain place.  E.g.:

(gdb) break function if $caller_is("main")

(Hmm.  Perhaps we should have just let the user register any callable object, instead of requiring a particular subclass.)

Value and its as-yet-unintroduced friend, Type, are actually a bit like Python’s ctypes — however, instead of working in the current process, they work on the process being debugged.  I actually looked a little at reusing the ctypes API here, but it was not a good fit.

Next we’ll revisit commands a little; specifically, we’ll see how to write a new “set” command, and we’ll explore the “require” command.  After that we’ll go back to writing new stuff.

If you’re like me, you’ve probably wished you could write new commands in gdb.  Sure, there is the define command — which I’ve used heavily — but it has some annoying limitations.  For example, you can’t make new sub-commands using define.  Also, arguments to define are parsed oddly; you can’t have an argument that has a space in it.  Instead, the user has to do the quoting, unlike other gdb commands.

Naturally, the Python extensions to gdb make these problems trivial to solve.  In this post, I’ll show you how to implement a new command to save breakpoints to a file.  This is a feature I’ve often wanted, and which, strangely, has never been written.

I think we’ll call our new command “save breakpoints“.  Given the existence of “save-tracepoints” you might think that “save-breakpoints” is a better choice; but I never liked the former, and plus this gives me a chance to show you how to make a new prefix command.  Let’s start there.

A command written in Python is implemented by subclassing gdb.Command:

import gdb
class SavePrefixCommand (gdb.Command):
  "Prefix command for saving things."

  def __init__ (self):
    super (SavePrefixCommand, self).__init__ ("save",
                         gdb.COMMAND_SUPPORT,
                         gdb.COMPLETE_NONE, True)

SavePrefixCommand()

That is simple enough.  Command‘s init method takes the name of the command, a command class (the choices are listed in the manual), the completion method (optional; and again, documented), and an optional final argument which is True for prefix commands.

You can try the above very easily.  Save the above to a file and then source it into gdb, telling gdb that it is Python code:

(gdb) source -p /tmp/example

Now take a look:

(gdb) help save
Prefix command for saving things.

List of save subcommands:

Type "help save" followed by save subcommand name for full documentation.
Type "apropos word" to search for commands related to "word".
Command name abbreviations are allowed if unambiguous.

Note how the class documentation string turned into the gdb help.

Now we’ll write the subcommand which does the actual work.  It will take a filename as an argument, and will write a series of gdb commands to the file.  This will make it simple to restore breakpoints — just source the file.

As you might expect, many of gdb’s internal data structures have Python analogs.  We’ve seen commands, above.  Our new command will examine Breakpoint objects.  The gdb manual has documentation for most of these new classes (but not all, yet.  In the early days of this work we were quite lax … we’re still catching up).

from __future__ import with_statement
import gdb

class SaveBreakpointsCommand (gdb.Command):
    """Save the current breakpoints to a file.
This command takes a single argument, a file name.
The breakpoints can be restored using the 'source' command."""

    def __init__ (self):
        super (SaveBreakpointsCommand, self).__init__ ("save breakpoints",
                                                       gdb.COMMAND_SUPPORT,
                                                       gdb.COMPLETE_FILENAME)

    def invoke (self, arg, from_tty):
        with open (arg, 'w') as f:
            for bp in gdb.get_breakpoints ():
                print >> f, "break", bp.get_location (),
                if bp.get_thread () is not None:
                    print >> f, " thread", bp.get_thread (),
                if bp.get_condition () is not None:
                    print >> f, " if", bp.get_condition (),
                print >> f
                if not bp.is_enabled ():
                    print >> f, "disable $bpnum"
                # Note: we don't save the ignore count; there doesn't
                # seem to be much point.
                commands = bp.get_commands ()
                if commands is not None:
                    print >> f, "commands"
                    # Note that COMMANDS has a trailing newline.
                    print >> f, commands,
                    print >> f, "end"
                print >> f

SaveBreakpointsCommand ()

I think the init method is fairly straightforward, given our earlier class definition.  The name of the command is “save breakpoints” — the Command initializer will parse this properly for us.  The command takes a filename as an argument, and because there is a built-in filename completer, we simply ask gdb to use that for us.  This means that completion on the command-line will automatically do the right thing.

The invoke method is where we do the real work — this method is called by gdb when the the command is executed from the CLI.  There are two arguments passed by gdb.  The first is the argument to the command.  This is either a string, if there is an argument, or None.  The second argument is a boolean, and is True if the user typed this command at the CLI; it is False if the command was run as part of a script of some kind, for instance a .gdbinit or breakpoint commands.

The body of invoke is quite simple: we loop over all existing breakpoints, and write a representation of each one to the output file.  This is simple, because the gdb command language is easy to emit, and because the Breakpoint class provides simple access to all the attributes we care about.

Note that the precise syntax of some things here is already slated to change.  We’re going to drop the “get_” prefixes at least; and some class attributes will move from method form to “field” form. We’ll be making these changes soon, before there is too much code out there relying on the current API.

Adding new commands is easy!  A few Python-based commands come with your gdb.  You can use the new “require” command to load new commands dynamically.  To see what is available, try:

(gdb) require command <TAB>

We’re also currently shipping an “alias” command, a simple version of “pahole” (find holes in structures), a replacement for “backtrace” which we’ll explore in a later post, and maybe more by the time you read this (I’m considering committing the “save breakpoints” command).

A new command isn’t always the best way to extend the debugger.  For example, it is fairly common for an application to have a user-defined command to pretty-print a data structure; but wouldn’t it be nicer for the user if this was automatically integrated into the print command?

It would also sometimes be more convenient to hook into expression evaluation — that would make breakpoint conditions more powerful, among other things.  This is what we’ll explore in the next post.

This is the first in what I hope will be a series on using the python-enabled gdb.

We’ll start at the very beginning: checking it out, building it, and then “hello, world”.

First, install the prerequisites — aside from the normal development stuff, you will need the python development packages.  You will also need git.  If you don’t have makeinfo installed, you will want that, too.  On Fedora you can get these with:

$ sudo yum install python-devel git texinfo

That was easy!  Now we will check out and build the gdb branch.  I like to prepare for this by making a new directory where I will keep the source, build, and install trees:

$ mkdir -p ~/archer/build ~/archer/install
$ cd ~/archer

“Archer” is the name of our project, in case you’re wondering.  Note that the clone step will take a while — it is downloading the entire history of gdb.  My source tree weighs in at about 50M.

$ git clone git://sourceware.org/git/archer.git
$ cd archer
$ git checkout --track -b python origin/archer-tromey-python

Now you have archer checked out and on the right branch.  If you’re curious, other work is being done in the Archer repository.  Currently each separate project has its own “topic branch”, but we’ll be merging them together for a release.  You can see the different topics with:

$ git branch -r | grep archer

This repository is also periodically synced from gdb.  For example you could look at the in-progress multi-process work — that is, patches to let gdb debug multiple processes at the same time — by checking out the branch gdb/multiprocess-20081120-branch.

Now build gdb.  Depending on your machine, this might take a while.

$ cd ../build
$ ../archer/configure --prefix=$(cd ../install && pwd)
$ make all install

(As an aside: I have typed that goofy --prefix line a million times.  I wish configure would just do that for me.)

Now you are ready to try your python-enabled gdb.  We’ll just add the right directory to your path and then do a smoke test for now; we’ll look at more functionality in the next installment.

$ PATH=~/archer/install/bin:$PATH
$ gdb
[...]
(gdb) python print 23
23
(gdb) quit

It worked!

Once you have this set up, future updates are even simpler — and faster.  Just pull and rebuild:

$ cd ~/archer/archer
$ git pull
$ cd ../build
$ make all install

We’re actively hacking on this branch, so you may like to do this regularly.  If you find bugs, feel free to email the Archer list.  (We’ll have bugzilla working “soon”, but for the time being just fire off a note.)

I think next time we will look at writing a new gdb command in Python.  Also, we’ll try a couple of new commands, written this way, that are shipped with the Python-enabled gdb.

In the future, it will be much simpler to get this gdb.  Like I said before, I want it to be in Fedora 11 (and yeah, making the feature page is on my to-do list; I’ll try to get to it next week).  Also, I think Daniel is making a Debian experimental package for it.

Update: fixed a bug in the checkout command.  Oops.

Clearly, we need to publicize the gdb/python integration a bit more.  It is easy to get, and reasonably functional.

I’ve been thinking a bit about how to get the word out better.  What is most effective?  Here are some ideas:

• A series of “how-to” blog entries (what do you want to do?)
• A “how-to” article in a magazine (what do you read?)
• A talk at a conference (which ones?)

I’ll probably do some or all of these — but what would be most useful to you?

I’m hoping we can ship a Python-enabled gdb in F11.  Hopefully that will boost adoption.  I’m also planning to ship a suite of libstdc++ pretty-printers in F11, so even if you don’t write any Python yourself, you can still benefit.  (For those not following the progress, we have a feature that lets you write custom visualizers based on type; this makes printing a std::vector, or whatever, much simpler.)

And, by the way — Dodji is indeed awesome.  Go give him an ohloh kudo.

After working on gdb for a while, I’ve noticed a funny flaw: gdb is too good at debugging itself.

I often hear stories of disappointment from gdb users.  It doesn’t scale well.  It doesn’t handle threads well.  It only has a vague understanding of the C++ that users insist on typing at it.  That was my experience, too, when I was working on C++ (or worse: Java) programs.

But gdb itself doesn’t use any of these features.  It is written in more or less plain C.  It is single-threaded.  It is not too big.  It does not rely on many shared libraries.  So, as a gdb developer, I find it is pretty easy to forget that it has flaws.

I once heard about a C++ compiler written in C++ where, in order to counteract this same sort of problem, its developers mandated that the compiler use every existing C++ feature.

For fun try to picture code review on that project.  “Bob, this patch looks ok, but I think you should use operator overloading and exceptions here.”

That said, I think the “share the pain” theory is one reason — among several — to change gdb’s core to be implemented in C++.

Another way to combat complacency would be to debug other programs while working on gdb.  I find this pretty hard to do, though.  I tend to allocate more and more of my time to hacking my main project at the expense of the secondary one.

Some days when I am debugging, I run across a bug in gdb and think, “I wish someone would fix that already”.

Then I remember that I’m working on gdb now.

One goal I have for integrating Python scripting into gdb is to be able to fully control gdb from Python. I’ve made a few steps toward this recently. Today I can do:

#! /home/tromey/gnu/python-gdb/build/gdb/gdb --python

import sys, gdb
from hack import inferior

child = inferior(sys.argv)
if not child.run():
    gdb.execute("bt 5")
    gdb.cli()

This is a short script that is marked executable. With this, I can invoke “stop-if-broken my-program arguments“. This will run the program; if it dies for some reason, it will print a stack trace and bring up the gdb command line. Otherwise, it exits.

I like to think you can already see the future of gdb here, though. And although this is pretty young, you can easily extend it a bit — for instance, you could add breakpoints between creating the inferior and starting it. Still, don’t be totally fooled; this is still a toy and there is a lot to do on the gdb side.

The “hack” stuff is because I have no clue about intelligently managing Python packages. I’m looking for a good guide here. What I really want to do, I think, is have classes like Inferior, Breakpoint, etc, directly in the gdb namespace, but I don’t want everything in one huge file. Or is this javathink? Writing “import gdb.inferior.Inferior” feels weird. (If you’re going to give me advice — which I really want — there is an added wrinkle, which is that the gdb module is created directly in the C code in gdb.)