Quick Multi-process Debugging Update

In my last post I mentioned that setting breakpoints is a pain when debugging multiple processes in GDB. While there are some bugs here (we’re actively working on them), it isn’t hard to make the basic case work.  In fact, there’s nothing to it.  Some background…

Starting with GDB 7.4, we changed how basic breakpoint specifiers (called “linespecs”) work.  Previously, a linespec applied somewhat randomly to the first matching symbol found in your code.  This behavior probably made sense in 1989, when all you had were statically linked executables; but nowadays it is much more common to have dozens of shared libraries, with the attendant name clashes.

So, instead of having GDB guess which symbol you meant, now a breakpoint just applies to all of them.  Our idea is that we’ll start supplying ways to narrow down exactly which spots you meant to name, say by adding syntax like “break libwhatever.so:function“, or whatever.

Anyway, this new work also applies across inferiors.  Here’s an example of debugging “make“, then setting a breakpoint on a function in libcpp (which itself is linked into a sub-process of gcc):

(gdb) b _cpp_lex_direct
Function "_cpp_lex_direct" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (_cpp_lex_direct) pending.
(gdb) run
Starting program: /usr/bin/make
gcc -g -o crasher crasher.c
[New inferior 8761]
[New process 8761]
process 8761 is executing new program: /usr/bin/gcc
[New inferior 8762]
[New process 8762]
process 8762 is executing new program: /usr/libexec/gcc/x86_64-redhat-linux/4.6.2/cc1

Breakpoint 1, 0x0000000000b156a0 in _cpp_lex_direct ()

The remaining issues have to do with breakpoint re-setting not doing the right thing with running inferiors. This causes some scary warnings when running, but I think for the time being you can just ignore those.

Well, I should say those are the known issues.  This feature hasn’t had as much use as I would like (judging from the low bug rate — I can’t tell if that is a good insight or a horrible realization).  So, try it out and report problems to GDB Bugzilla.  We’ll be making it work for you.

Share if you liked it


  • Very nice!

    Can I get it to easily break on any segfaults that happen in subprocesses?

    Also, I don’t know if this already exists, but is it possible to set conditions on a breakpoint like: only break on function foo() if function bar has already been called?

  • Stopping on a SEGV will happen automatically, by default. You can control this with the “handle” command. (Eventually we’re going to add a “catch signal” command as well, so you can attach conditions and commands to signals.)

    You can use “break func if expr” or “cond” to set conditions.

    Breaking on foo-after-bar is a bit tricky. This can be done with convenience variables and commands attached to breakpoints (see the “commands” command). So, you’d initially “set var $seen_bar=0”, then set $seen_bar=1 in bar’s breakpoint, then “break foo if $seen_bar”.

    You can make this more programmable by writing convenience functions in Python. E.g., one I use sometimes is a function that looks up the stack to see what the call chain looks like: break foo if $_caller_is(“bar”)

  • Hmm, maybe I should write up some tips and tricks. There are a bunch of hacks that are kicked around by gdb developers that maybe aren’t generally known.

  • I have followed gdb blog, and I have just started implementing some pretty printers for our types at work. Very nice over all.
    But how can I reload my updated printers.py while in a debug sesssion in gdb?

  • Björn, to reload a module, I use something like

    (gdb) python reload(sys.modules[‘qt473’])

    (qt473 is my – incomplete, and buggy – set of printers for Qt 4.7.3 types)

Join the Discussion

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>