4. KGDB

KGDB is kernel dark magic that allows you to GDB the kernel on real hardware without any extra hardware support.

It is useless with QEMU since we already have full system visibility with -gdb. So the goal of this setup is just to prepare you for what to expect when you will be in the treches of real hardware.

KGDB is cheaper than JTAG (free) and easier to setup (all you need is serial), but with less visibility as it depends on the kernel working, so e.g.: dies on panic, does not see boot sequence.

First run the kernel with:

./run --kgdb

this passes the following options on the kernel CLI:

kgdbwait kgdboc=ttyS1,115200

kgdbwait tells the kernel to wait for KGDB to connect.

So the kernel sets things up enough for KGDB to start working, and then boot pauses waiting for connection:

<6>[    4.866050] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
<6>[    4.893205] 00:05: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
<6>[    4.916271] 00:06: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A
<6>[    4.987771] KGDB: Registered I/O driver kgdboc
<2>[    4.996053] KGDB: Waiting for connection from remote gdb...

Entering kdb (current=0x(____ptrval____), pid 1) on processor 0 due to Keyboard Entry
[0]kdb>

KGDB expects the connection at ttyS1, our second serial port after ttyS0 which contains the terminal.

The last line is the KDB prompt, and is covered at: Section 4.3, “KDB”. Typing now shows nothing because that prompt is expecting input from ttyS1.

Instead, we connect to the serial port ttyS1 with GDB:

./run-gdb --kgdb --no-continue

Once GDB connects, it is left inside the function kgdb_breakpoint.

So now we can set breakpoints and continue as usual.

For example, in GDB:

continue

Then in QEMU:

./count.sh &
./kgdb.sh

rootfs_overlay/lkmc/kgdb.sh pauses the kernel for KGDB, and gives control back to GDB.

And now in GDB we do the usual:

break __x64_sys_write
continue
continue
continue
continue

And now you can count from KGDB!

If you do: break __x64_sys_write immediately after ./run-gdb --kgdb, it fails with KGDB: BP remove failed: <address>. I think this is because it would break too early on the boot sequence, and KGDB is not yet ready.

See also: