Linux Kernel Module Cheat

Being the hardcore person who fully understands an important complex system such as a computer, it does have a nice ring to it doesn’t it?

But before you dedicate your life to this nonsense, do consider the following points:

Are you fine with those points, and ready to continue wasting your life with this crap?

Good. In that case, read on, and let’s have some fun together ;-)

Related: Soft topics.

One of the major features of this repository is that we try to support the --dry-run option really well for all scripts.

This option, as the name suggests, outputs the external commands that would be run (or more precisely: equivalent commands), without actually running them.

This allows you to just clone this repository and get full working commands to integrate into your project, without having to build or use this setup further!

For example, we can obtain a QEMU run for the file userland/c/hello.c in User mode simulation by adding --dry-run to the normal command:

which as of LKMC a18f28e263c91362519ef550150b5c9d75fa3679 + 1 outputs:

So observe that the command contains:

For the specific case of running emulators such as QEMU, the last command is also automatically placed in a file for your convenience and later inspection:

Since we need this so often, the last run command is also stored for convenience at:

although this won’t of course work well for Simultaneous runs.

Furthermore, --dry-run also automatically specifies, in valid Bash shell syntax:

This repository has been tested inside clean Docker containers.

This is a good option if you are on a Linux host, but the native setup failed due to your weird host distribution, and you have better things to do with your life than to debug it. See also: Section 38.1, “Supported hosts”.

For example, to do a QEMU Buildroot setup inside Docker, run:

You are now left inside a shell in the Docker! From there, just run as usual:

The host git top level directory is mounted inside the guest with a Docker volume, which means for example that you can use your host’s GUI text editor directly on the files. Just don’t forget that if you nuke that directory on the guest, then it gets nuked on the host as well!

Command breakdown:

To use GDB step debug from inside Docker, you need a second shell inside the container. You can either do that from another shell with:

or even better, by starting a tmux session inside the container. We install tmux by default in the container.

You can also start a second shell and run a command in it at the same time with:

To use QEMU graphic mode from Docker, run:

and then on host:

TODO make files created inside Docker be owned by the current user in host instead of root:

THIS IS DANGEROUS (AND FUN), YOU HAVE BEEN WARNED

This method runs the kernel modules directly on your host computer without a VM, and saves you the compilation time and disk usage of the virtual machine method.

It has however severe limitations:

Still interested?

Compilation will likely fail for some modules because of kernel or toolchain differences that we can’t control on the host.

The best workaround is to compile just your modules with:

which is equivalent to:

Or just remove the .c extension from the failing files and try again:

Once you manage to compile, and have come to terms with the fact that this may blow up your host, try it out with:

You don’t need to depend on GitHub.

For a quick and dirty build, install Asciidoctor however you like and build:

For development, you will want to do a more controlled build with extra error checking as follows.

For the initial build do:

which also downloads build dependencies.

Then the following times just to the faster:

Source: build-doc

The HTML output is located at:

More information about our documentation internals can be found at: Section 38.5, “Documentation”

--gdb-wait makes QEMU and gem5 wait for a GDB connection, otherwise we could accidentally go past the point we want to break at:

Say you want to break at start_kernel. So on another shell:

or at a given line:

Now QEMU will stop there, and you can use the normal GDB commands:

See also:

Let’s observe the kernel write system call as it reacts to some userland actions.

Start QEMU with just:

and after boot inside a shell run:

which counts to infinity to stdout. Source: rootfs_overlay/lkmc/count.sh.

Then in another shell, run:

and then hit:

And you now control the counting on the first shell from GDB!

Before v4.17, the symbol name was just sys_write, the change happened at d5a00528b58cdb2c71206e18bd021e34c4eab878. As of Linux v 4.19, the function is called sys_write in arm, and __arm64_sys_write in aarch64. One good way to find it if the name changes again is to try:

or just have a quick look at the sources!

When you hit Ctrl-C, if we happen to be inside kernel code at that point, which is very likely if there are no heavy background tasks waiting, and we are just waiting on a sleep type system call of the command prompt, we can already see the source for the random place inside the kernel where we stopped.

tmux just makes things even more fun by allowing us to see both the terminal for:

at once without dragging windows around!

First start tmux with:

Now that you are inside a shell inside tmux, you can start GDB simply with:

which is just a convenient shortcut for:

This splits the terminal into two panes:

and focuses on the GDB pane.

Now you can navigate with the usual tmux shortcuts:

See the tmux manual for further details:

To start again, switch back to the QEMU pane with Ctrl-O, kill the emulator, and re-run:

This automatically clears the GDB pane, and starts a new one.

The option --tmux-args determines which options will be passed to the program running on the second tmux pane, and is equivalent to:

This is equivalent to:

Due to Python’s CLI parsing quicks, if the run-gdb arguments start with a dash -, you have to use the = sign, e.g. to GDB step debug early boot:

Bibliography: https://unix.stackexchange.com/questions/152738/how-to-split-a-new-window-and-run-a-command-in-this-new-window-using-tmux/432111#432111

https://stackoverflow.com/questions/28607538/how-to-debug-linux-kernel-modules-with-qemu/44095831#44095831

Loadable kernel modules are a bit trickier since the kernel can place them at different memory locations depending on load order.

So we cannot set the breakpoints before insmod.

However, the Linux kernel GDB scripts offer the lx-symbols command, which takes care of that beautifully for us.

Shell 1:

Wait for the boot to end and run:

Source: kernel_modules/timer.c.

This prints a message to dmesg every second.

Shell 2:

In GDB, hit Ctrl-C, and note how it says:

That’s lx-symbols working! Now simply:

and we now control the callback from GDB!

Just don’t forget to remove your breakpoints after rmmod, or they will point to stale memory locations.

TODO: why does break work_func for insmod kthread.ko not very well? Sometimes it breaks but not others.

TODO successfully debug the very first instruction that the Linux kernel runs, before start_kernel!

Break at the very first instruction executed by QEMU:

Note however that early boot parts appear to be relocated in memory somehow, and therefore:

Further discussion at: Linux kernel entry point.

In the specific case of gem5 aarch64 at least:

QEMU’s -gdb GDB breakpoints are set on virtual addresses, so you can in theory debug userland processes as well.

You will generally want to use gdbserver for this as it is more reliable, but this method can overcome the following limitations of gdbserver:

Known limitations of direct userland debugging:

GDB can call functions as explained at: https://stackoverflow.com/questions/1354731/how-to-evaluate-functions-in-gdb

However this is failing for us:

E.g.: if we break on __x64_sys_write on count.sh:

even though fdget_pos is the first thing __x64_sys_write does:

I also noticed that I get the same error:

when trying to use:

on many (all?) functions.

See also: https://github.com/cirosantilli/linux-kernel-module-cheat/issues/19

info all-registers shows some of them.

The implementation is described at: https://stackoverflow.com/questions/46415059/how-to-observe-aarch64-system-registers-in-qemu/53043044#53043044

For a more minimal baremetal multicore setup, see: Section 33.10.3, “ARM baremetal multicore”.

We can set and get which cores the Linux kernel allows a program to run on with sched_getaffinity and sched_setaffinity:

Source: userland/linux/sched_getaffinity.c

Sample output:

Which shows us that:

The number of cores is modified as explained at: Section 24.3.1, “Number of cores”

taskset from the util-linux package sets the initial core affinity of a program:

output:

so we see that the affinity was restricted to the second core from the start.

Let’s do a QEMU observation to justify this example being in the repository with userland breakpoints.

We will run our ./linux/sched_getaffinity.out infinitely many times, on core 0 and core 1 alternatively:

on another shell:

Then, inside GDB:

and we observe that info threads shows the actual correct core on which the process was restricted to run by taskset!

We should also try it out with kernel modules: https://stackoverflow.com/questions/28347876/set-cpu-affinity-on-a-loadable-linux-kernel-module

TODO we then tried:

and:

to switch between two simultaneous live threads with different affinities, it just didn’t break on our threads:

Note that secondary cores in gem5 are kind of broken however: gem5 GDB step debug secondary cores.

Bibliography:

We source the Linux kernel GDB scripts by default for lx-symbols, but they also contains some other goodies worth looking into.

Those scripts basically parse some in-kernel data structures to offer greater visibility with GDB.

All defined commands are prefixed by lx-, so to get a full list just try to tab complete that.

There aren’t as many as I’d like, and the ones that do exist are pretty self explanatory, but let’s give a few examples.

Show dmesg:

Show the Kernel command line parameters:

Dump the device tree to a fdtdump.dtb file in the current directory:

List inserted kernel modules:

Sample output:

Bibliography:

For when it breaks again, or you want to add a new feature!

See also: https://stackoverflow.com/questions/13496389/gdb-remote-protocol-how-to-analyse-packets

TODO: we would need a second serial for KGDB to work, but it is not currently supported on arm and aarch64 with -M virt that we use: https://unix.stackexchange.com/questions/479085/can-qemu-m-virt-on-arm-aarch64-have-multiple-serial-ttys-like-such-as-pl011-t/479340#479340

One possible workaround for this would be to use KDB ARM.

Main more generic question: https://stackoverflow.com/questions/14155577/how-to-use-kgdb-on-arm

Just works as you would expect:

In GDB:

and you now control the count.

KDB is a way to use KDB directly in your main console, without GDB.

Advantage over KGDB: you can do everything in one serial. This can actually be important if you only have one serial for both shell and .

Disadvantage: not as much functionality as GDB, especially when you use Python scripts. Notably, TODO confirm you can’t see the the kernel source code and line step as from GDB, since the kernel source is not available on guest (ah, if only debugging information supported full source, or if the kernel had a crazy mechanism to embed it).

Run QEMU as:

This passes kgdboc=ttyS0 to the Linux CLI, therefore using our main console. Then QEMU:

And now the kdb> prompt is responsive because it is listening to the main console.

After boot finishes, run the usual:

And you are back in KDB. Now you can count with:

And you will break whenever __x64_sys_write is hit.

You can get see further commands with:

The other KDB commands allow you to step instructions, view memory, registers and some higher level kernel runtime data similar to the superior GDB Python scripts.

Analogous to GDB step debug userland processes:

on host you need:

Our setup gives you the rare opportunity to step debug libc and other system libraries.

For example in the guest:

Then on host:

and inside GDB:

And you are now left inside the sleep function of our default libc implementation uclibc libc/unistd/sleep.c!

You can also step into the sleep call:

This is made possible by the GDB command that we use by default:

which automatically finds unstripped shared libraries on the host for us.

See also: https://stackoverflow.com/questions/8611194/debugging-shared-libraries-with-gdbserver/45252113#45252113

TODO: try to step debug the dynamic loader. Would be even easier if starti is available: https://stackoverflow.com/questions/10483544/stopping-at-the-first-machine-code-instruction-in-gdb

Bibliography: https://stackoverflow.com/questions/20114565/gdb-step-into-dynamic-linkerld-so-code

We used to "support" it until f8c0502bb2680f2dbe7c1f3d7958f60265347005 (it booted) but dropped since one was testing it often.

If you want to revive and maintain it, send a pull request.

It should not be too hard to port this repository to any architecture that Buildroot supports. Pull requests are welcome.

To have more control over the system, you can replace BusyBox’s init with your own.

The most direct way to replace init with our own is to just use the init= command line parameter directly:

This just counts every second forever and does not give you a shell.

This method is not very flexible however, as it is hard to reliably pass multiple commands and command line arguments to the init with it, as explained at: Section 7.4, “Init environment”.

For this reason, we have created a more robust helper method with the --eval option:

It is basically a shortcut for:

Source: rootfs_overlay/lkmc/eval_base64.sh.

This allows quoting and newlines by base64 encoding on host, and decoding on guest, see: Section 17.3.1, “Kernel command line parameters escaping”.

It also automatically chooses between init= and rcinit= for you, see: Section 7.3, “Path to init”

--eval replaces BusyBox' init completely, which makes things more minimal, but also has has the following consequences:

The best way to overcome those limitations is to use: Section 7.2, “Run command at the end of BusyBox init”

If the script is large, you can add it to a gitignored file and pass that to --eval as in:

or add it to a file to the root filesystem guest and rebuild:

Remember that if your init returns, the kernel will panic, there are just two non-panic possibilities:

Use the --eval-after option is for you rely on something that BusyBox' init set up for you like /etc/fstab:

After the commands run, you are left on an interactive shell.

The above command is basically equivalent to:

where the lkmc_eval option gets evaled by our default rootfs_overlay/etc/init.d/S98 startup script.

Except that --eval-after is smarter and uses base64 encoding.

Alternatively, you can also add the comamdns to run to a new init.d entry to run at the end o the BusyBox init:

and they will be run automatically before the login prompt.

Scripts under /etc/init.d are run by /etc/init.d/rcS, which gets called by the line ::sysinit:/etc/init.d/rcS in /etc/inittab.

The init is selected at:

More details: https://unix.stackexchange.com/questions/30414/what-can-make-passing-init-path-to-program-to-the-kernel-not-start-program-as-i/430614#430614

The final init that actually got selected is shown on Linux v5.9.2 a line of type:

at the very end of the boot logs.

Documented at https://www.kernel.org/doc/html/v4.14/admin-guide/kernel-parameters.html:

And you can try it out with:

From the generated QEMU command, we see that the kernel CLI at LKMC 69f5745d3df11d5c741551009df86ea6c61a09cf now contains:

and the init program outputs:

Source: userland/linux/init_env_poweroff.c.

As of the Linux kernel v5.7 (possibly earlier, I’ve skipped a few releases), boot also shows the init arguments and environment very clearly, which is a great addition:

Most modern desktop distributions have an initrd in their root disk to do early setup.

The rationale for this is described at: https://en.wikipedia.org/wiki/Initial_ramdisk

One obvious use case is having an encrypted root filesystem: you keep the initrd in an unencrypted partition, and then setup decryption from there.

I think GRUB then knows read common disk formats, and then loads that initrd to memory with a /boot/grub/grub.cfg directive of type:

Related: https://stackoverflow.com/questions/6405083/initrd-and-booting-the-linux-kernel

initramfs is just like initrd, but you also glue the image directly to the kernel image itself using the kernel’s build system.

Try it out with:

Notice how we had to rebuild the Linux kernel this time around as well after Buildroot, since in that build we will be gluing the CPIO to the kernel image.

Now, once again, if we look at the QEMU run command generated, we see all that QEMU needs is the -kernel option, no -drive not even -initrd! Pretty cool:

It is also interesting to observe how this increases the size of the kernel image if you do a:

before and after using initramfs, since the .cpio is now glued to the kernel image.

Don’t forget that to stop using initramfs, you must rebuild the kernel without --initramfs to get rid of the attached CPIO image:

Alternatively, consider using Linux kernel build variants if you need to switch between initramfs and non initramfs often:

Setting up initramfs is very easy: our scripts just set CONFIG_INITRAMFS_SOURCE to point to the CPIO path.

http://nairobi-embedded.org/initramfs_tutorial.html shows a full manual setup.

This is how /proc/mounts shows the root filesystem:

TODO: understand /dev/root better:

TODO we were not able to get it working yet: https://stackoverflow.com/questions/49261801/how-to-boot-the-linux-kernel-with-initrd-or-initramfs-with-gem5

This would require gem5 to load the CPIO into memory, just like QEMU. Grepping initrd shows some ARM hits under:

but they are commented out.

This could in theory be easier to make work than initrd since the emulator does not have to do anything special.

However, it didn’t: boot fails at the end because it does not see the initramfs, but rather tries to open our dummy root filesystem, which unsurprisingly does not have a format in a way that the kernel understands:

We think that this might be because gem5 boots directly vmlinux, and not from the final compressed images that contain the attached rootfs such as bzImage, which is what QEMU does, see also: Section 17.20.1, “vmlinux vs bzImage vs zImage vs Image”.

To do this failed test, we automatically pass a dummy disk image as of gem5 7fa4c946386e7207ad5859e8ade0bbfc14000d91 since the scripts don’t handle a missing --disk-image well, much like is currently done for Baremetal.

Interestingly, using initramfs significantly slows down the gem5 boot, even though it did not work. For example, we’ve observed a 4x slowdown of as 17062a2e8b6e7888a14c3506e9415989362c58bf for aarch64. This must be because expanding the large attached CPIO must be expensive. We can clearly see from the kernel logs that the kernel just hangs at a point after the message PCI: CLS 0 bytes, default 64 for a long time before proceeding further.

Files that contain device trees have the .dtb extension when compiled, and .dts when in text form.

You can convert between those formats with:

Buildroot builds the tool due to BR2_PACKAGE_HOST_DTC=y.

On Ubuntu 18.04, the package is named:

See also: https://stackoverflow.com/questions/14000736/tool-to-visualize-the-device-tree-file-dtb-used-by-the-linux-kernel/39931834#39931834

Device tree files are provided to the emulator just like the root filesystem and the Linux kernel image.

In real hardware, those components are also often provided separately. For example, on the Raspberry Pi 2, the SD card must contain two partitions:

See also: https://stackoverflow.com/questions/29837892/how-to-run-a-c-program-with-no-os-on-the-raspberry-pi/40063032#40063032

Good format descriptions:

Minimal example

Check correctness with:

Separate nodes are simply merged by node path, e.g.:

then dtc a.dts gives:

https://unix.stackexchange.com/questions/265890/is-it-possible-to-get-the-information-for-a-device-tree-using-sys-of-a-running/330926#330926

This is specially interesting because QEMU and gem5 are capable of generating DTBs that match the selected machine depending on dynamic command line parameters for some types of machines.

So observing the device tree from the guest allows to easily see what the emulator has generated.

Compile the dtc tool into the root filesystem:

-M virt for example, which we use by default for aarch64, boots just fine without the -dtb option:

Then, from inside the guest:

contains:

Since emulators know everything about the hardware, they can automatically generate device trees for us, which is very convenient.

This is the case for both QEMU and gem5.

For example, if we increase the number of cores to 2:

QEMU automatically adds a second CPU to the DTB!

The action seems to be happening at: hw/arm/virt.c.

You can dump the DTB QEMU generated with:

as mentioned at: https://lists.gnu.org/archive/html/qemu-discuss/2017-02/msg00051.html

gem5 fs_bigLITTLE 2a9573f5942b5416fb0570cf5cb6cdecba733392 can also generate its own DTB.

gem5 can generate DTBs on ARM with --generate-dtb. The generated DTB is placed in the m5out directory named as system.dtb.

TODO: we haven’t gotten it to work yet, but it should be doable, and this is an outline of how to do it. Just don’t expect this to tested very often for now.

We can test KVM on arm by running this repository inside an Ubuntu arm QEMU VM.

This produces no speedup of course, since the VM is already slow since it cannot use KVM on the x86 host.

First, obtain an Ubuntu arm64 virtual machine as explained at: https://askubuntu.com/questions/281763/is-there-any-prebuilt-qemu-ubuntu-image32bit-online/1081171#1081171

Then, from inside that image:

and then proceed exactly as in Prebuilt setup.

We don’t want to build the full Buildroot image inside the VM as that would be way too slow, thus the recommendation for the prebuilt setup.

TODO: do the right thing and cross compile QEMU and gem5. gem5’s Python parts might be a pain. QEMU should be easy: https://stackoverflow.com/questions/26514252/cross-compile-qemu-for-arm

While gem5 does have KVM, as of 2019 its support has not been very good, because debugging it is harder and people haven’t focused intensively on it.

X86 was broken with pending patches: https://www.mail-archive.com/gem5-users@gem5.org/msg15046.html It failed immediately on:

also mentioned at:

Bibliography:

Let’s run userland/c/command_line_arguments.c built with the Buildroot toolchain on QEMU user mode:

Output:

./run --userland path resolution is analogous to that of ./run --baremetal.

./build user-mode-qemu first builds Buildroot, and then runs ./build-userland, which is further documented at: Section 2.8, “Userland setup”. It also builds QEMU. If you ahve already done a QEMU Buildroot setup previously, this will be very fast.

If you modify the userland programs, rebuild simply with:

To rebuild just QEMU userland if you hack it, use:

The:

is needed because QEMU has two separate executables:

Automatically run all userland tests that can be run in user mode simulation, and check that they exit with status 0:

Or just for QEMU:

Source: test-executables

This script skips a manually configured list of tests, notably:

Tests under userland/libs/ are only run if --package or --package-all are given as described at userland/libs directory.

The gem5 tests require building statically with build id static, see also: Section 11.7, “gem5 syscall emulation mode”. TODO automate this better.

See: Section 38.16, “Test this repo” for more useful testing tips.

If you followed QEMU Buildroot setup, you can now run the executables created by Buildroot directly as:

To easily explore the userland executable environment interactively, you can do:

or:

Here is an interesting examples of this: Section 17.19.1, “Linux Test Project”

At 125d14805f769104f93c510bedaa685a52ec025d we moved Buildroot from uClibc to glibc, and caused some user mode pain, which we document here.

Example:

Running dynamically linked executables in QEMU requires pointing it to the root filesystem with the -L option so that it can find the dynamic linker and shared libraries, see also:

We pass -L by default, so everything just works.

However, in case something goes wrong, you can also try statically linked executables, since this mechanism tends to be a bit more stable, for example:

Running statically linked executables sometimes makes things break:

The following work on both QEMU and gem5 as of LKMC 99d6bc6bc19d4c7f62b172643be95d9c43c26145 + 1. Interactive input:

Source: userland/c/getchar.c

A line of type should show:

and after pressing say a and Enter, we get:

Note however that due to QEMU user mode does not show stdout immediately we don’t really see the initial enter a character line.

Non-interactive input from a file by forwarding emulators stdin implicitly through our Python scripts:

Input from a file by explicitly requesting our scripts to use it via the Python API:

This is especially useful when running tests that require stdin input.

Less robust than QEMU’s, but still usable:

There are much more unimplemented syscalls in gem5 than in QEMU. Many of those are trivial to implement however.

So let’s just play with some static ones:

TODO: how to escape spaces on the command line arguments?

GDB step debug also works normally on gem5:

Provided by BusyBox:

If you are feeling raw, you can insert and remove modules with our own minimal module inserter and remover!

which teaches you how it is done from C code.

Source:

The Linux kernel offers two system calls for module insertion:

and:

documents that:

finit is newer and was added only in v3.8. More rationale: https://lwn.net/Articles/519010/

Bibliography: https://stackoverflow.com/questions/5947286/how-to-load-linux-kernel-modules-from-c-code

Implemented as a BusyBox applet by default: https://git.busybox.net/busybox/tree/modutils/modprobe.c?h=1_29_stable

modprobe searches for modules installed under:

and specified in the modules.order file.

This is the default install path for CONFIG_SOME_MOD=m modules built with make modules_install in the Linux kernel tree, with root path given by INSTALL_MOD_PATH, and therefore canonical in that sense.

Currently, there are only two kinds of kernel modules that you can try out with modprobe:

We are not installing out custom ./build-modules modules there, because:

The more "reference" kernel.org implementation of lsmod, insmod, rmmod, etc.: https://git.kernel.org/pub/scm/utils/kernel/kmod/kmod.git

Default implementation on desktop distros such as Ubuntu 16.04, where e.g.:

gives:

and:

contains:

BusyBox also implements its own version of those executables, see e.g. modprobe. Here we will only describe features that differ from kmod to the BusyBox implementation.

OverlayFS is a filesystem merged in the Linux kernel in 3.18.

As the name suggests, OverlayFS allows you to merge multiple directories into one. The following minimal runnable examples should give you an intuition on how it works:

We are very interested in this filesystem because we are looking for a way to make host cross compiled executables appear on the guest root / without reboot.

This would have several advantages:

We can already make host files appear on the guest with 9P, but they appear on a subdirectory instead of the root.

If they would appear on the root instead, that would be even more awesome, because you would just use the exact same paths relative to the root transparently.

For example, we wouldn’t have to mess around with variables such as PATH and LD_LIBRARY_PATH.

The idea is to:

We already have a prototype of this running from fstab on guest at /mnt/overlay, but it has the following shortcomings:

Even more awesome than chroot would be to pivot_root, but I couldn’t get that working either:

A simpler and possibly less overhead alternative to 9P would be to generate a secondary disk image with the benchmark you want to rebuild.

Then you can umount and re-mount on guest without reboot.

To build the secondary disk image run build-disk2:

This will put the entire out_rootfs_overlay_dir into a squashfs filesystem.

Then, if that filesystem is present, ./run will automatically pass it as the second disk on the command line.

For example, from inside QEMU, you can mount that disk with:

To update the secondary disk while a simulation is running to avoid rebooting, first unmount in the guest:

and then on the host:

and now you can re-run the updated version of the executable on the guest after remounting it.

gem5 fs.py support for multiple disks is discussed at: https://stackoverflow.com/questions/50862906/how-to-attach-multiple-disk-images-in-a-simulation-with-gem5-fs-py/51037661#51037661

Text mode is the default mode for QEMU.

The opposite of text mode is QEMU graphic mode

In text mode, we just show the serial console directly on the current terminal, without opening a QEMU GUI window.

You cannot see any graphics from text mode, but text operations in this mode, including:

making this a good default, unless you really need to use with graphics.

Text mode works by sending the terminal character by character to a serial device.

This is different from a display screen, where each character is a bunch of pixels, and it would be much harder to convert that into actual terminal text.

For more details, see:

Note that you can still see an image even in text mode with the VNC:

and on another terminal:

but there is not terminal on the VNC window, just the CONFIG_LOGO penguin.

Enable graphic mode with:

Outcome: you see a penguin due to CONFIG_LOGO.

For a more exciting GUI experience, see: Section 14.4, “X11 Buildroot”

Text mode is the default due to the following considerable advantages:

Text mode has the following limitations over graphics mode:

x86_64 has a VGA device enabled by default, as can be seen as:

and the Linux kernel picks it up through the fbdev graphics system as can be seen from:

flooding the screen with colors. See also: https://superuser.com/questions/223094/how-do-i-know-if-i-have-kms-enabled

gem5 does not have a "text mode", since it cannot redirect the Linux terminal to same host terminal where the executable is running: you are always forced to connect to the terminal with gem-shell.

TODO could not get it working on x86_64, only ARM.

Overview: https://stackoverflow.com/questions/50364863/how-to-get-graphical-gui-output-and-user-touch-keyboard-mouse-input-in-a-ful/50364864#50364864

More concretely, first build the kernel with the gem5 arm Linux kernel patches, and then run:

and then on another shell:

The CONFIG_LOGO penguin only appears after several seconds, together with kernel messages of type:

The port 5900 is incremented by one if you already have something running on that port, gem5 stdout tells us the right port on stdout as:

and when we connect it shows a message:

Alternatively, you can also dump each new frame to an image file with --frame-capture:

This creates on compressed PNG whenever the screen image changes inside the m5out directory with filename of type:

It is fun to see how we get one new frame whenever the white underscore cursor appears and reappears under the penguin!

The last frame is always available uncompressed at: system.framebuffer.png.

TODO kmscube failed on aarch64 with:

Tested on: 38fd6153d965ba20145f53dc1bb3ba34b336bde9

Once you’ve seen the CONFIG_LOGO penguin as a sanity check, you can try to go for a cooler X11 Buildroot setup.

Build and run:

Inside QEMU:

And then from the GUI you can start exciting graphical programs such as:

Outcome: Figure 1, “X11 Buildroot graphical user interface screenshot”

We don’t build X11 by default because it takes a considerable amount of time (about 20%), and is not expected to be used by most users: you need to pass the -x flag to enable it.

More details: https://unix.stackexchange.com/questions/70931/how-to-install-x11-on-my-own-linux-buildroot-system/306116#306116

Not sure how well that graphics stack represents real systems, but if it does it would be a good way to understand how it works.

To x11 packages have an xserver prefix as in:

the easiest way to find them out is to just list "$(./getvar buildroot_build_build_dir)/x*.

TODO as of: c2696c978d6ca88e8b8599c92b1beeda80eb62b2 I noticed that startx leads to a BUG_ON: