17.13.4. Count boot instructions

TODO: didn’t port during refactor after 3b0a343647bed577586989fb702b760bd280844a. Reimplementing should not be hard.

Results (boot not excluded) are shown at: Table 1, “Boot instruction counts for various setups”

Table 1. Boot instruction counts for various setups
Commit Arch Simulator Instruction count

7228f75ac74c896417fb8c5ba3d375a14ed4d36b

arm

QEMU

680k

7228f75ac74c896417fb8c5ba3d375a14ed4d36b

arm

gem5 AtomicSimpleCPU

160M

7228f75ac74c896417fb8c5ba3d375a14ed4d36b

arm

gem5 HPI

155M

7228f75ac74c896417fb8c5ba3d375a14ed4d36b

x86_64

QEMU

3M

7228f75ac74c896417fb8c5ba3d375a14ed4d36b

x86_64

gem5 AtomicSimpleCPU

528M

QEMU:

./trace-boot --arch x86_64

sample output:

instructions 1833863
entry_address 0x1000000
instructions_firmware 20708

gem5:

./run --arch aarch64 --emulator gem5 --eval 'm5 exit'
# Or:
# ./run --arch aarch64 --emulator gem5 --eval 'm5 exit' -- --cpu-type=HPI --caches
./gem5-stat --arch aarch64 sim_insts

Notes:

  • 0x1000000 is the address where QEMU puts the Linux kernel at with -kernel in x86.

    It can be found from:

    ./run-toolchain readelf -- -e "$(./getvar vmlinux)" | grep Entry

    TODO confirm further. If I try to break there with:

    ./run-gdb *0x1000000

    but I have no corresponding source line. Also note that this line is not actually the first line, since the kernel messages such as early console in extract_kernel have already shown on screen at that point. This does not break at all:

    ./run-gdb extract_kernel

    It only appears once on every log I’ve seen so far, checked with grep 0x1000000 trace.txt

    Then when we count the instructions that run before the kernel entry point, there is only about 100k instructions, which is insignificant compared to the kernel boot itself.

    TODO --arch arm and --arch aarch64 does not count firmware instructions properly because the entry point address of the ELF file (ffffff8008080000 for aarch64) does not show up on the trace at all. Tested on f8c0502bb2680f2dbe7c1f3d7958f60265347005.

  • We can also discount the instructions after init runs by using readelf to get the initial address of init. One easy way to do that now is to just run:

    ./run-gdb --userland "$(./getvar userland_build_dir)/linux/poweroff.out" main

    And get that from the traces, e.g. if the address is 4003a0, then we search:

    grep -n 4003a0 trace.txt

    I have observed a single match for that instruction, so it must be the init, and there were only 20k instructions after it, so the impact is negligible.

  • to disable networking. Is replacing init enough?

    CONFIG_NET=n did not significantly reduce instruction counts, so maybe replacing init is enough.

  • gem5 simulates memory latencies. So I think that the CPU loops idle while waiting for memory, and counts will be higher.