Linux Kernel Module Cheat

35.2.1. Benchmark Linux kernel boot

Run all kernel boot benchmarks for one arch:

./build-test-boot --size 3 && ./test-boot --all-archs --all-emulators --size 3
cat "$(./getvar test_boot_benchmark_file)"

Sample results at LKMC 8fb9db39316d43a6dbd571e04dd46ae73915027f:

cmd ./run --arch x86_64 --eval './linux/poweroff.out'
time 8.25
exit_status 0

cmd ./run --arch x86_64 --eval './linux/poweroff.out' --kvm
time 1.22
exit_status 0

cmd ./run --arch x86_64 --eval './linux/poweroff.out' --trace exec_tb
time 8.83
exit_status 0
instructions 2244297

cmd ./run --arch x86_64 --eval 'm5 exit' --emulator gem5
time 213.39
exit_status 0
instructions 318486337

cmd ./run --arch arm --eval './linux/poweroff.out'
time 6.62
exit_status 0

cmd ./run --arch arm --eval './linux/poweroff.out' --trace exec_tb
time 6.90
exit_status 0
instructions 776374

cmd ./run --arch arm --eval 'm5 exit' --emulator gem5
time 118.46
exit_status 0
instructions 153023392

cmd ./run --arch arm --eval 'm5 exit' --emulator gem5 -- --cpu-type=HPI --caches --l2cache --l1d_size=1024kB --l1i_size=1024kB --l2_size=1024kB --l3_size=1024kB
time 2250.40
exit_status 0
instructions 151981914

cmd ./run --arch aarch64 --eval './linux/poweroff.out'
time 4.94
exit_status 0

cmd ./run --arch aarch64 --eval './linux/poweroff.out' --trace exec_tb
time 5.04
exit_status 0
instructions 233162

cmd ./run --arch aarch64 --eval 'm5 exit' --emulator gem5
time 70.89
exit_status 0
instructions 124346081

cmd ./run --arch aarch64 --eval 'm5 exit' --emulator gem5 -- --cpu-type=HPI --caches --l2cache --l1d_size=1024kB --l1i_size=1024kB --l2_size=1024kB --l3_size=1024kB
time 381.86
exit_status 0
instructions 124564620

cmd ./run --arch aarch64 --eval 'm5 exit' --emulator gem5 --gem5-build-type fast
time 58.00
exit_status 0
instructions 124346081

cmd ./run --arch aarch64 --eval 'm5 exit' --emulator gem5 --gem5-build-type debug
time 1022.03
exit_status 0
instructions 124346081

TODO: aarch64 gem5 and QEMU use the same kernel, so why is the gem5 instruction count so much much higher?

2017 Lenovo ThinkPad P51 Ubuntu 19.10 LKMC b11e3cd9fb5df0e3fe61de28e8264bbc95ea9005 gem5 e779c19dbb51ad2f7699bd58a5c7827708e12b55 aarch64: 143s. Why huge increases from 70s on above table? Kernel size is also huge BTW: 147MB.

Note that https://gem5.atlassian.net/browse/GEM5-337 "ARM PAuth patch slows down Linux boot 2x from 2 minutes to 4 minutes" was already semi fixed at that point.

Same but with Buildroot vanilla kernel (kernel v4.19): 44s to blow up at "Please append a correct "root=" boot option; here are the available partitions" because missing some filesystem mount option. But likely wouldn’t be much more until after boot since we are almost already done by then! Therefore this vanilla kernel is much much faster! TODO find which config or kernel commit added so much time! Also that kernel is tiny at 8.5MB.

Same but hacking BR2_LINUX_KERNEL_LATEST_VERSION=y and BR2_PACKAGE_HOST_LINUX_HEADERS_CUSTOM_5_3=y which reaches kernel 5.3.14 which closer to the LKMC one 5.4.3: 40s, which is very similar for the older kernel. Therefore it does not loook like it is a problem of kernel code changes, but rather of configs.

Same but with: gem5 arm Linux kernel patches at v4.15: 73s, kernel size: 132M.

On Ubuntu 20.04, LKMC d3f8d3e99f2e554aae6c3b325b350bcf7f3f087f (Linux kernel 5.4.3), gem5 6bc2111c9674d0c8db22f6a6adcc00e49625aabd (sept 2020):

./run --arch aarch64 --emulator gem5 --quit-after-boot

took 193s. With some minimal newer kernel boot patches:

kernel v5.7: 238s
kernel v5.8: 239s

On Ubuntu 20.04 gem5 3ca404da175a66e0b958165ad75eb5f54cb5e772 this took 22 minutes 53 seconds:

./run -aa -eg --cpus 2 --tmux --quit-after-boot -- --cpu-type DerivO3CPU --caches