Linux Kernel Module Cheat

24.16.6. gem5 Ruby build

gem5 has two types of memory system:

the classic memory system, which is used by default, its caches are covered at: gem5 event queue TimingSimpleCPU syscall emulation freestanding example analysis with caches
the Ruby memory system

The Ruby memory system includes the SLICC domain specific language to describe memory systems: http://gem5.org/Ruby SLICC transpiles to C++ auto-generated files under build/<isa>/mem/ruby/protocol/.

Ruby seems to have usage outside of gem5, but the naming overload with the Ruby programming language, which also has domain specific languages as a concept, makes it impossible to google anything about it!

Since it is not the default, Ruby is generally less stable that the classic memory model. However, because it allows describing a wide variety of important cache coherence protocols, while the classic system only describes a single protocol, Ruby is very importanonly describes a single protocol, Ruby is a very important feature of gem5.

Ruby support must be enabled at compile time with the scons PROTOCOL= flag, which compiles support for the desired memory system type.

Note however that most ISAs already implicitly set PROTOCOL via the build_opts/ directory, e.g. build_opts/ARM contains:

PROTOCOL = 'MOESI_CMP_directory'

and therefore ARM already compiles MOESI_CMP_directory by default.

Then, with fs.py and se.py, you can choose to use either the classic or the ruby system type selected at build time with PROTOCOL= at runtime by passing the --ruby option:

if --ruby is given, use the ruby memory system that was compiled into gem5. Caches are always present when Ruby is used, since the main goal of Ruby is to specify the cache coherence protocol, and it therefore hardcodes cache hierarchies.
otherwise, use the classic memory system. Caches may be optional for certain CPU types and are enabled with --caches.

Note that the --ruby option has some crazy side effects besides enabling Ruby, e.g. it sets the default --cpu-type to TimingSimpleCPU instead of the otherwise default AtomicSimpleCPU. TODO: I have been told that this is because sends the packet atomically,atomic requests do not work with Ruby, only timing.

It is not possible to build more than one Ruby system into a single build, and this is a major pain point for testing Ruby: https://gem5.atlassian.net/browse/GEM5-467

For example, to use a two level MESI cache coherence protocol we can do:

./build-gem5 --arch aarch64 --gem5-build-id ruby -- PROTOCOL=MESI_Two_Level
./run --arch aarch64 --emulator -gem5 --gem5-build-id ruby -- --ruby

and during build we see a humongous line of type:

[   SLICC] src/mem/protocol/MESI_Two_Level.slicc -> ARM/mem/protocol/AccessPermission.cc, ARM/mem/protocol/AccessPermission.hh, ...

which shows that dozens of C++ files are being generated from Ruby SLICC.

The relevant Ruby source files live in the source tree under:

src/mem/protocol/MESI_Two_Level*

We already pass the SLICC_HTML flag by default to the build, which generates an HTML summary of each memory protocol under (TODO broken: https://gem5.atlassian.net/browse/GEM5-357):

xdg-open "$(./getvar --arch aarch64 --gem5-build-id ruby gem5_build_build_dir)/ARM/mem/protocol/html/index.html"

A minimized ruby config which was not merged upstream can be found for study at: https://gem5-review.googlesource.com/c/public/gem5/+/13599/1

One easy way to see that Ruby is being used without understanding it in detail is to enable some logging:

./run \
  --arch aarch64 \
  --emulator gem5 \
  --gem5-worktree master \
  --userland userland/arch/aarch64/freestanding/linux/hello.S \
  --static \
  --trace ExecAll,FmtFlag,Ruby,XBar \
  -- \
  --ruby \
;
cat "$(./getvar --arch aarch64 --emulator gem5 trace_txt_file)"

Then:

when the --ruby flag is given, we see a gazillion Ruby related messages prefixed e.g. by RubyPort:.

We also observe from ExecEnable lines that instruction timing is not simple anymore, so the memory system must have latencies
without --ruby, we instead see XBar (Coherent Crossbar) related messages such as CoherentXBar:, which I believe is the more precise name for the memory model that the classic memory system uses: gem5 crossbar interconnect.

Certain features may not work in Ruby. For example, gem5 checkpoint creation is only possible in Ruby protocols that support flush, which is the case for PROTOCOL=MOESI_hammer but not PROTOCOL=MESI_Three_Level: https://www.mail-archive.com/gem5-users@gem5.org/msg17418.html

Tested in gem5 d7d9bc240615625141cd6feddbadd392457e49eb.