Linux Kernel Module Cheat

24.22.2. gem5 Python C++ interaction

The interaction uses the Python C extension interface https://docs.python.org/2/extending/extending.html interface through the pybind11 helper library: https://github.com/pybind/pybind11

The C++ executable both:

starts running the Python executable
provides Python classes written in C++ for that Python code to use

An example of this can be found at:

then gem5 magic SimObject class adds some crazy stuff on top of it further, is is a mess. In particular, it auto generates params/ headers. TODO: why is this mess needed at all? pybind11 seems to handle constructor arguments just fine:

Let’s study BadDevice for example:

src/dev/BadDevice.py defines devicename:

class BadDevice(BasicPioDevice):
    type = 'BadDevice'
    cxx_header = "dev/baddev.hh"
    devicename = Param.String("Name of device to error on")

The object is created in Python for example from src/dev/alpha/Tsunami.py as:

    fb = BadDevice(pio_addr=0x801fc0003d0, devicename='FrameBuffer')

Since BadDevice has no __init__ method, and neither BasicPioDevice, it all just falls through until the SimObject.__init__ constructor.

This constructor will loop through the inheritance chain and give the Python parameters to the C++ BadDeviceParams class as follows.

The auto-generated build/ARM/params/BadDevice.hh file defines BadDeviceParams in C++:

#ifndef __PARAMS__BadDevice__
#define __PARAMS__BadDevice__

class BadDevice;

#include <cstddef>
#include <string>

#include "params/BasicPioDevice.hh"

struct BadDeviceParams
    : public BasicPioDeviceParams
{
    BadDevice * create();
    std::string devicename;
};

#endif // __PARAMS__BadDevice__

and ./python/_m5/param_BadDevice.cc defines the param Python from C++ with pybind11:

namespace py = pybind11;

static void
module_init(py::module &m_internal)
{
    py::module m = m_internal.def_submodule("param_BadDevice");
    py::class_<BadDeviceParams, BasicPioDeviceParams, std::unique_ptr<BadDeviceParams, py::nodelete>>(m, "BadDeviceParams")
        .def(py::init<>())
        .def("create", &BadDeviceParams::create)
        .def_readwrite("devicename", &BadDeviceParams::devicename)
        ;

    py::class_<BadDevice, BasicPioDevice, std::unique_ptr<BadDevice, py::nodelete>>(m, "BadDevice")
        ;

}

static EmbeddedPyBind embed_obj("BadDevice", module_init, "BasicPioDevice");

src/dev/baddev.hh then uses the parameters on the constructor:

class BadDevice : public BasicPioDevice
{
  private:
    std::string devname;

  public:
    typedef BadDeviceParams Params;

  protected:
    const Params *
    params() const
    {
        return dynamic_cast<const Params *>(_params);
    }

  public:
     /**
      * Constructor for the Baddev Class.
      * @param p object parameters
      * @param a base address of the write
      */
    BadDevice(Params *p);

src/dev/baddev.cc then uses the parameter:

BadDevice::BadDevice(Params *p)
    : BasicPioDevice(p, 0x10), devname(p->devicename)
{
}

It has been found that this usage of pybind11 across hundreds of SimObject files accounted for 50% of the gem5 build time at one point: pybind11 accounts for 50% of gem5 build time.

To get a feeling of how SimObject objects are run, see: gem5 event queue AtomicSimpleCPU syscall emulation freestanding example analysis.

Bibliography:

Tested on gem5 08c79a194d1a3430801c04f37d13216cc9ec1da3.