Linux Kernel Module Cheat

24.22.3.1. gem5 `m5.objects` module

All SimObjects seem to be automatically added to the m5.objects namespace, and this is done in a very convoluted way, let’s try to understand a bit:

src/python/m5/objects/__init__.py

contains:

modules = __loader__.modules

for module in modules.keys():
    if module.startswith('m5.objects.'):
        exec("from %s import *" % module)

And from IPDB we see that this appears to loop over every object string of type m5.objects.modulename.

This __init__ gets called from src/python/importer.py at the exec:

class CodeImporter(object):
    def load_module(self, fullname):
            override = os.environ.get('M5_OVERRIDE_PY_SOURCE', 'false').lower()
            if override in ('true', 'yes') and  os.path.exists(abspath):
                src = open(abspath, 'r').read()
                code = compile(src, abspath, 'exec')

            if os.path.basename(srcfile) == '__init__.py':
                mod.__path__ = fullname.split('.')
                mod.__package__ = fullname
            else:
                mod.__package__ = fullname.rpartition('.')[0]
            mod.__file__ = srcfile

            exec(code, mod.__dict__)

import sys
importer = CodeImporter()
add_module = importer.add_module
sys.meta_path.append(importer)

Here as a bonus here we also see how M5_OVERRIDE_PY_SOURCE works.

In src/SConscript we see that SimObject is just a PySource with module equals to m5.objects:

class SimObject(PySource):
    def __init__(self, source, tags=None, add_tags=None):
        '''Specify the source file and any tags (automatically in
        the m5.objects package)'''
        super(SimObject, self).__init__('m5.objects', source, tags, add_tags)

The add_module method seems to be doing the magic and is called from src/sim/init.cc:

bool
EmbeddedPython::addModule() const
{
    PyObject *code = getCode();
    PyObject *result = PyObject_CallMethod(importerModule, PyCC("add_module"),

which is called from:

int
EmbeddedPython::initAll()
{
    // Load the importer module
    PyObject *code = importer->getCode();
    importerModule = PyImport_ExecCodeModule(PyCC("importer"), code);
    if (!importerModule) {
        PyErr_Print();
        return 1;
    }

    // Load the rest of the embedded python files into the embedded
    // python importer
    list<EmbeddedPython *>::iterator i = getList().begin();
    list<EmbeddedPython *>::iterator end = getList().end();
    for (; i != end; ++i)
        if (!(*i)->addModule())

and getList comes from:

EmbeddedPython::EmbeddedPython(const char *filename, const char *abspath,
    const char *modpath, const unsigned char *code, int zlen, int len)
    : filename(filename), abspath(abspath), modpath(modpath), code(code),
      zlen(zlen), len(len)
{
    // if we've added the importer keep track of it because we need it
    // to bootstrap.
    if (string(modpath) == string("importer"))
        importer = this;
    else
        getList().push_back(this);
}

list<EmbeddedPython *> &
EmbeddedPython::getList()
{
    static list<EmbeddedPython *> the_list;
    return the_list;
}

and the constructor in turn gets called from per SimObject autogenerated files such as e.g. dev/storage/Ide.py.cc for src/dev/storage/Ide.py:

EmbeddedPython embedded_m5_objects_Ide(
    "m5/objects/Ide.py",
    "/home/ciro/bak/git/linux-kernel-module-cheat/data/gem5/master4/src/dev/storage/Ide.py",
    "m5.objects.Ide",
    data_m5_objects_Ide,
    947,
    2099);

} // anonymous namespace

which get autogenerated at src/SConscript:

def embedPyFile(target, source, env):

for source in PySource.all:
    base_py_env.Command(source.cpp, [ py_marshal, source.tnode ],
                        MakeAction(embedPyFile, Transform("EMBED PY")))

where the PySource.all thing as you might expect is a static list of all PySource source files as they get updated in the constructor.

Tested in gem5 d9cb548d83fa81858599807f54b52e5be35a6b03.

Linux Kernel Module Cheat

24.22.3.1. gem5 m5.objects module

24.22.3.1. gem5 `m5.objects` module