Some Structure

Step 3 - Some Structure

This is the third step, adding structs - each step adds or improves a small feature, this text is just highlighting a few details along the way, the best way to work through is to build and run each one, reading through the code with this description alongside. Similarly reading this alongside the diff for each step will also mean more than either just reading the text or just reading the code alone.

That's what works for me anyway, YMMV.

Ok, so methods that take and return scalars are covered, now let's add on struct support - the ability to pass compound data types with named fields in both directions.

The demo struct

The demo library gets a TestStruct with a handful of fields:

struct TestStruct {
    int         m_int    = 12;
    double      m_double = 6.78;
    std::string m_string = "there";
};

And two new methods on Demo that take and return it:

TestStruct getStruct();
void       doStruct(TestStruct s);

Reflecting the struct

The top-level macro XSTRUCT_REFLECT(TestStruct) expands to registerStructReflect<TestStruct>("TestStruct"), which uses C++26 to walk the fields: (stop me if you're getting bored, but the pattern is the same again, we use a reflect operator on the struct type and use an expansion statement to loop over the members)

template <typename S>
void registerStructReflect(const char *name) {
    template for (constexpr auto Member :
        std::meta::nonstatic_data_members_of(^^S, std::meta::access_context::current()))
    {
        using F = typename[:std::meta::type_of(Member):];
        std::string fname(std::meta::identifier_of(Member));

        FieldInfo field;
        field.name = fname;
        field.type = makeTypeInfo<F>();
        Registry::instance().addStructField<S>(std::move(field));
    }
}

As before:

• ^^S - reflect operator, gives a std::meta::info handle for the struct type S
• nonstatic_data_members_of - returns the non-static data members; access_context::current() means "things visible from here" (you could use ::unchecked() to bypass access control)
• typename[:std::meta::type_of(Member):] - splices the member's type back into C++ so we can call makeTypeInfo<F>()

The StructBuilder

We can't put structs into our union the same way scalars, instead we'll make a "structbuilder" object and pass a pointer to that. The builder holds a map of XPlatValues keyed by name:

struct StructBuilder {
    std::string                        structName;
    std::map<std::string, XPlatValue>  fields;
};

The C API for building and reading structs:

void *XPLAT_createStructBuilder(const char *structName);
void  XPLAT_destroyStructBuilder(void *builder);
void  XPLAT_setBuilderField(void *builder, const char *fieldName, const XPlatValue *value);
int   XPLAT_getBuilderField(void *builder, const char *fieldName, XPlatValue *value);

And for querying struct metadata, our index based query methods:

size_t           XPLAT_getNumStructs(const Registry *registry);
const char      *XPLAT_getStructName(const Registry *registry, size_t index);
size_t           XPLAT_getStructFieldCount(const Registry *registry, size_t structIndex);
const FieldInfo *XPLAT_getStructField(const Registry *registry, size_t structIndex, size_t fieldIndex);
const char      *XPLAT_getFieldName(const FieldInfo *field);
XPlatType        XPLAT_getFieldType(const FieldInfo *field);

When XPLAT_invoke returns a struct-typed value, the structValue pointer in the result XPlatValue points to a StructBuilder that the caller is responsible for destroying, when passing a struct in as an argument, the caller creates a builder, fills the fields, and passes the pointer. (again we can tidy up our lifetimes later)

In the Python

At bind_library time we read the struct metadata and build a lightweight Python class for each struct, storing the field names and types on it:

num_structs = lib.XPLAT_getNumStructs(registry)
for i in range(num_structs):
    struct_name = lib.XPLAT_getStructName(registry, i).decode('utf-8')
    num_fields  = lib.XPLAT_getStructFieldCount(registry, i)
    fields = []
    for j in range(num_fields):
        field_ptr       = lib.XPLAT_getStructField(registry, i, j)
        field_name      = lib.XPLAT_getFieldName(field_ptr).decode('utf-8')
        field_type_info = _read_type_info(lib, lib.XPLAT_getFieldTypeInfo(field_ptr))
        fields.append((field_name, field_type_info))

    StructClass._xplat_fields = fields
    structs[struct_name] = StructClass

Two helpers handle the conversions, to send a Python dict into C++:

def _python_to_c_struct(self, python_obj, struct_class):
    builder_handle = self.lib.XPLAT_createStructBuilder(struct_class.__name__.encode('utf-8'))
    for field_name, field_type_info in struct_class._xplat_fields:
        if field_name in python_obj:
            val = self._python_to_c_value(python_obj[field_name], field_type_info)
            self.lib.XPLAT_setBuilderField(builder_handle, field_name.encode('utf-8'), ctypes.byref(val))
    return builder_handle

And to read a struct back from C++:

def _c_to_python_struct(self, builder_handle, struct_class):
    result = {}
    for field_name, field_type_info in struct_class._xplat_fields:
        val = XVar()
        if self.lib.XPLAT_getBuilderField(builder_handle, field_name.encode('utf-8'), ctypes.byref(val)) == 0:
            result[field_name] = self._c_to_python_value(val, field_type_info, True)
    self.lib.XPLAT_destroyStructBuilder(builder_handle)
    return result

Structs arrive in Python as plain dicts and can be passed back as plain dicts.

Usage

obj = demo.Demo()

s = obj.getStruct()
print(s)   # {'m_int': 12, 'm_double': 6.78, 'm_string': 'there'}

obj.doStruct({
    'm_int':    321,
    'm_double': 9.99,
    'm_string': 'from python',
})
print(obj.getStruct())
# {'m_int': 321, 'm_double': 9.99, 'm_string': 'from python'}