Winging it

Step 10 - Sharp Objects So someone asked, can you dynamically load the library in csharp just like python? Hmm. Probably.... That would be nice, and I'm pretty sure we can do a lot of manipulation on the go with C#, like we can in python, there's a comprehensive reflection and invocation interface. How hard can it be? Can we build a c# interface on the fly instead of outputting source code to be compiled? so, significant rummaging later and the answer is a definite "sort of". 👍😁 Emit emit emit Initially, I used Reflection.Emit to generate CLR code at runtime... this worked and provided a fully dynamic binding doing the same as Python, loading the C++ library, calling our C API to interrogate it and building matching C# CLR objects at runtime. yay great success, move on.... but, developing with it is "challenging" you need to know all the available functions and types, write all the code wire up a command line build and crank the handle. So poss...

Preamble I've always been better at the 'doing' rather than the explaining, so as part of my “what can we do with C++26” exercise I thought this would be a good opportunity to attempt to practice the 'explain it' bit. (In other words this might not be great! let me know if I can improve or change this) C++26 has now introduced reflection capabilities ( https://learnmoderncpp.com/2025/07/31/reflection-in-c26-p2996/ ) as a standard language feature, previously this is something you'd need to build yourself using various template and macro trickery. I find the only way I can learn how to use new features is to actually try to use them to do something useful. So my plan was - can I use the new C++26 reflection features to create a dynamic language binding system, and if so, can I do so step by step to make it easy/obvious how it could be improved or extended rather than just write a completed final "ta da" here's a C++ reflection system. Since th...

Step 9 - But what about Java This is the final step, adding a Java generator - each step adds or improves a small feature, this text is just highlighting a few details along the way, the best way to work through is to build and run each one, reading through the code with this description alongside. Similarly reading this alongside the diff for each step will also mean more than either just reading the text or just reading the code alone. That's what works for me anyway, YMMV. Now the swan song. We used and stuck with Python bindings from the start, partly because it's just so simple to prototype/test/debug/develop with against a C library.... but the whole point of this idea is to be language agnostic, we have a C API so we should be able to interface with any language that provies a C calling mechanism. Java is an attractive target. How hard can it be? :) Python built its binding dynamically at runtime, query the registry, create callable stub objects, profit. Java howeve...

Step 8 - Not remotely funny This is the eighth step, adding out-of-process calling - each step adds or improves a small feature, this text is just highlighting a few details along the way, the best way to work through is to build and run each one, reading through the code with this description alongside. Similarly reading this alongside the diff for each step will also mean more than either just reading the text or just reading the code alone. So far we're sticking to loading the marked up libdemo.so into the same process as the python interpreter. Now it gets interesting, what if we have multiple C++ libraries with clashing dependencies? Or the simple fact that if the C++ libary crashes our python interpreter will terminate. Since we have a simple clear C API at the interface, we could pipe that over an inter process comms (IPC) system into a separate process, and if we've got the plumbing right it should "just work" without needing any additional work to ou...

Step 7 - gnioG kcaB nI emiT This is the seventh step, adding c++20 compatability - each step adds or improves a small feature, this text is just highlighting a few details along the way, the best way to work through is to build and run each one, reading through the code with this description alongside. Similarly reading this alongside the diff for each step will also mean more than either just reading the text or just reading the code alone. All the steps so far have required a C++26-compliant compiler with -freflection support. However, we might still want to be able to run this on slightly older compilers. We won't need to change the API, runtime logic or Python binding, all we need to do is use the old style meta-template expansion and populate our now well defined metadata dictionary and type structures. The build change Drop the freflection flag and switch to C++20: CXXFLAGS = -O0 -g -Wall -Wextra -std=c++20 -fPIC -Ixplat -Idemo We'll define and use XPLAT_HAS...

Step 6 - Turtles all the way down This is the sixth step, adding recursive structures - each step adds or improves a small feature, this text is just highlighting a few details along the way, the best way to work through is to build and run each one, reading through the code with this description alongside. Similarly reading this alongside the diff for each step will also mean more than either just reading the text or just reading the code alone. That's what works for me anyway, YMMV. Previously on the show (aka Step 3) we added struct support, but with a limitation, struct fields had to be scalars, you couldn't have a struct that contained another struct. So now we'll remove that limitation with a minor tweak. The problem When registerFieldConverter registers a setter/getter for a struct field it uses a chain of if constexpr branches - one per supported type. Before this step that chain ended with std::chrono::year_month_day , any field with an aggregate type ...

Step 5 - Containment This is the fifth step, adding collection - each step adds or improves a small feature, this text is just highlighting a few details along the way, the best way to work through is to build and run each one, reading through the code with this description alongside. Similarly reading this alongside the diff for each step will also mean more than either just reading the text or just reading the code alone. That's what works for me anyway, YMMV. So far every type has been a scalar - a single value that fits in one slot of the XPlatValue union. However, containers like std::vector and std::map are different, with an arbitrary number of elements, we won't know that at compile time. We'll need some more helpers to add to the API to pass the information across the boundary. The ArrayBuilder The solution is another builder, with a set of C API calls that can be used to construct the array on the heap within the library and pass one of our opaque ...

Step 4 - A new type. std::chrono::y_m_d This is the fourth step, adding a new type, date - each step adds or improves a small feature, this text is just highlighting a few details along the way, the best way to work through is to build and run each one, reading through the code with this description alongside. Similarly reading this alongside the diff for each step will also mean more than either just reading the text or just reading the code alone. That's what works for me anyway, YMMV. We've got the framework working nicely, we can markup classes and basic types, passing in and returning pod types or dictionaries as structs. Now let's add another native type to show the steps needed. Dates makes sense, we have native versions in Python (and Java) and we can now map to a C++20 std::chrono::year_month_day . Encoding How to wire this new type into the variant we have? I don't want to add new types into the union for every new type, but all I really need here i...

Step 3 - Some Structure This is the third step, adding structs - each step adds or improves a small feature, this text is just highlighting a few details along the way, the best way to work through is to build and run each one, reading through the code with this description alongside. Similarly reading this alongside the diff for each step will also mean more than either just reading the text or just reading the code alone. That's what works for me anyway, YMMV. Ok, so methods that take and return scalars are covered, now let's add on struct support - the ability to pass compound data types with named fields in both directions. The demo struct The demo library gets a TestStruct with a handful of fields: struct TestStruct { int m_int = 12; double m_double = 6.78; std::string m_string = "there"; }; And two new methods on Demo that take and return it: TestStruct getStruct(); void doStruct(TestStruct s); Reflecting the st...

Step 2 - Complexity This is the second step, adding enums and method wrappers - each step adds or improves a small feature, this text is just highlighting a few details along the way, the best way to work through is to build and run each one, reading through the code with this description alongside. Similarly reading this alongside the diff for each step will also mean more than either just reading the text or just reading the code alone. That's what works for me anyway, YMMV. Step 1's Python binding worked but was deliberately naive: any method call on a PythonClassInstance was blindly forwarded to XPLAT_invoke via __getattr__ . Here we can fix that - at library load time we'll now iterate the full metadata and pre-build typed stub methods, one per real C++ method. Let's also add a new type in addition to the primitives. Another type - Enums The demo now gets a MyEnum and we expose it with a matching macro: enum class MyEnum { Red, Green, Blue }; XENUM_R...

Step 1 - Open the PoD bay doors This is the first step, basic types and calls - each step adds or improves a small feature, this text is just highlighting a few details along the way, the best way to work through is to build and run each one, reading through the code with this description alongside. Similarly reading this alongside the diff for each step will also mean more than either just reading the text or just reading the code alone. That's what works for me anyway, YMMV. In C++26 with the std::meta features it's possible to interrogate langage metadata at compile time. The plan is to use this to build up a metadata dictionary for a library that can be interrogated dynamically at runtime through an external C API. With that, it will be possible to dynamically generate a Python binding - no pybind, no SWIG, no hand written glue - to interface with the library. Our library can remain a plain C++ library with no need to have any knowledge or special cases for each t...