Specialization: Reflection

About this project

During the last semester at The Game Assembly, we get to specialize in a specific subject of our choice. This specialization is done for a period of 4 weeks.  I took this chance to dive deeper into what reflective programming is. When I started this project, I had never had a use case for reflection before and therefore just briefly touched on it in college. I found it interesting and would automate a big part of the engine we were currently working on.


Background

Reflective programming is a tool that is built into most common programming languages. It allows the programmers to inspect and work with members of a type dynamically at runtime. The use cases of reflections are endless and are a great tool when developing a game engine. Some use cases of reflection are:

  • Scene serialization
  • Exposing components available
  • Exposing variables in components for modification in runtime
  • Exposing functions and binding them as callbacks for events


For my specialization, I wanted to implement my take on reflection in C++ that I can then use in our in-house engine. Some features I would require were:

  • Automatically register members marked with a modifier.
  • Member name as a string
  • Ability to modify a member
  • Ability to compare the type in runtime


My implementation

The first step was setting up a class that would hold all information about each registered class. This class would be my core and all members would be registered to this class for storing.  


I quickly ran into my first issue which was how am I going to modify a member in runtime? I decided to create an integer which stores the memory size of all previously registered types. This way I would be able to take a void* and add this offset to get the memory location of said type.

And this worked, for a bit. I realize that this solution is not feasible as it relied on 2 things to be functioning correctly

  1. All members must be registered and in order
  2. No virtual methods exist in said class


Due to this, I had to come up with a new solution. Instead of keeping a counter of the memory offset, I create a macro that creates a member of said type and then calculates the offset between the class's pointer and the pointer to the member. This would remove both my issues as its calculated once at the start. The macro ended up looking like this

To note is that the variable is allocated on the heap using malloc to make sure the constructor is not being called and free'd to prevent any memory leaks. Another downside to this solution is that every class that has an exposed variable needs to have this Registry as a friend class in case the variable is private. This was easily solvable in our engine as we already had a Component macro required for other engine features. 


This solution worked well until I tried it with a template type. The problem with macros is that they don't understand C++ code in itself and would interpret the comma in the template as the next argument even though it is inside a template.  To work around this I had to declare a new name for it by creating a using-declaration which can be seen below.

The Generator

So far I had to manually add all fields I wanted reflectable to a file myself and that is not a suitable solution. The plan was to have another program scan through all project files and generate this information for me automatically. In visual studio, you can add commands that will run either pre-build, pre-linking, or post-build. With this, I was then able to automatically run my program before I start building my project. 


For my specific use case, I decided to create an empty definition that I would use before all members I wanted to be reflectable. This made my work with the generator simpler. I would simply search the file for the specific keyword created by the define and then read line for line upwards until I find what type the variable is defined in.  The final generated file can be seen below.

Use cases

The first thing I used this data for was automatically generating handles for components in our editor. Using this simple code I can then compare every component field and generate handles accordingly, as seen below. 

The result of this can be seen here. Every handle for each component is automatically generated using the EXPOSED macro.

The second thing I used my reflection on was a binary scene serializer. Just looping through each component and gathering their metadata allows me to save the data stored at each location without having to write any specific save code for each component.

One problem with this solution as-is for most binary serializers is that you need to specify edge cases for all types that are saved on the heap. Example: Pointers, std::vectors, std::maps, and such. To work around this I first run the type through a save method that compares the type to custom save methods for that type. This requires the user to manually define the save and load method for these types but I don't see any 1 solution fits all case for this and therefore didn't go into depth trying to solve this for my project. 

Future work

Even though I am happy with how the project turned out if I were to continue working on this is what I would prioritize.

1. Not all syntax of C++ is accepted by the parser and will be ignored. I would look more into how compilers parse C++ code and find a new solution rather than just comparing strings to different cases. 

2. The system still doesn't take into account inherited classes and cant be compared to them. This leads to exposed values in the base class won't show up in the inspector unless an edge case is made specifically for it.