Progress Update: September 2019


#1


Editor on-demand drawing in action

Hello everyone, it’s time for another progress update!

Last month I’ve focused on two major changes/additions that took almost all of my time (meaning a bit less work on bugs and lesser features):

  • Enhanced RTTI
  • On-demand drawing

Both of the features involved massive refactors of their respective systems and I’m actually fairly surprised I managed to get them done in one month. Lets go over what these features are and why are they important.

Enhanced RTTI

This feature is focused on refactoring the RTTI system so it can also be used for networking (and especially the replication system). RTTI system is currently used primarily for serialization when saving/loading data, but I wanted to extend the system so it can also be used for automated replication of objects and data over a network.

Because data size is so important for networking, some changes had to be made to the existing RTTI system:

  • Compression - On the lowest level we want to be able to encode primitives in smaller formats. e.g. a boolean doesn’t require more than a asingle bit, an integer can often be encoded as a var-int (described in a previous progress update), normalized values might require less space than non-normalized ones and so on.

    RTTI system fields have been refactored to use Bitstream class that may be used to write any such common compressed encodings. A special compression flag has been added to BinarySerializer to enable or disable compression. Additionally you can provide a RTTIFieldInfo structure that allows you to customize exactly how a field should be compressed. And you can implement your own forms of compression by specializing RTTIPlainType same as you would for normal serialization!

    Finally BinarySerializer itself was modified so it stores meta-data in a compressed format when the compression flag is enabled.

  • Schemas - So far when you serialized an object, all the meta-data would be stored with every object and field of that object. This ensures the reader can easily figure out the versioning and read older versions of encoded objects. Over the network however this meta-data is a waste of space, as it’s not going to change during a single connection session (under normal circumstances).

    For this reason I have added RTTI schemas. RTTISchema and relevant types provide this same meta-data but as a separate object hierarchy instead of being encoded in the serialized data itself. BinarySerializer::decode() can now accept a RTTISchema as input and use it to derive the relevant meta-data instead of reading it from the serialized data. Similarly you can provide a BinarySerializerFlag::NoMeta flag to the serializer to have it avoid writing the meta-data. The idea is that you would transfer the schema over the network once, store it and then use it for subsequent deserializations, which considerably saves on space.

    Of course if your project ensures all clients must use the same version you don’t even need to do that, as you can just use the existing schema from the local client. But if you use schemas, this also means you can technically run a networked game with clients running on different versions of the client!

    You can use RTTIType::getSchema() to retrieve a schema for a particular type.

The feature still needs some proper testing but this lays a lot of the ground work for the replication system, which will be an exciting thing to finally get working!

On-demand drawing


Editor on-demand drawing in action

The goal of on-demand drawing is to ensure that the editor uses as little power as possible. This is essential for making the editor a commercial product as many developers use laptops where power usage is important, as well as avoiding thermal throttling.

It is of course also benefitical for other users as well, as it means the editor will work on weaker hardware, respond to input more quickly, and your games will also be impacted less by rendering the GUI.

So what does it actually do?

  • GUI caching - Previous version of the engine used to re-draw the GUI every frame. Current version of the engine will instead cache rendered GUI elements per GUIWidget in their own textures. It will then track user interactions (such as button presses, text input, etc.) and trigger re-draw of those textures as required. This means drawing happens ONLY when user interacts with the GUI in some way.

    Even better, user interacting with one GUIWidget will not affect other widgets. So instead of 60 frames per second of full-screen GUI rendering, you now get maybe 1 FPS on a subset of the GUI. We also handle animated GUI elements specially, so they can be re-rendered more frequently without impacting the other static GUI!

  • Camera on-demand drawing - Camera has been updated with an CameraFlag::OnDemand flag. This flag will stop the camera from rendering every frame and it will instead only render when explicitly requested via Camera::notifyNeedsRedraw().

    All editor viewports have been updated with this flag, ensuring their cameras will only render when the user interacts with the scene in some way. While the game is playing, or you’re previewing animation or particles the game will render at full framerate.

  • Avoding swap completely - Finally, renderer has been refactored so that when no 3D geometry is rendered, and no renderer extension reports needing redraw, no swap will happen at all! This means in the majority of cases when you have the editor open, the GPU is doing absolutely nothing - which is pretty awesome compared to previous 60 FPS! (see the screenshot above)

Minor features

Framework

  • GUI render elements refactored with a cleaner approach with less code and virtual calls
  • Added methods to render window to notify it about external events by @hhyyrylainen

Editor

Bugfixes

Framework

  • Replaced type punning with a memcpy to fix a compile warning by @hhyyrylainen
  • RenderTarget now has an update count that increments any time something is drawn to it

Editor

What’s next?

I’ll be continuing the work on networking. When the RTTI refactor is done I need to implement systems to generate compressed object states and send them over the network, as well as system to track object state and send compressed diffs over the network to update object state on clients. There’s also a lot of design issues to work out, but everything should start falling into place once I have these fundamentals in place.

Once I wrap up GUI caching I’ll also be starting on a new feature. It’s looking like it might be some form of a displacement mapping shader, to allow simple geometry surfaces to have more detail. Motion blur shader might also make it in this month.

That’s it for now, talk to you again soon!