Progress Update: May 2019


#1


Hello everyone, progress update for the month of May is here!

The big feature that was being worked on this month was MoltenVK support (Vulkan on macOS), along as macOS support for the editor in general.

The second focus was getting the editor stable. There have been huge strides done this month, with over 40 bugs squashed, both big and small. Some particularly nasty bugs were part of the new async scene loading and async asset import systems, which were both high risk changes but haven’t been properly tested yet, so it’s nice to get those issues out of the way.

We’ve been getting an influx of new editor users as things are getting more stable, and some internal projects are ramping up, which means there have been a lot of new bug reports, as new systems are tested and strained. Ultimately this is a good thing as we’re making the framework as a whole a lot more stable and scalable, but it also means there is less time to work on new features until the issues are resolved.

Major features

Editor macOS support


Editor running on macOS

The plan is to get the editor stable on Windows, Linux and macOS. macOS was the platform that has so far received the least love, and in fact was not even advertised as a supported editor platform.

But I’m happy to announce this was rectified this month, and the editor will now compile and run on macOS under OpenGL!

However as soon as it was running we discovered some issues with the OpenGL render backend on macOS, on certain systems. Those would require non-trivial fixes, and instead of wasting a week or two resolving those problems, the decision was made to move from OpenGL to Vulkan. Since Apple deprecated OpenGL this would need to be done eventually anyway, but it also comes with other cool benefits - read below.

MoltenVK support


LowLevelRendering example running on Vulkan on macOS

MoltenVK is a wrapper that allows Vulkan applications to run on macOS, using their Metal render backend. So far we’ve relied on OpenGL on macOS - but recently Apple has deprecated OpenGL. This itself isn’t much of a problem (yet), but the bigger problem is that the latest supported version is OpenGL 4.1. This is a very old OpenGL version that doesn’t support modern features such as compute shaders, unordered read/writes, arbitrary buffers, etc…

This ended up being a huge time-sink when developing the renderer, as modern techniques would often require alternative implementations just so we can run them on macOS, using a more old-school approach.

It’s needless to say that having to write two versions of rendering features is a lot of extra work. Which is why the move to MoltenVK will save a lot of time in the long run, and be beneficial to non-macOS users as well, especially as new renderer features start being added.

Note MoltenVK is not fully supported yet, but a lot of work has been done on that front and it’s pretty close.

Minor features

Framework

  • PhysicsQueryHit now returns the original mesh’s triangle index for easier lookup - by @mworchel
  • Empty resource handles can now be created from nullptr
  • RTTI system was extended in preparation for network replication. Each field can now contain RTTIFieldInfo structure with various meta-data (including replication flags and options)
  • UPtr are now significantly nicer to use due to a switch to a deleter struct - by @mworchel
  • Added an option to build and install all render API backends at once (useful for runtime switching between multiple backends)
  • Add a BinaryCompare class that can compare two IReflectable objects without needing to serialize them first
  • RTTIPlainType specialization for std::list was added - by @ataulien
  • PlayInEditor refactored for nicer API and auto-exported to C#
  • Added documentation on how to update built-in assets, generate script bindings and generate documentation
  • ClearLoadStore material now has support for clearing structured buffers
  • Vulkan now supports separate sampler + texture states, meaning shaders can now waste less register space on needless samplers
  • Linux now suppport externalWindowHandle for Window creation (useful for interop with SDL and similar) - by @hhyyrylainen

Editor

  • Awesome new look for the About window - by @guestnone
  • Degree and Radian inspector fields now support sliders
  • All component inspectors have been reworked using the new inspector drawing API, and all are now fully or 90% auto-generated, making maintenance for a huge portion of the editor much easier
  • Added a camera preview window for previewing currently displayed game camera in Scene view - by @jayrulez
  • New internal issue tracking system to help manage all incoming bugs, prioritize and schedule them better

aboutWindow
@guestnone’s new About window

Bugfixes

Framework

  • Fix a crash when attempting to access and out-of-bounds material on a Renderable object
  • Default sphere reflection probe radius is now 1
  • Custom threading primitives moved to ‘bs’ namespace to avoid polluting the global namespace
  • Null physics library now gets installed properly
  • Fixed invalid binary diff generated when array entries of the new state contain null objects
  • Fixed memory corruption & crash on Linux due to invalid EDID parsing
  • Fixed culled animations returning garbage data and corrupting the animation
  • Fixed an issue where array diffs would not get recorded if existing data was the same, but array sizes differed
  • Fixed incorrectly registered C# interop method GUIElement.Internal_GetVisualBounds - by @jayrulez
  • Multi-threaded debug messages to Visual Studio output window will no longer be garbled
  • Occlusion queries on Vulkan now use the correct query type
  • Pipeline stages on Vulkan now use the correct stage enum, fixing some potentially weird issues caused by it
  • Loading dependencies for a resource that is already being loaded will no longer sometimes cause a deadlock
  • Resource load will now properly notify the resource handle when load finishes, even if it failed
  • Waiting for task completion can no longer deadlock if there are many other tasks executing
  • Core objects are no longer initialized on the internal queue, as it was causing ordering issues with per-thread queues
  • Fixed a nasty issue where fetching MonoClass from a C# type would sometimes fail
  • Changing component active state for non-always-run components no longer places them in an invalid state list
  • Casting RRef<T> with Resource no longer returns RRefBase, so it can be cast the same as RRefs with specific resource types
  • Command queue flushing and queue reuse re-worked so it can no longer cause race conditions
  • Command queue locks are no longer accidentlly shared across threads, causing an exception when two threads try to lock them
  • Stopping components with animation playing, in editor, will no longer cause a crash due to attempting to access a non-existing CBone component
  • Deserialized CRenderable objects now properly register with the resource listener and will get updated when their resources change externally.
  • Reimporting a texture while its active in scene, will now properly refresh the renderable and GPU parameters

Editor

  • Creating a new SceneObject from the menu item no longer causes a crash
  • When switching between Custom and built-in Shaders in the Material inspector, don’t lose the referenced custom material - by @jayrulez
  • Fix cursor getting stuck when attempting to drag scene handles on macOS
  • Toggling between docked windows now works properly once more
  • Fixed a crash when selecting multiple objects in the Hierarchy window
  • Array/List/Dictionary inspectors are now properly updated when their underlying objects change outside of the inspector
  • .meta files will no longer be marked modified in your VCS if they haven’t actually changed
  • Project library meta files will no longer continuously accumulate inactive resource entries, bloating import times
  • Correct icons will now render in the Scene view, when multiple icons are present
  • When importing a resource via drag and drop properly display the resource in inspector after import
  • Mesh inspector reimport button now properly displays the in-progress import state
  • Exiting play mode will now restore selection to the last selected scene object(s)
  • Assigning a texture to a Material through the inspector will now properly trigger load for the texture
  • Fixed a non-fatal assert that would sometimes happen on scene load, due to scene handle system not being properly initialized
  • Reimporting a mesh or material will now correctly update the existing instances of those resources in the scene
  • Reimporting a texture displayed in the scene will now properly update the displayed texture
  • Loading a scene while its resources are not yet imported will now properly load those resources in the scene, as soon as they’re imported

What’s next?

The priority remains the editor, and getting it in shape for release. MoltenVK functionality needs to be wrapped up and all rendering features need to fully run on it. A lot of editor issues have been resolved, but there are still more remaining and the goal is to get these done in a month or two. Linux version also needs some love and it should hopefully get in runnable state this month as well.

I’m eager to continue with the networking features for v1.2 of bsf, but this might need to be on the backburner until we get the editor in tip top shape and released, and after that we can continue with new framework features as usual.

Due to some internal project requirements, the work is going to start on a variety of new high-end 3D rendering effects sooner than planned on the roadmap. The existing features remain on the roadmap, but we will likely squeeze in some rendering features in v1.2, while pathfinding and a second batch of rendering enhancements will be released as a separate v1.3.

That’s it for now, stay tuned!