Hello all!
Well, Februrary turned out to be a bit of a ‘low throughput’ month sorry. It kind of turned into a ‘research’ month as I started a few things but ended up filing them under ‘later’ as they were obviously going to take considerably more planning and thought than I had originally envisioned. Still, bugs were fixed, tweaks were made etc so it wasn’t a complete washout.
First things first though, If you haven’t seen user Ethernaut’s awesome plane demo, please check it out now! The URL is:
http://monkeycoder.co.nz/bananas/Plane/Plane.html
Ok, it’s not a whole game or anything, but it’s getting there!
There is also a new prebuild of monkey2 up at the itch.io for patreon supporters, I’ll be sending the password out after I’ve done this post.
Anyway, to make up for the lack of much apparent progress, I thought I’d attempt to layout a roadmap of sorts – like you get on ‘normal’ projects(!) – so here’s a bunch of things I plan to be concentrating on in the near future.
Mojo3d
* Scene serialization. This is something Ethernaut has been hacking away at over the last year or so, which has mainly involved him bugging me on and off for improvements to the monkey2 reflection system – which has been kind of a PITA, but I also knew/felt it would be cool stuff to add in the long run, and indeed it has been.
Basically, the idea here is pretty simple: to implement Scene.Load() and Scene.Save() functions – how tricky could that be?!? Well, pretty tricky as it turns out. The main problem here is that ‘literal’ serialization, where you just write out all objects and their properties etc doesn’t really capture how the scene nodes ‘came into existence’ in the first place, and this is a huge contributing factor towards how a scene should be saved/loaded.
For example, there is a lot of difference between Model.CreateBox() and Model.Load(). Yet they both result in a model being added to the scene, but literal serialization can’t really handle this – unless you’re trying to create a file format or something. The solution I’m working on here is to implement the serialization of ‘invocations’ (ie: function calls), and to remember how each entity (ie: scene node) was created by storing a ‘constructor’ invocation along with each entity. Then when you save the scene, these serialized ctor invocations are written to file along with the actual entity properties. Probably sounds horrific, but it will hopefully be completely transparent as mojo3d will quietly create these invocations for you when you call Model.CreateBox(), Model.Load() etc – plus you’ll be able to write your own too (somehow!).
In addition, a node’s ‘initial state’ will be stored once it’s created, so that when it’s saved only the difference between its current state and its initial state will be saved. So there will be no need save the Mesh property of a model if it hasn’t changed since it was created, which will be the case if a model was created via Model.Load.
All this is kind of only useful if you’re writing an editor or something – but that’s cool, I want people writing editors for mojo3d and the ability to just be able to go Scene.Save() would certainly make creating editors a LOT easier. Besides, I still want to do a Maplet 2 one of these days…
* RenderQueue tweaks. I have a few tweaks and improvements I want to make to mojo3d’s internal ‘render queue’ system. These include such things as the ability to cull render ops against the camera frustum, better sorting of transparent materials (ie: sort them by local ‘material’ bounds origin instead of just entity origin). I also want to streamline the ‘render op’ system a bit. Currently, if you want to add render ops when a scene is rendered (see the ‘morpher’ mojo3d test), a RenderOp instance is created on the fly every frame using ‘New’, which is kind of a cool testament to monkey2’s kick ass incremental GC, but unnecessary most of the time. If you’re just updating vertices in a vertex buffer – or even just doing nothing in the case of a static mesh – you should be able to just reuse the same render op. I will definitely be cleaning this up at some point.
* Material system cleanup. The material system in mojo3d is actually working very nicely IMO, but it’s likely to be a bit impenetrable to anyone without a pretty good understanding of pbr renderers! I don’t think it’s gonna take too much work to make it more user friendly though, I just need to stick all the gnarly glsl stuff into an ‘include’ file and it should all look a lot more comprehensible! PostEffect shaders are another story and I really need to do more work on these, but that’s another story…
* Other stuff, in no particular order: A terrain material, customizable particle emitter, textured lights… Also, lights still aren’t being culled AT ALL, which will need to be fixed, especially for the forward renderer/mobile.
MojoX
Next up for mojox is some kind of ‘PropertyView’ system for editing properties either via reflection or explicitly via use of an ‘IntPropertyView’ or something. Ultimately, the goal is to be able to provide something like ‘EditObject’, which will show you an objects properties and allow you to edit them. If nothing else, this should make ‘Options’ dialogs a lot nicer to write!
Discussions are also underway for how to implement something like ‘OnPropertyChanged’ so that GUIs can ‘sync’ with changes to properties in realtime. Note that it is already quite easy to implement something like this explicitly, eg:
Class Thing
Field DataChanged:Void()
Property Data:Int()
Return _data
Setter( data:Int )
_data=data
DataChanged()
End
End
Function Main()
Local thing:=new Thing
thing.DataChanged+=Lambda()
Print “Data changed to:”+thing.Data
End
thing.Data=10 ‘prints “Data changed to:10”
thing.Data=20 ‘prints “Data changed to:20”
thing.Data=30 ‘prints “Data changed to:30”
End
(sorry about formatting, Patreon’s ‘post’ system is ridiculously primitive!).
However, there is currently no way to just ‘plug’ the Data property (or even DataChanged event) into a gui element, which is really the goal in the long run.
So to start with at least, app’s will have to manually sync views to their data. In the case of reflection based property views, views will hopefully be able to auto ‘poll’ properties for changes via DeclInfos using timers.
A ‘PropertyChanged’ style system is definitely under active research for the future though.
Mx2cc
Things are getting a little gnarly inside the actually compiler, but instead of rewriting the thing from scratch as is my usual habit, I’ll just fix what’s there this time around! In particular, the ‘Translator’ component has become well out of hand and needs to be split up into bits as it’s just so unwieldy. In fact, there are already a base Translator_CPP and extending Translator classes (or vice versa), but overtime the distinction has become pretty meaningless – it’s really just one big mega class in reality.
I think what I’ll do here is try a ‘component’ approach, ie: have a core Translator class, and a bunch of component-style classes like TransBuffer, TransDeps, TransGC, TransMunger, TransValue, TransType, TransStmt etc. The components will all be able to ‘see’ each other via the core Translator class so this doesn’t really change the guts of how anything actually works – all I’m really after here is a way to cleanly chop a ‘mega class’ up into bits to make it more manageable. If any ‘real’ programmers have any advice here, please feel free to leave a comment!
The benefit to having a cleaner mx2cc code base is of course that it will make it easier to add new features! There are a mounting number of things I want to fix (global initialization order) and add (return type based function overloading) which I feel are quite possible to do, but are just made a lot harder due to mx2cc’s current state. Another thing I would like to try and that probably wont be as hair-raising as it sounds is a ‘header-file-per-class’, which could theoretically considerably reduce build times.
Further adventures in threading
After the inevitable, eagerly awaited ‘how do I load an image asynchronously’ question on the forums, I tucked my shirt in (metaphorically), forked monkey2 (locally) and had a quick hack at writing a ‘threaded’ version. And about a week later admitted defeat – for now! It is certainly possible, the big question is how to do it efficiently, and the main problem here is probably monkey2’s incremental garbage collector.
Incremental GC involves the use of what’s known as a ‘write barrier’ in compiled code – a small snippet of code that is executed each time a (heap) variable is assigned. This becomes a bit of a problem with threading as the write barriers must be synchronized so that they don’t interfere with each other, which is even more overhead. After trying several ‘fast’ approaches involving fancy pants spinlocks, ‘bemaphores’ (google it!) and a few other ‘as seen on TV’ ideas, I eventually had limited success with a (slow but safe) plain old recursive mutex. However, this mutex was being locked in a LOT of places and while I didn’t do any real speed tests, I’ve already decided there are better ways to go, but they WILL involve some serious mx2cc twiddling, which is even more motivation for an mx2cc cleanup!
So the state of threaded monkey2 is still up in the air – it’s something I still want to do, but it will have to wait at least until the mx2cc code base is back in fighting form. In the meantime, I am tempted to hard code some async image/sound loaders in using std c++ threads. This would likely involve moving some of the ‘stream protocol’ stuff (eg: “asset::myfile.png” ) to c++ too, but I don’t think that’s necessarily a bad thing anyway.
Bye!
Mark
(Hmmm, that wasn’t too bad, I might just resurrect the site roadmap page…)
