Hey everyone!
It's been a good week and, thankfully, I finally caught a break in my work on moving UI engine-side. Today's log will get a bit technical, but this is an interesting topic and I'd like to detail my experience with it in the hopes that it may be of use to other developers who face the same decision in their work.
Strap in, it's a long one...
GUI Architecture: Immediate or Retained?
I mentioned in last week's pseudo-log that there are effectively two major approaches to architecting a UI. The two (very different) styles are known as immediate mode (IM) and retained mode (RM). RM is more traditional and probably what you're used to if you've used a GUI framework directly from code before. IM, however, has enjoyed a reasonable amount of attention in the past few years, and I have seen more and more developers asking about the viability of it. This is no doubt in part due to Omar Cornut's excellent Dear ImGui library, which has demonstrated to many (including myself) the tremendous power that the IM paradigm offers for creating interfaces with minimal fuss.
Briefly, one can illustrate the difference in approach by glancing at some example usage code. Here's what some typical RM pseudocode would look like:
Code: Select all
g = GUI.Canvas()
g.add(GUI.Window("Universe Configuration")
.add(GUI.Checkbox("Infinite Mode").bindTo(&config.infinite))
.add(GUI.Slider("Average Systems Per Region", 10, 1000).bindTo(&config.regionSize))
.add(GUI.Slider("Average System Connectivity", 1, 10).bindTo(&config.connectivity))
.add(GUI.GroupHorizontal()
.add(GUI.Button("Create", createUniverse))
.add(GUI.Button("Cancel", cancelUniverse))
)
)
...
g.update()
g.draw()
Again, that's just pseudocode, and there are a million ways to structure such APIs for convenience, but the general idea is that interface pieces are created like objects, and we control the UI by calling various functions on those objects. RM style is very much a classic, object-oriented approach to UI.
On the other hand, have a look at what IM looks like:
Code: Select all
GUI.BeginWindow("Universe Configuration")
GUI.Checkbox("Infinite Mode", config.infinite)
GUI.Slider("Average Systems Per Region", 10, 1000, config.regionSize)
GUI.Slider("Average System Connectivity", 1, 10, config.connectivity)
GUI.BeginGroupHorizontal()
if (GUI.Button("Create")) createUniverse()
if (GUI.Button("Cancel")) cancelUniverse()
GUI.EndGroup()
GUI.EndWindow()
At first glance this may look similar, but, in fact, the IM approach could hardly be any more different! Here we are not dealing with objects. Instead, we're just making function calls. An immediate mode interface is inherently interwoven with the interface's data and logic. No widget demonstrates this more aptly than a simple button, which is perhaps the most famous example of the difference between IM and RM. Whereas an RMGUI button typically involves setting a 'callback' function (which the interface will call if the button is pressed), an IMGUI button is actually a function that returns true if pressed.
To really appreciate the massive ramifications of the choice between RM and IM requires that you implement a significantly complex interface in both styles. Luckily, I have done so, and will therefore lay out the most prominent pros/cons that I have found in my own interface-building work. Unfortunately, it turns out to be the case that both approaches come with very real pros and cons, and neither, in isolation, is ideal.
Immediate Mode
Both the strengths and weaknesses of IMGUIs come directly from the fact that a vanilla IMGUI does not exist separately from the data that it controls and the logic it executes.
Pros
- UI code is extremely concise, easy-to-understand, and quick to write
- The UI does not need any 'extra information' -- sliders/checkboxes/radio groups don't need pointers to the data they represent, buttons don't need callbacks for the actions they trigger
- Changing data never requires special logic to inform the UI about changes, making it effortless to build UI for dynamic data that changes frequently (like game data!)
- Easy to use from scripts; no special binding code necessary
Cons
- Automatic layout is heavily-restricted
- 1-frame delays are common and sometimes unavoidable; can result in 'popping' and added input latency
- Higher CPU usage
Now, please note that each of these cons is very sensitive to implementation details and requires a lot more explaining to understand the full story, so please don't take the above list as a good "generalization" of IMGUIs. Especially the last point -- I have seen a great deal of misinformation about immediate mode performance characteristics online. Much of the difficulty in IMGUI implementation is concerned with mitigating these cons in various ways. That being said, it is fair to say that complex layout is a rather fundamental problem for a traditional IMGUI implementation.
Retained Mode
Again, the strengths and weaknesses of RMGUIs are a direct consequence of the fact that an RMGUI is built from first-class objects that exist in and of their own right.
Pros
- Automatic layouts are easy and can be made arbitrarily-complex
- No frame delays, minimal input latency
- Minimal CPU usage (predicated, of course, on a well-optimized implementation)
Cons
- UI code is often verbose
- UI must have knowledge about the data it is displaying and must understand how to trigger functionality -- pointers and callback mechanisms are typical
- Users must take special care to ensure the UI stays in-sync with data -- complex, frequently-changing data requires significant consideration to work properly
- Usage from scripts requires special consideration; UI must know how to access script data & functions
Stuck Between an Asteroid and a Hard Place
Looking at the pros/cons of each paradigm, it's not hard to see that IMGUIs and RMGUIs are effectively polar opposites. Where one excels, the other lacks, and vice-versa. It is the absolute epitome of a difficult and highly-consequential trade-off.
Last week, I implemented an IMGUI in our engine. It was my first time implementing an IMGUI, and, while I understood the cons beforehand, I didn't know how they'd pan out in practice -- how well could I mitigate them? The answer turned out to be: quite well, but still not well enough. In particular, the restrictions on automatic layout forced a no-win choice between having to litter my UI code with fixed sizes, or having to accept noticeable, 1-frame-long pops / delays. It is an absolutely fundamental limitation of a standard IMGUI. I believe I used the word "heartbreaking" in one of my posts last week, and it was not an exagerration. The IM paradigm affords such incredible easy and clarity in creating game UI, yet the drawbacks were too much to stomach.
On the other hand, I've implemented RMGUIs many, many times before. I'm more than familiar with those cons. The added complexity in using RMGUI from script is, for me, something to avoid at all costs. After all, the impetus for this recent effort to move our UI code to C stemmed from a burning desire to view & interact with gameplay mechanics with minimal pain. For me, the RM paradigm flows counter to that goal!
If only we could have all the things. If only we could immedify our retained mode, or retain our immediate mode. If only.
As you've already guessed, it turns out: we can
Hybrid Mode GUI
Thus far, my discussion of IMGUIs has been rather specific to what I've called a 'standard' or 'vanilla' implementation. That's because, in reality, one can go much, much further under the hood of an IMGUI in order to mitigate or even defeat the stated weaknesses. At some point, the line between immediate and retained can start to blur...and in that lovely gray area lies the answer to all our problems. The paradigm that I will describe to you now could be considered as simply an 'advanced' IMGUI implementation, however, for the sake of clarity, I will call it 'hybrid mode' -- HMGUI
Much of the beauty of IMGUI comes from the friendliness of the user-facing API. Calling a sequence of functions to implicitly map an interface onto a set of data and functionality is simply easier than creating explicit constructs to do so. At the same time, trying to perform standard internal GUI work like layout and input handling without having advance knowledge of the entire interface results in the inherent restrictions discussed above. But suppose we were to 'retain' all necessary information from our 'immediate'-style functions, and defer that internal work to after the entire interface has been specified? It's a winning combination.
The basic premise of an HMGUI is that, each frame, we will build a somewhat-traditional widget hierarchy under the hood as the user is issuing IM-style calls. We'll retain just enough information to be able to perform automatic layout on the UI and to handle input later. Once the user has finished calling into the API, we'll go back through our hierarchy and perform all the standard GUI logic: layout, input handling, and whatever else.
As a consequence of deferring the work, hybrid mode can handle the full gamut of complex, automatic layout functionality that one would expect from retained mode. We can have widgets stretch to fill available area, align themselves within a group, automatically compute group sizes, etc -- all without specifying explicit sizes and without a one-frame delay (one of which would be necessary under a pure IMGUI). We can have selectable widgets like buttons respond to mouse-over immediately, minimizing perceptual latency. We can layer widgets in arbitrarily-complex ways, performing on-the-fly z-reodering without fuss.
So...where's the catch? Surely there can be no free lunch. Well, HMGUI isn't really a free lunch: of the retained, immediate, and hybrid paradigms, hybrid mode takes the most work to implement. That's not surprising when you consider that, under the hood, HM is just a clever mixing of IM and RM, thus requiring much of the implementation work of both. That being said, from the perspective of the user of a hybrid API, it really is a free lunch
And if you know me, you know that's exactly the kind of system I love. Push all of the hard work into the engine/systems, leave the game/application code as clean and simple as possible.One might also point out that, of the three, HM consumes the most CPU time, due to the fact that it involves all of the CPU work of IM plus the layout work of RM. However, in practice, such code can be made so blazingly-fast that the point is moot (especially when written in well-optimized C
). As always, performance or lackthereof is almost entirely the result of the implementation quality.A Few Examples
I haven't implemented the more complex widgets yet, as I focused heavily on core details this week. I also haven't worked much on graphics. As we all know, making things shiny is a beloved hobby of mine, but best saved for...later
Still, even with only basic widgets and fairly rudimentary rendering, a close look at HMGUI already reveals the superiority.Take, for example, my little todo list from last week's IMGUI:
There are a number of annoyances here. Checkboxes aren't correctly aligned with text. That's my fault, not a limitation of IMGUI, but it happened because writing the IMGUI code to manually align and lay things out is quite a tedious and error-prone endeavor. The code that shows this list is littered with 'magic' size constants -- the window width, for example, is a constant and would not grow based on the contents. Again, in IMGUI we have to accept such constants or the frame delay problem. Despite the simplicity of an immediate mode API, trying to achieve a polished, consistent look can quickly turn the code messy.
From this week's HMGUI:
Consistent, polished, and the code for creating it is cleaner thanks to the fact that all the layout work is handled automatically. Everything is aligned, padded, and spaced with precision. I have even swapped the checkbox to right-justified to demonstrate automatic stretching (again, problematic for IMGUI). This window is sized automatically and will grow accordingly should I add new, longer todo items.
A bigger example:
Sorry again for the rough graphics...but the beauty here is in the functionality. This stream-of-consciousness-style test window has more automatic layout going on than you can shake a stick at! This one is really not going to happen in a vanilla IMGUI implementation. At least, I wouldn't want to see the code for it
In HMGUI, however, it's absolutely straightforward. Notice how even the embedded todo list has expanded the checkbox elements slightly due to the fact that the split code view on the bottom is dictating the window's width. It's all in the details! Conclusion
GUI paradigms present a difficult choice for developers. The simplicity of an immediate mode API is tantalizing. Creating interfaces is a breeze, and the resulting increase in productivity should not be taken lightly. On the other hand, retained mode offers precise control over complex layouts that are difficult if not impossible to achieve in immediate mode. By combining the front-end elegance of the IM paradigm with the back-end power of RM, we can, thankfully, have the best of both worlds! Hybrid mode GUI is the way to go
I'm very satisfied to have finally found some success with this work, and I'm glad that I took the time to experiment with a new paradigm, as I would never have come to this solution without knowledge of both. Always a treat when failure leads to reward. Although the feature set of this new HMGUI implementation is still rather slim and the aesthetics quite programmer-artsy, the foundation is layed and the road has been paved. I already have enough power to get back to doing what I wanted to do in the first place: move forward with gameplay interaction. I'll be continuing the implementation of more advanced features as the need arises, so I'm sure we'll be hearing more about HMGUI in the future. For now, I'm happy to call the porting of another major system to C a success, and excited to move back into gameplay work with my new toys
Enjoy your Friday!
