3) ElixirConf 2017 - Elixir Native UI - Boyd Multerer



Love to see the interest. Was just reading the above thread…

Think of it is that it is a retained-mode model specifically designed to take advantage of OTP.

All UI scenes are GenServers. Scenes can reference (embed) other scenes. Input is all message passing/filtering/handling. This also means that developers can build “components”, which are just scenes/GenServers that can be reused by other devs. Nice for extensibility.

Drivers (rendering) are abstracted away from the scenes so that they can be swapped to run on different hardware without re-writing the UI logic. This allows me to run my app on Mac to debug, change a target, and build firmware for a raspberry on Nerves. Everything looks and works the same across the two.

The driver model has also been really good for getting remoting working. More on that later…

A big thing I’m going for is reliability. Devices in the field are going to encounter errors, whether it be code, unexpected data, malfunctioning sensors or whatever. This was harder to get right than I expected, but now any piece of UI can crash and things recover sensibly into a good state.

Multiple people have said they can tell it looks like something someone with game experience would make. Partly that was to get my dependencies down to just OGL. But also because transforms are just a great way to manipulate things.

As far as overall status goes, I’ve stopped adding features and am now focused on the large amount of supporting things around it. (Including stripping out the code from various dead-ends and experiments). I want the API to be pretty stable before it goes out to lots of people.


One more thing…

On ports vs. NIFs. I landed on Ports. The performance is fine for what I’m trying to achieve. I did prove to myself that I could get it working as a NIF in the future under Linux and such, but losing OTP is a heavy price to pay in the event of an error. Too much C code in the render and not enough benefit to make it a NIF.

Besides, it is really cool to force a crash in the render and just watch everything recover.

By the way, pretty sure it is not possible to do it as a NIF on the Mac. Turns out you can only ever open a window under OSX on the main thread of an application. That’s something that dates back to the NeXT. Trying to do so in a threaded NIF is a fast way to bring down OTP on a Mac.


One more one more thing.

The math library is a NIF. It does a lot of matrix math and that warranted being a NIF.


Just WOW, this is how exactly I imagined creating some UI apps. Lots of processes with parent monitoring process. When entering Elixir it was only one thing I wanted to have.

For example as a total C++ noob I was thinking about making tabbar and each tab content in separate process and embed it in parent window. I have even asking about inter process communication ways at stackoverflow, but at end I did not have enough knowledge to do it well. Elixir processes, supervisors and all related topics looks much simpler for me and I really wanted to see some library which could be written in such awesome language.

Amazing. I believe such comfort would be very more than welcome in lots of languages.

Hmm, not sure if I get it. Do you mean something like builtin VNC for every app? Ahh, after small time I probably got it. You want to work always on same machine and debugging it remotely, right?

So you are working on 1.0.0-rc.0 version?

If you don’t mind I have some questions:

  1. If I well remember it allows to work with OpenGL. Does it have any limitations (except Linux video card drivers)? Did you think about Vulcan?
  2. Will you create some elements already or it should be a part of library which uses your project? If so then will be there auto and percentage height/width support available or we will need to implement it?
  3. I have watched one video and you are using there RGB(A) format and named colors. Did you think about support other color representations?
  4. Not really remember in which language and GUI library (probably C++/Qt) there was a way to embed one window into another. I was imagining about adding mplayer/mpv support in my apps. Do you have any idea if it could be even possible for us?

If you will find some time I would also say about one problem that I’m facing in all GUI libraries I know about. I have always a problem with finding easy solution for one thing … I always wanted to simply place a tab bar in title bar like lots of browsers have. Also I wanted to somehow automatically fetch icons and styles (like color and background color) based on current desktop environment (GTK+ or QT theme) and/or currently used distribution/operating system. In short I wanted to make window looks like native, but with tabs + optionally move it on bottom, left or right side (as user wishes). No matter which language and library I was looking for I always needs to create a custom frame-less window with basically everything including tab animations and styles in worst cases. All of this just for moving, resizing one bar and adding tabs into it. For an C++/Qt (or any other combination) beginner it was lots hours of really hard work. I though that GTK+ could help me a lot with their special API for elements in title bar, but just simple question shows me that’s not so easy topic. At the end I did not found even a limited solution (let’s say adding tabbar, but without ability to move and resize). I’m even considering paying someone for helping me with this part and making a simple library for generating windows with such bar.


I ended up using NanoVG deep in the renderer. It can support the newer cards, so there will be an upgrade path for that over time. Worrying about OpenGL/Vulcan/Whatever, however, isn’t really surfaced up to the app and would only be in a faster, or new driver.

There is a basic set of elements in it already. Enough for me to build simple UIs with.

Overall… This first version is very much aimed at fixed-screen embedded style devices. I’m not trying to make it look like a native app in a windowed host environment. Could be done, but isn’t a goal.

On the other hand, if you want to move around a UI element, that’s just a matrix transform (not as scary as that sounds) and works really well.


hmm, NanoVG as well as I remember supports only 2D vector drawing using OpenGL, but maybe something have changed from last time I saw this project

Will be there a chance for 3D graphics in future? Of course I’m not planning creating AAA game, but maybe there would be something to do with 3D animations.

About “framework” part. From video I did not noticed any “magic”. Is it as same framework like Phoenix is called (people says that’s basically works more like library than framework).

ok, I see

sure, anyway nice to see that it will work on so much environments

I believe it’s less scary if you are playing with OpenGL-like API “everyday”, so if you are creating button from scratch then it should not be a big deal to also draw some animations. From beginner side it’s much more trouble especially when you need to play with things you should not to like frame-less windows for custom title-bar.

I will definitely place a job offer at least to create library with what I have described and maybe there would be something more also like card game generator + launcher or even also advanced reader which I have in my crazy mind. :smiley: