WX is focused more on building traditional UI: windows, buttons, etc. It does support using OpenGL, a drawing toolkit, in its UIs. The interface is super awkward in Elixir, as you have noted.
It currently is, but it sounded like a move to NIFs is planned.