A Ramble Filled Topic About Exray Development

Hello, g’day and howdy-doo!

I’ve created this topic to have a space where I can post cool things I’ve been working on in Exray. This initial post isn’t going to be empty, far from it. It’s gonna be how things have been going from 0.6.0 → 0.7.0 (wip), the Raylib update from 5.0.0 → 5.5.0 and a bunch of other junk that’ll razzle and perhaps even dazzle your neurons.

So to start things off, yesterday when I posted the initial “Here’s Exray” topic, about an hour later Raylib shadow-drops 5.5.0. The important changes are mainly for WebAssembly build targets, which made my brain itch a little. I had a look at the current pipeline for Exray and noticed things were looking a little static. Mainly, the precompiled binary blobs which Bundlex uses to generate compile-time OS dependencies. Sure, I could leave it as the stock standard PLATFORM_DESKTOP with no Raygui, no custom frame control and no real options for customization- Specifically in the direction of optimization and logging. However, I decided to take a little bit of a different approach as an experiment.

So the exray pre-precompiler was made. It’s early stages, but the idea is that people can fork this project, build Raylib to their exacting specifications and upload a release tarball to their git. Then, in functions I haven’t written yet, be able to hand args to the Compiler.Exray mix task for the custom URL. This will let people use whatever flavour of Raylib they want, even stripping out all the functionality and just having Window functions and Raygui in an ANDROID build target. Or even the N64 as a build target, I think- Raylib added new targets. However, actually packaging that to hand to an N64 is a totally separate issue which I won’t dive into because I’ve got other priorities.

I tinkered with Livebook and to my absolute shock it actually compiled and ran. Granted, it took a little strong arming to mock a project definition into existence, but I figured it out in the end.

Here’s an example .livemd where you can just open and close a window. Nothing fancy, nothing un-fancy:

Livebook MD Source
    # Exray Livebook Testing
    
    ```elixir
    Mix.ProjectStack.start_link([]) |> dbg
    Mix.State.start_link([])
    Mix.TasksServer.start_link([])
    
    defmodule ExrayLivebook.Project do
      use Mix.Project
    
      def project do
        [
          app: :exray_livebook,
          version: "1.0.0",
          deps: [{:exray, "0.6.0"}],
          compilers: [:exray] ++ Mix.compilers()
        ]
      end
    end
    
    Mix.Task.run("deps.get")
    Mix.Task.run("compile.exray")
    ```
    
    ## Open and close a window
    
    ```elixir
    import Exray.Core.Window
    
    if not window_is_ready?(), do: init_window(500, 500, "Hello World!")
    ```
    
    ```elixir
    import Exray.Core.Window
    
    if window_is_ready?(), do: close_window()
    ```

When I created that livebook and I closed the window twice, knowing it’d crash, I decided it was time I started wrapping the unsafe C in safe C. This has led to me writing out unit tests on expected functionality for the Core modules, ensuring that instead of always returning an :ok we in some instances want to return a :noop due to the fact we’re not running the NIF as expected. For example…

alias Exray.Core.Window
alias Exray.Core.Drawing

# We haven't initialized a window. Let's call begin_draw()
Drawing.begin_draw()
#> :noop

Window.close_window()
#> :noop

Window.init_window(200, 200, "beans")
#> :ok   (window starts!)

Drawing.begin_draw()
#> :ok

Window.init_window(500, 100, "toast")
#> :noop

While the last Window.init_window call returns :noop, I’m looking into ways to increase the ability to make Raylib more distributed, so we can call Drawing, Cursor, Keyboard, physics updates and animation stuff from GenServer which calls back to the main window process with what it’s got.

In summary, busy day. Learned a lot, had a lot to share and this topic is where I’m gonna just scream into the void with out-of-the-box solutions to problems nobody has.

Replies welcome and encouraged! :heart: :coffee:

3 Likes

Nah, I’m not dead; don’t stress mate :slight_smile:

One thing led to another and I’ve created a config-like cmake builder so I don’t have to mess with pre-pre-post-pre compilers of versions with 1 flag, or 2 flags. I got hung up on a bug with functions and context for a good 4 months on top of all the other development time, but right now it’s pretty stable in the development branch.

So in a nutshell, instead of the classic folder hierarchy of…

project_root:
- mix.exs  
- config/
-- nonsense.exs
- native_c/
-- like_a_gorillion_loose_files_and_a_makefile*
- lib/

It’s instead more like…

project_root:
- mix.exs
- compile/
-- my_nif.cmake.exs
- natives/
-- my_nif/
--- my_nif.cpp

So this my_nif.cmake.exs file is cool, because it contains Elixir code that’s kind of like Cmake code that turns into CMakeLists.txt which can compile whatever you throw at it. The following is an example from a testing file I use named record_testing.cmake.exs.

# record_testing.cmake.exs
use Cmakex

cmake do
  set(RECORD_TESTING, "record_testing")
  set(NIFPP, "nifpp")

  project(RECORD_TESTING)

  include_fetch_content()

  fetch_content_declare(NIFPP, [
    :REQUIRED,
    GIT_REPOSITORY: "https://github.com/goertzenator/nifpp.git",
    GIT_PROGRESS: true
  ])

  fetch_content_make_available(NIFPP)

  add_library(get(RECORD_TESTING), ["\"test.cpp\""])

  target_include_directories(get(RECORD_TESTING), [:PRIVATE, "${nifpp_SOURCE_DIR}"])
end

So this file here, when we run mix cmakex.natives in a project that has a cmake.exs or dependencies that have cmake.exs files is come through and build out a CMakeLists.txt files based off of the contents of the cmake code block. The nifty thing is, it allows us to stamp out all of the boilerplate for NIF paths with 0 issue since they’re all allocated from Elixir itself, meaning if the user is on Windows their paths will just be correct when they generate the cmake to compile their file (hopefully). What the above would turn into is something like this:

cmake_minimum_required(VERSION 3.10)

# REGION:Elixir & Erlang Environment
# Add Mix environment configurations
set(MIX_TARGET host)
set(MIX_ENV dev)
set(MIX_BUILD_PATH #snip#)
set(MIX_APP_PATH #snip#/cmakex/_build/dev/lib/cmakex)
set(MIX_COMPILE_PATH #snip#/cmakex/_build/dev/lib/cmakex/ebin)
set(MIX_CONSOLIDATION_PATH #snip#/cmakex/_build/dev/lib/cmakex/consolidated)
set(MIX_DEPS_PATH #snip#/cmakex/deps)
set(MIX_MANIFEST_PATH #snip#/cmakex/_build/dev/lib/cmakex/.mix)

# Rebar naming
set(ERL_EI_LIBDIR /usr/lib/erlang/usr/lib)
set(ERL_EI_INCLUDE_DIR /usr/lib/erlang/usr/include)

# erlang.mk naming
set(ERTS_INCLUDE_DIR /usr/lib/erlang/erts-15.2.2/include)
set(ERL_INTERFACE_LIB_DIR /usr/lib/erlang/usr/lib)
set(ERL_INTERFACE_INCLUDE_DIR /usr/lib/erlang/usr/include)

# Disable default erlang values
unset(BINDIR)
unset(ROOTDIR)
unset(PROGNAME)
unset(EMU)
# END_REGION:Elixir & Erlang Environment

set(RECORD_TESTING record_testing)
set(NIFPP nifpp)
project(RECORD_TESTING)
include(FetchContent)
FetchContent_Declare(
	NIFPP
	REQUIRED
	GIT_REPOSITORY https://github.com/goertzenator/nifpp.git
	GIT_PROGRESS true
)
FetchContent_MakeAvailable(NIFPP)
add_library(${RECORD_TESTING} "test.cpp")
target_include_directories(${RECORD_TESTING} PRIVATE "${nifpp_SOURCE_DIR}")
target_include_directories(${RECORD_TESTING} PRIVATE "${ERTS_INCLUDE_DIR}")

The newlines and comments part has been turned off for a bit because I refactored it heaps; so the end product is kind of a very functional blob. Shoutout to the elixir_make people for helping me find the correct paths.

So yeah, overall extremely useful for when we want to build Raylib from source with specific flags or from a fork. It would also make additions to the engine more accessible, as changing the source of the project is as simple as changing the code, running mix cmakex.natives to recompile the shared library file. While it’s not something we can easily hot-reload (the .so file), it speeds up adoption time since developers can just git clone any project with NIFs in it and not have to worry about dependency issues or build environment or pathing issues or any one of hundreds of absolute nightmares I have encountered.

With this, I’m finally back to actually writing Exray. I’ve totally re-jigged the project. In short, it’s a from-scratch situation without Unifex. Since I’m using Cmakex, it’s much nicer to segfault because it’s my own fault as opposed to something I cannot account for. Unifex is fantastic, I highly recommend it for projects that don’t need as much control. I plan to make a full engine out of Exray once the bindings are ready again, as my needs changed so have my libs.

I’ll post up Cmakex in it’s current state when I document anything in it, because right now it’s pure chaos. Exray hasn’t been cleaner, though. I’ve made a custom logger that attaches to a GenServer PID so whenever we get Raylib logs, we aren’t blocking STDIO in any way. In terms of memory safety, I’m going the extra mile and try/catching function args with nifpp::get_throws to ensure that if I invoke the argument, it won’t just Segmentation fault (core dumped) on us.

But yeah, lots in the brain and not much on the page. Check in with me whenever you like, if you bothered to read this; I love to chat.

1 Like