Zigler: zig nifs for Elixir

Currently just starting out on a new mini-project - getting zig NIFs to run in elixir.

The idea here is to make the zig NIFs be “embedded” in the elixir code, much like an “asm” call in C. It also makes the bridge between the zig function call and the elixir “basically transparent”. Several adapters are provided: i64, f64, string (as a c string or a zig slice), i64 arrays, f64 arrays. Of course it’s not terribly hard to do the unmarshalling of the erlang terms yourself inside the NIF.

Haven’t started documenting yet, but if you want to see how it works, the test/ directory has quite a few instructive examples which are all passing as of zig 0.4.0

Comments and criticism appreciated!

17 Likes

Lol, that’s cool. ^.^

1 Like

Thanks! That means a lot to me. I think zig is a great fit for elixir. Incidentally, do you know of any good references on refrences? I’m a little bit blocked on my understanding of them.

Edit:. Oops, resources, not references

Very cool project…

Do you mean OTP resources? Where you can store native code specific data pointers and so forth? Or something in Zig?

Yep, otp resources. I’m trying to get my head around what the right way is to square them with zig’s opinionated (but optional) allocator semantics. One thing I’m unsure about is if otp resources can be realloc’d, since there is no resource realloc function. Probably not hard to try though, once I figure out how to even make one.

I would just have you allocate a resource with enough space to hold a zig pointer and store that pointer in it. Have the resource deallocation function deallocate the zig object (if you are done with it of course). That’s generally what I do regardless, I rarely have resource be ‘big’, I treat them as basically just a pointer to my internal object or pointer.

So no, you can’t realloc them, treat them as handles and you are good to go.

1 Like

When I try this out locally I don’t seem to have the mix task.

  defp deps do
    [
      {:zigler, git: "https://github.com/ityonemo/zigler.git"}
      # {:dep_from_hexpm, "~> 0.3.0"},
      # {:dep_from_git, git: "https://github.com/elixir-lang/my_dep.git", tag: "0.1.0"}
    ]
  end

Then I ran mix do deps.get, deps.compile

Then

> mix zigler.get_zig latest
Compiling 1 file (.ex)
Generated zig_test app
** (Mix) The task "zigler.get_zig" could not be found

Any clue what I’m doing wrong here?

Edit it seems like only Linux is supported atm and I’m running macOS

that’s super strange, it should work though (just tested the mix task myself on a completely fresh system).

ityonemo@hadamard:~/code$ git clone https://github.com/ityonemo/zigler
Cloning into 'zigler'...
remote: Enumerating objects: 176, done.
remote: Counting objects: 100% (176/176), done.
remote: Compressing objects: 100% (108/108), done.
remote: Total 176 (delta 80), reused 148 (delta 52), pack-reused 0
Receiving objects: 100% (176/176), 33.32 KiB | 897.00 KiB/s, done.
Resolving deltas: 100% (80/80), done.
ityonemo@hadamard:~/code$ cd zigler
ityonemo@hadamard:~/code/zigler$ mix zigler.get_zig latest
Unchecked dependencies for environment dev:
* mojito (Hex package)
  the dependency is not available, run "mix deps.get"
** (Mix) Can't continue due to errors on dependencies
ityonemo@hadamard:~/code/zigler$ mix deps.get
Resolving Hex dependencies...
Dependency resolution completed:
Unchanged:
  castore 0.1.3
  mint 0.4.0
  mojito 0.5.0
  poolboy 1.5.2
* Getting mojito (Hex package)
* Getting castore (Hex package)
* Getting mint (Hex package)
* Getting poolboy (Hex package)
ityonemo@hadamard:~/code/zigler$ mix zigler.get_zig latest
===> Compiling poolboy
==> castore
Compiling 1 file (.ex)
Generated castore app
==> mint
Compiling 1 file (.erl)
Compiling 22 files (.ex)
Generated mint app
==> mojito
Compiling 14 files (.ex)
Generated mojito app
==> zigler
Compiling 5 files (.ex)
Generated zigler app

09:13:22.758 [info]  downloading zig version 0.5.0 and caching in /home/ityonemo/code/zigler/zig.
ityonemo@hadamard:~/code/zigler$ mix test
Compiling 5 files (.ex)
Generated zigler app
...................................

Finished in 4.9 seconds
35 tests, 0 failures

In theory there is no reason why most of this stuff shouldn’t work on OSX, but I need to get a few paths correct, but i have no way to test it. If you would like to try getting it to work on OSX, any help would be appreciated! In the meantime I’m going to put a warning about OS at build time.

1 Like

Yeah, I got it almost working int this test PR https://github.com/ityonemo/zigler/pull/16. It seems like the mix task had a hardcoded check for :os.system. But you’re right, it seems the only real blocker is a path issue.

ok, I will try to generalize the operating system when i get a chance to work on this tonight, will let you know how it goes.

2 Likes

Awesome! If it would be helpful, I’ll happily test things out on my system.

1 Like

I would just have you allocate a resource with enough space to hold a zig pointer

lol i can do that? Wow. Thanks! This will be relatively easy.

1 Like

This looks really cool! Looks like I’ll have to add this to the todo list of things to play with alongside Rustler

I’m going to 0.1.0 on Hex.pm by the end of the month. Currently I’m working on three last major features - seamless integration of zig tests into ExUnit, translation of compiler warnings to reflect the actual location of code, and assembly of zig docstrings into full zig documentation as a part of ExDoc. =D This might take some time since I need to revamp the parser to be not a terrible ad-hoc solution.

However aside from the QoL features it basically “just works”, so I’d suggest checking it out now. You’ll have a much easier time getting up and running with zigler than rustler since in zigler you literally write your zig code inside the elixir.

Also having eyes on MacOS and an expert in Erlang Windows would be awsome, since I don’t have the capability to test those platforms ATM.

1 Like

This looks really interesting!

I wonder if you’ve considered creating a zigler mix compiler? That way we could write .zif files that are compiled as an alternative to sigils? I wonder if that might feel more natural?

Why not both?

I do like the idea of it being like the c “asm” keyword, you can call out zig file dependencies with zig’s @include directive if you need more code than you’d like to ship (and you should do that anyway) (in this case, path relative to the elixir module) and currently “it just works”. The nice thing about making it module local is that it carries with it inferrable information about how to bind it in using erlang’s nif adapters, which you’d have to manually configure if you lose the association with an elixir module.

I probably should also give a common location setting to draw files from.

Sorry if I implied replacing your currently strategy. I mean “in addition to”. @include sounds a good half-way house but I wouldn’t suggest my opinion carries any weight.