Pointers shared across function bounds in Rustler?

Questions & Help Questions
1

Hello! I think the Rust [sparrowdb] crate is a superb choice for modern graph dbs; I’m hoping to package this up in an elixir pkg.

[sparrowdb]: crates.io: Rust Package Registry

Here’s the sample code from their README:

    let db = GraphDb::open(std::path::Path::new("social.db"))?;

    db.execute("CREATE (alice:Person {name: 'Alice', age: 30})")?;
    db.execute("CREATE (bob:Person   {name: 'Bob',   age: 25})")?;
    db.execute("MATCH (a:Person {name:'Alice'}), (b:Person {name:'Bob'}) CREATE (a)-[:KNOWS]->(b)")?;

    // Who does Alice know? Who do *they* know?
    let fof = db.execute("MATCH (a:Person {name:'Alice'})-[:KNOWS*1..2]->(f) RETURN DISTINCT f.name")?;
    // -> [["Bob"], ["Carol"]]  (Carol is a friend-of-friend)
    let _ = fof;
    Ok(())

I can think of some simple changes to make this more approachable as an elixir module. Primarily, loading the db variable should be done in one function call, and each call to execute should be another function call.

I’ve come across this in another rust db package; how does one keep a reference to a rust object, such as db, loaded from a rustler function call? How do future rustler-bound functions refer to the same rust object (db) without leaking or cleaning up the memory address?

I’m new to Rust so I probably need to learn some lifetime attribute concepts. If there are common themes from other rustler packages that I can learn from, I’d be thrilled to find something that I can repurpose across the language boundary.

2

That’s a resource in rustler. Have a look at this thread: Rustler return and pass reference from Rust to Elixir and back to Rust

3

At the risk of forking another thread to discuss the same problem, I’m including my progress from today. There seems to have been a bunch of discussion on this before!

In my case, I’m bouncing between two different errors as I decide if I need to impl Resource or if I need to make a new type that mimics GraphDb with the necessary trait (seems likely).

base code:

use rustler::ResourceArc;
use rustler::Resource;
use rustler::{Env};

use sparrowdb::GraphDb;

rustler::atoms! { ok, error }
rustler::init!("Elixir.Spare", [open]);

enum Response {
  Success(ResourceArc<sparrowdb::GraphDb>),
  Failure(String),
}

// #[rustler::nif(schedule = "DirtyCPU")]
// #[rustler::nif(schedule = "DirtyIo")]
#[rustler::nif]
fn open(base: &str) -> Response {
  match GraphDb::open(std::path::Path::new(base)) {
      Ok(graph) => Response.Success(ResourceArc::new(graph)),
      Error(e) => Response.Failure(e.to_string())
  }
}

I may be soooooo off here, this is a hodgepodge from combining examples.


error[E0277]: the trait bound `GraphDb: Resource` is not satisfied
  --> native/spare/src/lib.rs:20:54
   |
20 |       Ok(graph) => Response.Success(ResourceArc::new(graph)),
   |                                     ---------------- ^^^^^ the trait `Resource` is not implemented for `GraphDb`
   |                                     |
   |                                     required by a bound introduced by this call
   |
note: required by a bound in `ResourceArc::<T>::new`
  --> /home/calliope/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/rustler-0.38.0/src/resource/arc.rs:39:8
   |
39 |     T: Resource,
   |        ^^^^^^^^ required by this bound in `ResourceArc::<T>::new`
...
43 |     pub fn new(data: T) -> Self {
   |            --- required by a bound in this associated function

error[E0277]: the trait bound `GraphDb: Resource` is not satisfied
  --> native/spare/src/lib.rs:20:37
   |
20 |       Ok(graph) => Response.Success(ResourceArc::new(graph)),
   |                                     ^^^^^^^^^^^^^^^^^^^^^^^ the trait `Resource` is not implemented for `GraphDb`
   |
note: required by a bound in `ResourceArc`
  --> /home/calliope/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/rustler-0.38.0/src/resource/arc.rs:27:8
   |
25 | pub struct ResourceArc<T>
   |            ----------- required by a bound in this struct
26 | where
27 |     T: Resource,
   |        ^^^^^^^^ required by this bound in `ResourceArc`

Then, if I include this impl Resource, the error changes:

#[rustler::resource_impl]
impl Resource for sparrowdb::GraphDb
{
  fn destructor(self, env: Env<'_>) {
    drop(self)
  }
}

error[E0117]: only traits defined in the current crate can be implemented for types defined outside of the crate
  --> native/spare/src/lib.rs:26:1
   |
26 | impl Resource for sparrowdb::GraphDb
   | ^^^^^^^^^^^^^^^^^^------------------
   |                   |
   |                   `GraphDb` is not defined in the current crate
   |
   = note: impl doesn't have any local type before any uncovered type parameters
   = note: for more information see https://doc.rust-lang.org/reference/items/implementations.html#orphan-rules
   = note: define and implement a trait or new type instead
4

The rustler API seems to have changed a bit. Functions now get auto registered when defining the #[rustler::nif] macro.

You can only define traits for structs when you either define the trait or define the struct. But here the trait is defined by rustler and the struct defined by sparrowdb. So we have to wrap it in our own type.

rustler also converts a Result type automatically to a tagged tuple in Elixir. Either {:ok, _} or {:error, _}.

use rustler::{Resource, ResourceArc};
use sparrowdb::GraphDb;

rustler::init!("Elixir.Spare.Native");

struct GraphDbResource(GraphDb);

#[rustler::resource_impl]
impl Resource for GraphDbResource {}

#[rustler::nif]
fn open(base: &str) -> Result<ResourceArc<GraphDbResource>, String> {
    match GraphDb::open(std::path::Path::new(base)) {
        Ok(graph) => Ok(ResourceArc::new(GraphDbResource(graph))),
        Err(e) => Err(e.to_string()),
    }
}

#[rustler::nif]
fn execute(graph_resource: ResourceArc<GraphDbResource>, cypher: &str) -> Result<String, String> {
    let db = &graph_resource.0;
    match db.execute(cypher) {
        Ok(result) => Ok(format!("{:?}", result)),
        Err(e) => Err(e.to_string()),
    }
}

With the example from the sparrowdb README:

iex(1)> {:ok, db} = Spare.Native.open("social.db")
{:ok, #Reference<0.3497539546.3541958666.218184>}
iex(2)> Spare.Native.execute(db, "CREATE (alice:Person {name: 'Alice', age: 30})")
{:ok, "QueryResult { columns: [], rows: [] }"}
iex(3)> Spare.Native.execute(db, "CREATE (bob:Person   {name: 'Bob',   age: 25})")
{:ok, "QueryResult { columns: [], rows: [] }"}
iex(4)> Spare.Native.execute(db, "MATCH (a:Person {name:'Alice'}), (b:Person {name:'Bob'}) CREATE (a)-[:KNOWS]->(b)")
{:ok, "QueryResult { columns: [], rows: [] }"}
iex(5)> Spare.Native.execute(db, "MATCH (a:Person {name:'Alice'})-[:KNOWS*1..2]->(f) RETURN DISTINCT f.name")
{:ok,
 "QueryResult { columns: [\"f.name\"], rows: [[String(\"Bob\")]] }"}

Happy hacking!

5

Amazing! I copied your code and was able to spin it up on a greyhound bus. I found a helpful example also in ryugraph_ex, and I’m bringing in pieces that seem useful. sparrow seems to miss some of the cypher spec, so I could perhaps make a benchmark using the opencypher cucumber specs. I’d like to also produce a Neo4j-compatible harness, to make use of their explorer and visualization ecology.

For people going through the same burrow, my first rabbit hole with rust embedded databases was fjall crates.io: Rust Package Registry - I assume this approach will be useful there also.