Using Smart Cells in an attached runtime

Smart cells usually come from packages and they are registered, here is an example from :kino_vega_lite. So to make the smart cells available in attached runtime you would need to have the corresponding packages as dependencies in your application. That’s why it’s oftentimes a better idea to use the default standalone runtime, connect to the app node and run things on that node using :erpc, while having the ability to install any visualisation packages or smart cells in the notebook, without clobbering app dependencies : )

I’d love to go the RPC route, but I’m trying to figure out how to do that and still have my livebook notebooks written using what looks like vernacular Elixir. Using a simple Ecto example, I want to be able to write

alias MyApp.Data.Schemas.Foo
alias MyApp.Repo

Foo |> Repo.aggregate(:count)

but with :gen_rpc, I’ve only been able to call it as:

alias MyApp.Data.Schemas.Foo
alias MyApp.Repo

:gen_rpc.call(:"myapp@127.0.0.1", Repo, :aggregate, [Foo, :count])

I can’t figure out at all how I would write anything that uses macros.

I really want all the richness and syntax of Elixir with the flexibility of RPC. Do you know if that’s possible?

Yeah that’s the tradeoff of using RPC calls, if you need to use the remote modules heavily it’s a bit awkward.

I can imagine some helper would make it feel more readable, here’s a quick example:

defmodule DistUtils do
  def connect_remote(node, cookie) do
    Node.set_cookie(node, cookie)
    Node.connect(node)
    :persistent_term.put(:remote_node, node)
  end

  defmacro remotely!(fun) do
    quote bind_quoted: [fun: fun] do
      node = :persistent_term.get(:remote_node)

      myself = self()
      ref = make_ref()

      {pid, monitor_ref} =
        Node.spawn_monitor(node, fn ->
          result = fun.()
          send(myself, {:result, ref, result})
        end)

      receive do
        {:result, ^ref, result} ->
          result

        {:DOWN, ^monitor_ref, :process, ^pid, reason} ->
          raise "failed with reason: #{inspect(reason)}"
      end
    end
  end
end

Then connect with

require DistUtils

DistUtils.connect_remote(node, cookie)

and run code on the remote node:

DistUtils.remotely!(fn ->
  Foo |> Repo.aggregate(:count)
end)

(@josevalim if that solution is not sane let me know, or if you have any other ideas : D)

That’s actually kind of the solution I was thinking of. I’m going to experiment with it a bit unless/until Jose pops in and says it’s a terrible idea.

This is actually working great. Thanks so much for the example!

Maybe it’s time for me to play with creating an RPC Smart Cell…

@mbklein there is actually :erpc.call, so there’s no much need for the macro:

:erpc.call(node, fn ->
  Foo.foo()
end)

A smart cell could make it more approachable! The only factor is that the smart cell editor won’t have the regular intellisense (not much issue for modules, since they are likely only available on the remote node anyway, but relevant when using built-in functions).