fhunleth

fhunleth

Co-author of Nerves

We’re using Livebook for data analysis and are putting all of our notebooks in a common private git repository. We’ve built up some utility code to help with querying our data warehouse and other things. It’s nice to maintain that code in the same git repository since it’s tightly coupled to the notebooks. I was wondering how others did this, since what we’re doing right now seems “just ok”.

Our current solution is to put the utility code in a little library and Mix.install’ it as a path dependency from each Livebook. The directory structure looks like this:

.
├── notebooks
│   ├── xyz.livemd
│   └── ...
├── sandbox
│   └── ...
├── our_utils
│   ├── lib
│   ├── mix.exs
│   └── README.md
├── README.md
├── template.livemd
└── tutorial.livemd

The notebooks and sandbox directories contain Livebooks organized at various path depths. When creating a new notebook, we copy template.livemd which has a good Mix.install line and a couple generic sections. The Mix.install includes the our_utils library.

This mostly works, but could be improved.

For example, here’s the dependency snippet to find the our_utils directory since it could be at any depth:

base_dir = Stream.iterate(Path.expand(__DIR__), &Path.dirname/1) |> Enum.find(&File.dir?(Path.join(&1, "our_utils")))
Mix.install([
  {:kino_explorer, "~> 0.1.20"},
  {:our_utils, path: Path.join(base_dir, "our_utils")},
])

It’s a one-liner and spares us from errors based on different directory depths. It doesn’t work when the notebook hasn’t been saved in a subdirectory of our main project.

We’ve talked about switching the dependency to a github reference, but the branch would almost certainly effectively be main. It would also require switching back to a path dependency every time we work on it. Not terrible, but also seems like it could be better.

I suppose I’ll leave it at that and just see what has worked for others in this situation.

Showing Posts 1 to 2

Eiji

Eiji

How about environment variable?

base_dir = System.get_env("OUR_APP_GIT_DIR")

Simply add this env once to your shell script (like ~/.basrc) and you can easily use it even with git:

git clone url "$OUR_APP_GIT_DIR"

Yes, it’s “extra work” for each team member, but it’s better not only because it’s shorter, but also because it’s using a common way for environment-specific configuration. Also Livebook should support environment variables as well.

fhunleth

fhunleth OP

Co-author of Nerves

Hi @Eiji,

Thanks. An environment variable seems reasonable.

It’s hard for me to argue that any of the “extra work” is difficult in isolation, but somehow it comes together. Your point is well taken, and I’ll try to embrace the environment variable and look for friction reduction elsewhere.

— All posts loaded —

Where Next? Top

Trending in Questions Top

Hellowman
Hello everyone! I am new to coding with Elixir/Erlang. And i am looking for some utilities for fuzzing testing like AFL++. Does anyone ...
New
yarrichar
I’m very new to Elixir, so sorry if this is a silly question: I am wondering what happens to the mailbox of a process when the machine i...
New
pragdave
How do I tell Hologram not to escape HTML when substituting into templates? {Textile.render(@product.long_description)} escapes the HTM...
New
Fl4m3Ph03n1x
Background When trying to execute mix release on a Windows 11 machine for a Phoenix project I get the following error: * assembling mark...
New
mandar1045
I load tested a plain Phoenix chat app backed by Postgres with 100 million messages in it, everything on default settings, to see what br...
New
rlipscombe
I’d like to create a file with 600 permissions (rw-------), but there’s a race condition if I create it and then change the permissions. ...
New
saleyn
Is there a way to import record definitions in the iex shell for pretty printing (similar to the rr command in the erl shell)? This woul...
New

Other Trending Topics Top

dominicletz
Hey there, I haven’t seen a list of useful Elixir specific skills for the popular AI tools yet so I’m starting one here with my first. ...
New
raa-ziq
Hi everyone :waving_hand: I wanted to share something I’ve been building — NowBlind, a random chat app designed to help people discover ...
New
mudasobwa
I fully migrated to my own harness from Anthropic/Gemini and I think it’s time to share it. Welcome DSH, the DeepSeek Harness, fully writ...
New
OndrejValenta
(I just needed to vent somewhere and LinkedIn is full of hope, or hype, I’m not sure which exactly) AI dream has many faces, but general...
New
kip
Localize is the next generation localisation library for Elixir. Think of it as ex_cldr version 3.0. The first version will be released ...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews