Best way to restrict file access to one single Elixir/Erlang process

Hi all,
suppose you have an Elixir process that internally accesses a file. For clarity, let’s say that it’s started like this:

{:ok, pid} = MyGenServer.start_link("path/to/file.xyz")

You want to make sure that only that process can access the file. In other words, you want to prevent the user to create more processes accessing the same file:

{:ok, pid} = MyGenServer.start_link("path/to/file.xyz")

MyGenServer.start_link("path/to/file.xyz") # this should return :error, as the file is already used
MyGenServer.start_link("path/to/another_file.xyz") # this is ok, the file is a different one

Ideally, this should work no matter if the two processes are started as part of the same VM, or in different ones. Something similar to file locking, but working for Erlang/Elixir processes rather than OS processes.

I can think of some ways to achieve this, but they all look a bit cumbersome, and maybe there is a simple solution to this that I am missing.

1 Like

The Erlang :global.set_lock could probably be used to achieve this, but it does not work across VM instances (e.g. if I start two iex sessions).

One solution would be to use both :global.set_lock and a “traditional” flock, but maybe someone has a simpler solution?

Hi @lucaong!

Have you thought about a named global process? You could set one up for each file (the name can be the filename) and these are available across the instances.

From what you shared, it should fit your use case.

I wrote a little bit about this: https://pdgonzalez872.github.io/articles/001_named_global_genservers.html

Hope this helps.

Have you considered using the client side storage? This might scale better…

Hi @pdgonzalez872, thanks for the interesting article :slight_smile:

The problem with named global processes, similarly to global locks, is that they would not prevent me from accessing the same file from two processes if, for example, I open two iex sessions and create a process accessing the file in each of the sessions (without linking the sessions in a cluster).

I know this is not a typical problem in Erlang/Elixir, and I am not asking in order to structure an application. I need this to implement a feature enhancement for a library (therefore @sd4code comment is correct, but not relevant to my case).

If you want to know the actual use-case, my goal is to show a helpful error to users of CubDB if they happen to mistakenly create two DB processes on the same data directory. I want to do something similar to what SQLite does to prevent users to mistakenly start two SQLite processes writing on the same data file, but I need to do that with Elixir/Erlang processes instead of OS processes (as CubDB is native Elixir). CubDB allows concurrent operations on the database, but a single genserver should be responsible of managing file access.

My default solution (which in the end would work reasonably well) is to use a mix of :global.set_lock (to prevent two processes in the same VM writing on the same file) and a “traditional” file lock (to prevent processes in different VMs to write to the same file). I think this is reasonable, but I wanted to check if someone has a better idea in mind.

2 Likes

@lucaong glad you found it interesting, hope you liked my design skills as well :slight_smile:

without linking sessions in a cluster

Very interesting!

Wow, awesome use case. Sorry my suggestion didn’t help, I’m eager to see what others propose here. Hope we see some interesting solutions :slight_smile:

1 Like

This definitely won’t be an Elixir/BEAM question, more of how would you do this via the OS, because that’s the only way to reliably do it. Maybe see how sqlite does it. (Probably a lock file or so I’d guess)

3 Likes

Yes, SQLite uses file locking. I would use both a file lock and a Erlang global name or lock, to restrict access both within and across Erlang VMs.

Thanks everybody! It looks like I was more or less on the right track. I just wanted to make sure that I was not missing some simple solution.

1 Like

So the solution will be to use either :global.set_lock or a global named process to ensure exclusive access within the same VM instance or cluster. On top of it, use OS-level file locking to restrict access across VM instances on the same machine (not clustered), for example on different IEx sessions.

I will mark this as the solution, but if someone has a better one please point that out :slight_smile:

6 Likes