binarypaladin
Using a compile time mutable registry?
Let me start by saying that while I have done some basic coding in Elixir and messed with Ecto and Phoenix I am coming from a mostly Ruby/class-based OOP background, and I can tell I have a lot brain-training to do. The main issue is dealing with immutable structures. It’s been smooth sailing thus far actually but… in a project I’m messing around with I’ve hit an issue.
Let’s say I have a struct/module named Thing. A Thing can be related to other things via some kind of value—let’s say I just have a key named relationships. The problem I see is when you have a struct of thing1 and thing2 that relate back and forth. If this was OOP and I was building a class or even a simpler structure like a hash, thing1 would get created first then you’d create thing2 and probably have it point back to thing1. From there you’d mutate thing1’s relationships to point to thing2. However, since I cannot mutate thing1, changing the relationships results in a new struct which points to thing2 which points to the old thing1.
At runtime it seems like you can accomplish this with a GenServer. All the Thing things understand that they need to talk to a GenServer and related a name. In this case, thing1 would have a reference to a name “thing2” that, at some point in the future, would resolve to the actual thing2 once defined.
What I really want is a global map available at runtime but generated dynamically at compile time. In terms of OOP, these structs are just “instances” of Thing. What would be better would be to have a module named ThingsRegistry with a function called get that took a string or atom and returned a named struct but, unlike a GenServer is not a process but rather the map that get is reading from has been set at compile time and perhaps register to replace the name map.
Can a macro take an existing function, get the return data, and then overwrite it?
I understand that I might be thinking about this all wrong and if that’s the issue, please feel free to let me know.
First Post!
mudasobwa
I am not sure I understand the requirements properly, but this is the raw implementation of the module holding map built in the compile time
defmodule AllInOne.Thing do
defstruct params: %{}
end
defmodule AllInOne.Registry do
@things %{
foo: %AllInOne.Thing{params: %{id: 1}},
bar: %AllInOne.Thing{params: %{id: 2}}
}
@spec things :: map()
def things, do: @things
@spec thing(atom()) :: %AllInOne.Thing{} | nil
def thing(name), do: Map.get(@things, name)
end
AllInOne.Registry.thing(:foo)
# ⇒ %AllInOne.Thing{params: %{id: 1}}
Here I built the map statically, but one might easily call another Elixir code there to build it dynamically. The only requirement, the called code should have been already compiled (it might be in the same project, but it cannot be in the same module, because modules get compiled “all-at-once”.)
Can a macro take an existing function, get the return data, and then overwrite it?
Sure it can. Here is an example from my Telemetría library, that grabs a function and wraps the call. You might want to explore this library source because it also modifies a code a lot during compile time.
Most Liked
mindok
As always, it depends what you’re trying to do…
If you are looking at maintaining relationships between Things, probably the easiest way to look at it is to separate the entities from their relationships (i.e. model it as a digraph with nodes and edges - you could take a look at GitHub - bitwalker/libgraph: A graph data structure library for Elixir projects · GitHub for inspiration).
Other than that, looking up other entities by some kind of key is a pretty common approach. How that is implemented depends on what you are trying to achieve (in detail). For example, GitHub - lau/tzdata: tzdata for Elixir. Born from the Calendar library. · GitHub provides a timezone database for elixir - it builds the data and relationships in ets on startup (and periodically updates on a schedule when timezone databases update) and provides data back to consumers via lookups on the ets tables. Incidentally, it is building links between entities as part of the database build process to allow alias names for timezones.
al2o3cr
What you’re describing sounds a lot like ETS except for the compile-time stuff.
The stdlib uses ETS to store graphs (see :digraph and :digraph_utils) because it’s a mutable key-value store that can be shared with other processes.
For a simpler example, here’s an Advent of Code solution from the 2018 problems that uses an ETS table as a doubly-linked circular list:
Each element in the list has an identifying “value” and the values of its left and right neighbors; the list is traversed by repeatedly looking up entries by the first value and then recursively following the links.
For that problem, the structure was really convenient: each update was a fixed, small number of steps way from the previous update and inserted or removed a single element. That meant changing three entries in ETS, versus rewriting every following entry in a map or list.
Qqwy
(emphasis mine)
I think we might have a case of the xy-problem here: You are looking for help with your supposed solution, but do not give us the full problem which makes it difficult to reason about the solution space. We’re slowly getting closer, as we now know that what you actually want to do has very little to do with ‘compile-time registries’.
You think you want to use bidirectional relationships between e.g. parents and children. However, what would you use these relationships for?
In object- and class-oriented languages where we deal a lot with inheritance, (bi-directional) relationships see a lot of use, especially since pass by (mutable) reference is very common, which is what gave rise to Joe Armstrong’s famous quote You wanted a banana but what you got was a gorilla holding the banana and the entire jungle.
In an immutable functional language such as Elixir, we’d model the problem usually in a very different way where these mutable references are not needed. In these situations we often pick only one direction to go in (such as only parent → child) and that is how we fill in our structs. This is what e.g. Ecto’s relationships use.
But say we are indeed trying to model the ancestry of a family of humans, which is one of the few cases where indeed relationships in both directions might be important. For instance, we want to figure out who all the nieces and nephews of a particular family member are.
In that case, we use a dedicated graph datastructure. (which can be implemented under the hood in a number of ways on an immutable system, which I won’t go in here as it’s not important when using them). One example might be using libgraph but there are other libraries as well.
Last Post!
binarypaladin
After sleeping on the issue I realized a couple things. Making the registry without metaprogramming was actually not complicated. The fact that it could be hand coded, even if awkwardly, made the clear enough. I just created a new Registry struct the keeps the map of the things as they are added and has a couple functions for associating things and it’s really, really simple—which is what I was going for.
I can carry on with no metaprogramming for now.
The only thing I needed to change was where some of the functions that focus on Thing, they need the whole registry in some cases—basically anything that needs to transverse relationships, which is a pretty small subset.
I think one of the things that @Qqwy hit on, even if not directly, with Joe Armstrong’s quote about the gorilla and his jungle wasn’t so much whether I want the banana but, in this case, I actually want the gorilla and his banana stash! If I want the gorilla (or the state he represents in what is becoming an increasingly absurd analogy) I need my functions to accept that state as opposed to looking for it implicitly—which is, obviously, one of the biggest gremlins in class-based OOP. (The gorilla cares about the bananas. The bananas do not care about the gorilla.)
Whether I ultimately store it in an ETS, a GenServer, or have some default hanging around in Mix.Config or FastGlobal is an implementation detail. In my tests, I can just create a Registry struct and make sure everything works.
Another added benefit of this approach is I can add other configuration/customization details to the registry that I was beginning to put into Mix.Config.
Also, with my goofy focus yesterday, I kept meeting Ecto. As it turns out there’s source code I can look at, lol. I start by saying “no metaprogramming” and then evolve to looking for a solution in that regard. Old habits die hard.
Thanks for your input everyone.
Popular in Questions
Other popular topics
Latest on Elixir Forum
Sponsor Spotlight
Supporting innovation across the BEAM ecosystem.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #elixirconf-us
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #security
- #hex
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking for Elixir devs who love to ship. Start your free account.
Practical resources that improve the lives of professional developers.
Producing high quality Elixir screencasts since 2017.
Enabling companies to succeed by building software people love.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"