jonator
FLAME is basically the dream for running AI agents that need system command access such as coding agents. However, I have concerns about scaling to a very high number of nodes (many thousands). I can just write application Elixir code and scalability is batteries included; incredible devx.
With distributed erlang, the default is for the nodes to not run in “hidden” mode and to instead create many-to-many connections to all nodes, which severely limits the overhead of each additional node that is added to the cluster. I noticed this seems to also extend to the default in FLAME as I did not see an option to support hidden node detection. (I may be wrong, was a fairly quick scan).
Would the solution be, in the case of the Fly backend, to run each node using a separate docker image that passes the “-hidden” flag then fork/update FLAME to support usage of nodes(hidden) for spawned workers? (Or maybe this is already achieved somehow). My ideal is the main cluster that runs the web servers and Oban workers for each agent is in the “non hidden” cluster, and they each only know about some number of dedicated node connections running each agent (a hub and spoke architecture).
Regardless, am curious to see what you think. Cheers.
Where Next?
Trending in Discussions
Other Trending Topics
Latest Phoenix Threads
Latest on Elixir Forum
Sponsor Spotlight
Bringing learning, work and compliance together since 1987.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #discussion
- #code-sync
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ai
- #ecto-query
- #elixirconf-us
- #blog-post
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #elixirconf-eu
- #api
- #forms
- #metaprogramming
- #hex
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Practical resources that improve the lives of professional developers.
Producing high quality Elixir screencasts since 2017.
**Building the future of software with AI, trust, and human connection.**
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"
Showing Posts 1 to 8- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
paulsabou
Maybe it’s worth considerding libcluster with partisan and a custom partitioning mechanism. this way you could avoid the mesh & decide how to split your nodes into small clusters => you could run flame in each small cluster
This should scale well I believe
nulltree
Hijacking with a noob question here:
partisansolves a different problem thanpg’s process group scopes, correct?I assume
partisanavoids the full mesh at the network level and process group scopes create an overlay network within the full mesh used for group membership propagation?cevado
it’s not a problem of pg but actually a problem of how disterl works… it is explained on partisan docs
but is worth reading the full section in the doc:
dimitarvp
AFAIK, Flame allows you to define pools where you define your own minimums, maximums, what kinds of machines will spawn, and others.
nulltree
Thank you - I did and that lead me to the question of if they solve two fundamentally different problems (specifically as described), not if
partisansolves a problem withpg.I’m now more confident they do.
chrismccord
note that FLAME nodes are assumed trusted nodes (shared erlang cookie, network access to everyone, mutable beamfiles on disk, etc), so you cannot execute untrusted code there unless you are running that execution in some sandbox env of its own.
jonator
I was thinking of running them as a highly limited linux user in the containers with only access to their own git repos as well as cli dev tools. It’s a convenient way to continue to leverage distributed pubsub and message passing per agent. LMK if there’s any possibilities of vulnerabilities there!
Are you aware if it’s possible to connections from FLAME nodes to be exclusive to the parent?
hauleth
If on the FLAME node there is any RCE then it can connect to the parent and all that mitigations you have described doesn’t make any sense.