shahryarjb

How to compile new dependencies without restart elixir iex or phoenix server

Hi friends,
imaging you run an elixir project, after that somebody edits the mix file and add new dep like:

defp deps do
  [
    ## old deps
    ....
    # new dpes
    {:x_plugin, "~> 0.0.6"}
  ]
end

So without exit from iex or restart the phoenix server, I want to get new deps and compile it in runtime after that I can access to this new libraries and use it in the project!!
How can I do this? Because it is not a file of elixir to compile in runtime; it is a full project.

This can help me to install new component in my CMS.
Thank you.

6 comments

#dependencies #runtime

0 3130 6

2023-10-18 23:56:42 UTC

Most Liked

LostKobrakai

That’s not really true stated like that. All the capabilities are available if you look at hot code updates. Even in releases you can just copy paste modules into the console and they’re compiled and kept in memory until the vm stops. The BEAM was built to explicitly allow for not needing to shutdown the system while the underlying codebase is continuously updated.

But there’s a lot more to maintaining a working system than compiling and loading modules and those are the problematic pieces. There’s updating state correctly besides changed code, there’s maintaining compile time dependencies between modules if macros are involved, there’s runtime dependencies between applications and modules, which require maintaining a certain load/startup order. And all of those also might fail, and those failures need handling.

The available APIs for the above are either quite low level (see e.g. Code) or to my understanding tightly coupled to building releases and doing hot code updates. There’s probably also a lot in the code bases of mix or rebar when it comes to dependency management, but I doubt there being much public API for those pieces.

By the parts I know I’d expect someone attempting to build such a system to need to know not only elixir programming, but also quite a bit about how the beam load/starts applications, how releases are structured, where compiled files need to go, how to cleanup things supposed to be removed and as mentioned how hot code updates deal with all the runtime complexities of updating code for a running system.

The security factor imo in the end boils down mostly to how well the code is audited before being loaded, given there’s no sandboxing on the beam just like in php. This feels like a separate topic to being able to actually load the code though. To me a plugin system from a security standpoint is no different to adding dependency :xyz to a plain old mix project and deploying the changes. When nobody looked at :xyz before putting it on a running system that’s a problem.

Post #5

fireproofsocks

I developed in PHP for many years and I even authored 2 books on PHP CMS’s, including WordPress. PHP is a very different beast than Elixir. Importantly, PHP executes only in the runtime: each and every request must parse the files and execute the code in them, over and over (barring any optimizations). So there is no penalty for adding or removing or modifying files on the fly.

Elixir is compiled, so it cannot easily add, remove, or modify its modules at runtime.

There are a few exceptions to this that add some flexibility to the landscape, but they do not overcome the inherent reality of it:

Inside iex, you can, for example, paste the contents of a module and then execute its functions.
You can recompile modules from within iex, e.g. iex> r MyModule
.exs files are always evaluated at runtime; you can leverage this via Code.eval_file/2 and its cousins.

So, to tie this all back together, yes, you could build a CMS in Elixir that would load plugins at runtime and you’d end up with something similar to what PHP CMS’s provide. HOWEVER, functionality comes at a significant cost:

you take a performance hit when you evaluate code at runtime
there are serious security considerations to loading up and executing arbitrary code at runtime.

These are pretty significant consolations that would badly handicap Elixir to the point of questioning its use for such a case. This is part of what explains the general consensus “don’t do it.”

Why? Because you would effectively be granting any plugin author the ability to see and do anything with your site and its code. It’s a huge risk. This is part of why WordPress and other PHP platforms are frequently targeted by hacks. I was offered $5k a month to maintain backdoors in WordPress plugins after one of my plugins was effectively hijacked.

Put another way, the same features that make WordPress popular (e.g. easy accessibility, easy to twiddle with) are the same features that make it a nightmare to support, secure, and scale. Sure, it’s one way of doing things, but there’s a reason that some projects/technologies don’t bend that particular way (at least not easily).

Post #3

rhcarvalho

The best solution I’ve found so far is to run :init.restart() in iex. Credit to https://stackoverflow.com/a/71368204/963287.

Post #6

Last Post!

shahryarjb

I did this:

If you are using Phoenix as developer mode, please disable live_reload in dev.exs. Please add reloadable_apps: [:mishka_installer] to your endpoint config in config.exs file.

in my library
https://github.com/mishka-group/mishka_installer

Post #7