How to add and install deps without stopping phoenix server (hex package)

I have written Need advice to implement custom extension and template installation in an elixir CMS post that I need your suggestion to improve my project. Still, I decided to separate some questions into different topics.

Is there a way when my application is running, I add some packages in my mix.exs without stopping the server, and it should be compiled and usable?

I searched about hot code reloading in Elixir, but I could not find anything about hex packages. I am using elixir 1.13+

Any suggestions?

Yes…

Don’t do it.

BTW It’s easy to check it is not possible, just try to edit mix.exs of a running server :slight_smile:

2 Likes

It is not possible as adding new dependency may require recompilation of other dependencies, and there is no way for Elixir to know that (it is knowledge that only Mix has).

2 Likes

The beam has means to load code at runtime. But that’s all it does: Load arbitrary code.

All the additional requirements like replacing running processes, dependency management and such are afaik still handled at compile time when building the release to update to. That’s the data in the appup file mentioned in places.

These points may not matter for plugins on a CMS. At best files loaded are completely independent of each other and have no compile time dependencies to each other, but there’s nothing in elixir or on the beam to enforce that at runtime.

3 Likes

Wouldn’t this be possible with the new Mix.install facility? But I haven’t checked.

2 Likes

I’m pretty sure this is the correct advice:

As a thought experiment, there might be a convoluted and risky way. I’m not remotely suggesting you actually do this, but I’d be interested to hear from knowledgeable folk here about what I have right here (& not).

You can add dependencies at runtime using Mix.install (as commonly used in LiveBook).

But Mix.install can’t be used within a Mix project (it works by dynamically creating a new one). So you’d need to do something like host and run elixir scripts, maybe similarly to LiveBook (I don’t know how that works). But the packages installed within those scripts wouldn’t be available to regular Phoenix projects, which obviates the point. You could perhaps work around that by running scripts which each set up their own Phoenix-like pipeline (eg. as a starting point example phoenix.exs · GitHub). Or (more realistically perhaps) create a custom plugin API allowing the CMS’s Phoenix process to call on these added scripts functionality via message passing.

[sorry @dimitarvp , didn’t mean to step on your toes]

5 Likes

Unfortunately, I don’t know what developers want to create; create a hex package with many dependencies or no dependencies

My problem comes from when my user has no knowledge of elixir and just a user who want to use my CMS.

It was a demo for You… to test that the server will complain when You modify the mix.exs file of a running server and reload the page.

1 Like

I think if I handle it with docker is better option instead of elixir side. And I think downtime is too low to know user 's site is down or not.

You might have security problems if You allow users to load random modules.

3 Likes

For installation custom template, the developers just can replace some only pre-specified files.
By the way, it is not a real replacement because they just notice the CMS loads some pre-specified files from their own hex package. I save their package module name in database and state.

State for calling developer module name every loading pages.
Database for storing developer module name and some information, for example, if State gets into trouble it restarts itself and takes information of the database.

I have written about my idea:

It is like Joomla and WordPress, the core of CMS has no responsibilities about the bugs will be created by developers

if you have suggestion please let me know

Thank you

1 Like

It’s possible to write plugins in Lua via luerl and load them at runtime GitHub - rvirding/luerl: Lua in Erlang

It works

3 Likes

What is your suggestion about this Distributed tasks and tags - The Elixir programming language

For example, I create another node with new dependencies-compiled and after that I can get access to this from main node!!

My main source starts with this: iex --sname main -S mix phx.server
My component one or all the dependencies: iex --sname dep -S mix

Now can run this command in main node like this:

Node.spawn_link :"dep@shahryars-iMac", fn -> Hello.world end

I have no experience of this way, but do you have any suggestion about this?

@LostKobrakai @kokolegorille

I’d suggest diving deep into how hot code updates work. Doing a hot code update should involve all the low level parts you’d need to do for handling a plugin architecture using elixir/erlang code. You probably won’t find much like a documentation or tutorial about this given this is not a usual way people would setup an elixir project (for all the drawback enumerated in this and other threads about elixir cms’s/plugins here on the forum).

1 Like

Hmm, I have researched several times; as you said, I have not found the reference I can understand, but the only way I could see the way to do my task is not hot code update or getting from runtime is creating different nods. I did not test this way, just tested some simple phoenix app.

But thank you.

It’s also possible to pass cookie as well…

You did not mention what your plugins are going to do…

If I had to, I would use Luerl, because You can extend your application with it, as Lua is done for this.

Otherwise, You can do like Joe Armstrong was doing… create a bunch of empty gen_servers, and use hot code reload to do this.

Going the node way is possible, but Elixir is not the right language to do Joomla, or Wordpress.

1 Like

I have described my problems very severely; I have two sections in my project that help developers add some features without changing the core of my CMS.

  1. In this section, I named the plugin. Maybe this is different with a plugin word in your mind. I have many events like after_login, before_register in my CMS that you can create a hex or a module for an event to do something. For example, Sending SMS after success_login.
    More information: GitHub - mishka-group/mishka_installer: Mishka Installer is a system plugin manager and run time installer for elixir
    Simple example: GitHub - mishka-group/mishka_social: MishkaSocial is a plugin for MishkaCms as a social networking gateway

With the top plugin, I mean MsihkaSocial, I put login icons like GitHub and Google, which let users log in and register with social networks, but I did not change my CMS core. So it is basically a hex package, and the users should have the ability to install it in the graphical admin dashboard. Still, now they are forced to put it in mix file and config it in application.ex manually.


  1. In this section, I named the component; it is a complete project like a restaurant manager or a store builder like Shopify.
    Hence, developers will create a system like Shopify and add to my CMS and call some functions or, for example, my user manager to connect their project to my CMS.
    It should be noted whether their components even have some plugins or not. Still, now they are forced to put the complete project in my CMS, stop the server, and restart it again.

When I do not add a graphical admin dashboard to let users install a plugin and component easily, the clients and developers are not attracted to my CMS; Please forget Joomla or WordPress; just imagine I am going to create a project that does not let users easily have own facilities.


Because of these problems and hard to understand hot code reloading without good documents, I want to select creating different nodes if it has no problems

I have been developing my CMS with phoenix LiveView and elixir power for more than 16 Months, I can not cancel this open source project now :slight_smile:, my project will support English language in 4 months later and until now I can say no one create a CMS like me with elixir. It has many APIs and its dashboard is very UI friendly.

So I have to find a way to create some normal features to persuade users it can be a good target instead of Joomla or WordPress. Even is not, so we have some normal features and in different paradigm we can do better.

CMS link: GitHub - mishka-group/mishka-cms: MishkaCms an open source and real time API base CMS Powered by Elixir and Phoenix

Thank you in advance.

If you allow them to load Elixir code, then your CMS security is affected, because then they can run any arbitrary code they want, even if you only allow html templates. Please remember that we can run Elixir code on templates.

To be clear your CMS will become a target of attackers via the plugin system, just like in the Wordpress and Joomla plugins.

An attacker can be what you think it is a legit user or can trick a legit user to install the plugin, because the plugin is awesome and free, and/or is a version of a paid plugin. Does this ring bells from Wordpress and Joomla? If it doesn’t then I am really concerned that you aren’t aware of such issues on those platforms.

1 Like

I believe when you prepare a space for developers to start their project faster because many essential items have been made before for them, it would be an excellent target to make stuff for clients because they do not need to implement all the things together and can just focus on a specific feature.

After years of the CMS’s development, it will become a framework.

In the end, I have another question, why was Phoenix created and Phoenix LiveView either? When everybody can create a project which can be a security issue?!

Even if they create security issues, these problems can help the CMS be safe than before, and I will try to improve it.

Thank you for your comment, all the comments you send can help me to improve myself.

By the way, it is self-host for now.