Web Forum Software in Elixir / Phoenix

but it has not had any github updates in 9 months

I thought about writing a CMS in Elixir but there is a not so trivial problem:
Every CMS should have a plugin system (my opinion) and that plugin system should be manageable during runtime so the admin user can search, install, update and delete plugins while the application is running. But how do you do this in a language that uses a long running process and also needs to compile code? It’s tricky. You also have to take care of database migrations too (because plugins might need to store data). I am sure with a bit of magic trickery, you will be able to get something together, but how well does it work in the end?

There is a reason most CMS and even eCommerce systems are written in PHP. PHP makes those dynamic plugin systems quite easy because there is no long running process and no compilation involved.

Now you can say that people should just add a mix dependency and restart the Elixir process, but that is far from enduser friendly.

I was super optimistic about Firestorm in contrast to the incumbent for modern-day forum tech, Discourse, which is what Elixir Forum uses. Discourse is fairly pleasant and snappy as an end user, but it is a chore from an operator’s perspective because it’s Rails trying to do soft-realtime, and with a strange, bespoke deployment workflow that manages to undermine the value/repeatability/standardization of Docker-based deployment almost entirely. So much of the hand-wringing in that ancillary project is unnecessary in a BEAM environment, not that we don’t have our own wrinkles around deployment.

I’m disappointed but not surprised that Firestorm has fallen idle. Most crowdfunded, change-the-face-of-niche-software projects seem to do that. It never reached an especially usable state, the last time I checked in on it, although kudos to them for at least attempting to dogfood it.

It’s on my long list of things to do, fork firestorm and finish the todo list

Wouldn’t it work well if the CMS let those plugins communicate through hooks, which would allow the core application to run without a care in the world while the admin would add and remove plugins. It’s could be way safer as well, depending on how it is done.
As for the database, it would be a huge pain to let the app and plugins to share the same DB. IMHO each plugin should have its own DB so that removing them is always painless, and when plugins need to VIEW some data from the app DB it should do so via the available hooks.

But how does the plugin get installed? It needs to be pulled from somewhere, compiled, and started.

Have you looked at existing solutions? That’s how it works.

That’s not enduser friendly or manageable at all. Imagine you have 20 plugins installed, you would need 20 databases.

IMHO, what you suggest is even worse than adding a dependency and restarting the app, from an enduser perspective.

Plugins could either be coded in a scripting language to remove such inconveniences, or have a standard way to do what you said that they need to conform to (could be one plugin <> one docker container), in which case it could be made automatic as well.

I have not, but working on a project in PHP I am not surprised that everything is spaghetti code, including the way the DB is handled.

In which case can a few tens of DBs be a problem?
It’s much simpler to manage as each plugin creator has to manage his own DB, without side effects, without conflicts with other plugins, and etc.
It’s also much simpler for the website admin, as removing a plugin won’t fuck up anything EVEN IF the plugin creator created a mess. Updating would be a breeze, and downgrading wouldn’t break the DB half of the time.

With the usual plugin systems, any plugin installing, upgrade and downgrade can break everything, especially in the DB, and they can steal all the info they want too. I dunno, but as a end user I wouldn’t feel safe (and I guess that’s part of the reason why I have always avoided CMS).

For a forum I’d probably do it as a set of elixir/erlang/etc files, the BEAM can compile directly (and so can elixir) so you can hot-load easily, they should just be their own applications that register hooks to be called. With that you could implement scripting languages as well if you wanted.

Agree, it’s not hard to have them share.

And they wouldn’t be able to share data effectively.

Well look at Discourse, this forum, it is a pluggable hook-based style.

That doesn’t make any sense, it’s not the right approach. I’m not a database expert, but I think this will hurt performance so badly, as database’s connections are expensive. It’s totally fine if you need a table for each plugin (like most of the existing solutions), but not a different database. A typical structure may look like this:

- CMS
|- Plugins
|-- Id
|-- Name
|-- Description
|-- Active
|-- Roles
|-- ...
|
|- Plugin_News
|-- Id
|-- Title
|-- Body
|-- Plugin_Id
|-- ...

Another way is to use a “manifest” file to define the metadata for a plugin, so you won’t need to store its information in the database. However I’d say that the database approach is much better, in the end most of the plugins may need a table to store some kind of information, you can also mix both approaches like Orchard CMS does.

Ignoring remote installation of plugins (ugh, that should be very complex), but an alternative solution instead is to distribute a set of plugins with the app and enable or disable them through the admin panel using the database to store its state, that should be enough as a starting point. Next you can think how to implement a more complex solution, but your project doesn’t need to start as a full-featured WordPress, have in mind that it has more than a decade of development.

Firestorm is a very interesting piece of software and the concept itself is worth exploring, but I’m not sure this is what I’m after.

@Phillipp I’m still relatively new to both Elixir and Phoenix but I don’t think a plugin system is something that would be hard to implement. No matter the language, I would design a production grade CMS to accept signed packages from the admin system with the optional override with a huge warning. Each package would have a standard format for installation, migration, upgrading, and deletion whether it be local or remote.

As for handling plugin compilation, I need to explore OTP more, but I don’t see why this would be a problem. Could you be more specific as to why you couldn’t use a newly installed plugin?

Related: Does hot code-swapping come for free? - #3 by OvermindDL1

You raise a good point about PHP. Maybe I’m just too naive to the drawbacks of Elixir + Phoenix, but I really don’t see the issue here. I’ll have to explore some strategies and will hopefully get back to you soon.

@shanesveller Discourse is certainly better than most if not all forum software but it’s not something I would wish to install and run myself. A conversation organizer might be a more accurate description of what I might attempt to build.

In my opinion, very few people need Docker, and I also don’t like the idea of downloading entire environments. Install Nix package manager, install it on an Arch machine, and call it a day.

Related: Preface - Nix Pills

I wasn’t aware of Firestorm being crowdfunded. Do you feel the team delivered or was it a pump-and-dump of dev that you normally see on Kickstarter?

@frigidcode With the latest updates to Elixir, Phoenix, and LiveView, do you think the effort of a fork is worth it? Sometimes re-inventing the wheel helps you understand the pain points that eventually lead to a better solution.

@Cochonours I’m not so sure about the idea of hooks. I understand that method is popular with Wordpress but I don’t like the concept of anything from anywhere being able to alter anything. I would try to keep it more consistent with concept of Phoenix pipelines if possible. Hopefully you would agree that a good plugin system wouldn’t require any code modifications outside of maybe a configuration file. Are you aware of lesser known alternatives to the hook-style system?

Although it sounds super cool to allow plugins to communicate with each other, I would consider this an anti-pattern. Unless you’re designing a set of plugins that need to communicate with each other, there’s no way this isn’t a debug nightmare. Do you know of a scenario where this would be considered a good idea?

One DB per-plugin doesn’t sound ideal to me. I’ve been a fan of database.schema.table for a good decade or two. In your production grade applications, do you often need an entire database for a single feature? Normally you would only do this if the DB was shared with other applications and those applications existed before your requirements. I can certainly see why there might be headaches in terms of permissions. I’m not saying it’s the most practical thing to do, but you could probably have two different DB users, being a general CMS user, and a plugin CMS user. Basically the plugin CMS user wouldn’t be able to touch certain schemas or would have read-only access. I’m also not too worried about the setup and cleanup of a plugin because I would design a system that would only accept a single standard that addressed both migrations and table ownership. I would go as far as programmatically creating a new DB user for each plugin and lock that user into a schema created just for the plugin. The only real hurdle is being able to get the permissions correct and being reasonable about how to adjust them for a non-technical user.

A plugin being coded in a different language sounds like a nightmare to me but I’m open to new ideas if you have some candidates? You’re basically describing FFI from what I understand. If I were to take this route, all plugins would have to be written in this language. I would only build a system that supported a single language simply from a maintenance perspective.

Related: GitHub - clojerl/clojerl: Clojure for the Erlang VM (unofficial) · GitHub

@OvermindDL1 A hook-based system might’ve worked for Discourse, but it’s also written in what I believe to be a less powerful language. I’m not sure what system I’m going to come up with or if it’s even going to be successful, but for sure I think we can do better than hooks and events.

@adrianrl I agree with you about DB performance. You also have to think about connection pools and I don’t think having a connection pool for an example of 20 plugin-specific DBs is going to be something I’m willing to do. Maybe there’s a good reason to do this?