Discourse Needs Phoenix

Discourse is a great and awesome forum system, no doubts, the best out there! I feel like Discourse has done a lot of good for the community making such a great to use product that we (the Elixir community) should give back.

Through some experience using it on the developer side, I’ve noticed some limitations and pain points. The first being, migration. Migrating a million+ post project is very painful and slow. It uses a lock against redis so more machines doesn’t really help (at work we threw over 100 machines at our migration and it didn’t run faster than 1 machine). Second major point is polling! This leads me to believe Rails is now the wrong tech choice for Discourse. In addition to that, the current discourse codebase (ruby backend) could be in better shape in my opinion (to put it nicely). It is not easy to extend or make changes. Also, it should have been built as a Rails Engine IMO.

I feel like all of these problems are trivial to solve in Elixir. Migration would be best suited using GenStage, Phoenix Channels for real time, ect.

I was curious what others thought. Anyone agree? Who would be interested in starting a project like this? Being that the front end is a well done ember app, it would be mostly backend API changes until channels were introduced…

I think this could also help move the needle for Phoenix to help get more exposure and attention.

6 Likes

I’d played with the thought, would not be hard but would take a long time. Thankfully Discourse is built in such a way that the front-end and back-end are separate, so you could easily replace the back-end while using the same front-end. :slight_smile:

Sadly, right now, I doubt I would have much time beyond making a few tiny module parts or so, always busy with work. :slight_smile:

And yes, I’ve been looking at Rails more lately (one of the few mainstream languages/frameworks I’ve not learned), and I am really disliking it, like really badly… >.>

P.S. Rewriting the front-end in Elm would be interesting, though Elm is still shaking out some features, like scroll events and control are lacking that discourse would require…

2 Likes

Agreed, it would take some time.

I like the idea of using Elm as well but that can even be done completely independently. Also, I think at the current moment, the need is larger for replacing Rails. Using Elm would be considered a nice to have IMO, and swapping both techs would likely see lots of :thumbsdown:

I’d just like to gauge the interest in such a project, and if it were to be started I’d like to be involved.

Yeah, the back-end is what hurts. Discourse server-side requires a fairly hefty server when serving any kind of decent load, would be far better on the EVM/BEAM, caching practically built-in, ecto for DB work, phoenix websockets. I am not sure what would need to be externally hosted other than PostgreSQL, significantly easier to install.

Yes I had also forgot about caching. Eliminates the need for Redis as well. :thumbsup:

Do you know where to find complete API documentation for discourse?

I think the most ‘official’ documentation is: https://github.com/discourse/discourse/wiki/API-Endpoints-Documentation

However Discourse could use some API changes regardless, so it should not necessarily be mimicked unless specifically wanting to use their front-end. Honestly I’d probably use graphql instead of pure json to save a few round-trips on the pages, one of the few times graphql would likely be worth it.

I like this topic, especially the performance, self-recovery and reliability that is all good of Elixir/Phoenix.

I already encountered many times that can’t connect to ElixirForum and shows either server side error messages or network connection error. I might capture some and upload later.

Out of curiosity , does anybody else also encountered this issue? Are we deserve a forum with all the Discourse goodies but build on Elixir/Phoenix?

I’ve seen it a time or two, what do you expect for ruby though. :wink:

This is exactly why I want ElixirForum on Elixir, that will show case the Elixir’s reliability with growing.

My two cents:

  • No matter what people will think ElixirForum built on Elixir, people knowing the fact much less than new comers
  • Reliability vs functionality, R is first and F can grow. Better solve the problem before it grows to unsolvable.

I’ve personally not experienced such issues and I am on the forum pretty much all day.

The only times I can think you might have experienced issues (on the server itself) is when I am upgrading either the forum or docker - which requires a restart of docker or a rebuild of the forum. Discourse is actually updated quite a bit so this may be a blessing and a curse :lol:

Network issues should also be minimal as we use a decent provider - but, as I am sure you will know, some network issues will always be expected as many factors play a part. Also peak traffic may sometimes lead to ‘blips’ with frontends like HAProxy, but again I think those would be rare occurrences.

With regards to a backend for Discourse, you would need to convince the people at Discourse and I don’t think they will be open to the idea given their lead developers are Ruby experts and as far as I know, haven’t migrated to Elixir yet. Also I think many Elixir people may prefer to channel their efforts on a native Elixir forum such as FireStorm :slight_smile:

1 Like

I’ve seen it very random, like you know Discourse uses JSON requests from the server to display things, occasionally one of those 500’s, or a page load will 500, it is only like 1 request in every thousand but I see them on occasion. It might be when you are updating, but when I ran discourse an update took a lot longer than one second. ^.^

1 Like

Yeah I started this topic before Firestorm was announced. I agree that may be the better way to go once it’s complete.

Here are some links:

  1. ElixirDaze 2016 - Processing 2.7 million images with Elixir (vs Ruby) by David Padilla - an example migration optimisation when switching from Ruby to Elixir
  2. Announcing GenStage - this is really good tutorial
  3. Flow github project - Flow is now a separate project that depends on GenStage
  4. The future of GenStage and Flow

Hope it’s helpful in this topic.

3 Likes