Creating a cache for non-elixir apps with elixir?

Questions / Help
1

Hi everyone, I have a legacy Python system who’s backend and database in mongodb is your archetypal Big Ball of Mud. It will be a long while before we can totally replace it, so in the meantime, I’m looking at solutions to just make parts of the app run faster. We’re going to try out elixir for a few things anyway, and I’m wondering if it would be practical to make a caching micro-service in elixir? Does anyone have experience having other systems hitting Elixir to get cached json data or even materialized views, and if so, what do you recommend to use and do you have any general tips?

We have a lot of queries that run like stuck pigs with nasty mongodb pseudo joins, but I think the results could be cached most of the time and then updated with eventual consistency some of the time. So (I think?) I’d be looking at two things:

  • Python helper hitting microservice elixir cache for data when it can
  • Python process sending out RabbitMQ messages to something on the updates to recalc and update the cache.

If this would work well, it would be a great advocacy point for us to get more Elixir into the stack too. Any input much appreciated.

2

Why not use an actual cache like varnish? Or maybe redis.

The closest thing in elixir would be ets tables, probably. But they are much less feature-rich than redis.

3

The only reason not to use a cache like redis (which I’m also considering) would be that we are hoping to move to elixir for microservices anyway and I think I might want the caching layer to perhaps be more programmatically smart. But yes, that could be better, hence me asking.

4

Using ets tables over redis makes sense if you are already using elixir or erlang since they help to avoid impedance mismatch. In a microservice architecture I would probably use redis.

5

Thanks for the input. Another driver is to keep as much new-smarts out of the legacy code as possible, so I had been thinking that a caching micro-service external to the Ball of Mud might be a good fit for elixir. But that is me brainstorming, so maybe I’m just wrong there. :wink: