msw10100

msw10100

I’ve got a Phoenix application we’ve been running for some time. We now need to add AMQP as a message source. The AMQP messages will generally match the Phoenix routes in functionality. But we have substantial business logic in the Phoenix pipelines and controllers that we can’t easily refactor at this time.

In Phoenix app test code, we can use Phoenix.ConnTest to build up conn structs and dispatch calls to our Phoenix endpoints without actually going through TCP/HTTP.

Is it possible to use Phoenix.ConnTest at deployed runtime from our AMQP message subscriber, create a matching HTTP request in the form of a conn struct, and then use ConnTest.dispatch/4 or /5 to invoke our application endpoint so that the HTTP pipelines and business logic can be executed? Then extract the response from the returned conn struct and build a reply AMQP message?

Showing Posts 1 to 3

msw10100

msw10100 OP

Sort of answered my own question. I created a module as seen below that uses build_conn() to generate a basic conn struct and then passes it into the get() macro with the “/” route. When invoking it manually at runtime, the endpoint is hit successfully and the result is a conn mutated by the route logic.

defmodule TryConnTest do
    @endpoint TestControllerCallsWeb.Endpoint
    import Phoenix.ConnTest

    def go do
        build_conn()
        |> get("/")
    end
end

Presumably, I could manipulate the conn struct, adding HTTP headers, constructing a post body if necessary, etc… and invoke the Phoenix endpoints internally from an AMQP subscriber in the same application.

My question then becomes … is there a downside to doing this in production until we get round to refactoring the business logic so that it can be invoked from both AMQP and the Phoenix controllers?

al2o3cr

al2o3cr

The biggest downside that comes to mind: your management decides that readable code isn’t worth the effort again (they already decided that once to get you here) and the refactoring never happens.

Other, more politically-realistic downsides you could cite:

  • authentication / authorization is going to complicate things, as will things like CSRF tokens. Code CAN be written to fake all of these and/or short-circuit them for “faked” requests, but it adds complexity - and if not done carefully, can accidentally introduce security bugs

  • you’re likely to need wrapper code to deal with turning typed AMQP data back into the parameter shapes your controllers expect, and then MORE wrapper code to transform the output back from a stream of bytes into typed data. It’s not going to be complicated code, but it’s going to need maintenance every time the controllers etc change

msw10100

msw10100 OP

Many thanks for the thoughtful comments, al2o3cr.

My team agrees with each one of your points. The root of the problem is that we already have two parallel implementations of the service; one that does HTTP only and is highly scalable, multi-tenant and with API keys and advanced granular authentication (elixir, kubernetes, cloud-based) and another that does AMQP and HTTP, but is single-tenant only, with scalability limitations (C#, desktop-server-based).

That so much business logic is in the HTTP routes has mostly to do with the fact that the team (me included) were new at elixir and Phoenix when we built it (you mean Phoenix wasn’t the app?)

Now in our desktop server product we want to address scalability and add granular authorization and multiple isolated dataspaces on a per-user or user group basis (aka multi-tenancy).

So our best bet would seem to be to instead add AMQP to our elixir application, keeping all its granular auth, separate data spaces and other goodness, and scale it down from kubernetes to an elixir release deployment on a desktop. We would completely deprecate our desktop-only version. But that means we need AMQP. Sigh…

Your point about CSRF is a really good one, and the HTTP logic DOES definitely deal with that. We’ll have to look into how to deal with that when the request comes from AMQP. MANY THANKS for pointing that out!

We’re already looking into how to address security from the AMQP side, because those messages don’t currently carry the API key (or login session key). Further, AMQP is nowhere near as secure in this environment because anyone connecting to the bus can snoop everything.

At least we share most of the data structures in terms of data shapes between the two implementations, since they share a single HTTP API. There’s not 100% congruence between AMQP data and HTTP data, but they’re very close. The translations will be minimum and already generally understood. It helps that we won’t be changing the controllers much because of REST data contracts. We’re more likely to add new routes than change an existing one.

Our preliminary testing doesn’t indicate any significant technical downsides. We’re really looking for some downsides before we commit to management that this approach will more than likely meet schedule requirements.

al2o3cr, again, many thanks for your thoughtful reply!

— All posts loaded —

Where Next? Top

Trending in Questions Top

water
I’m aware of 2 such projects, are there others? git: GitHub - team-alembic/realworld: A fullstack Phoenix LiveView application with back...
New
katta
I having some trouble figuring out if I have set myself too strict of standards for my production server. Currently I can handle 75% of r...
New
peck
Hi everyone. My team and I have been working on a fairly modest app based around video streaming and chat, but we’ve landed a customer t...
New
bradley
I really like the adapter patterns that ecto, nebulex, waffle, etc. use and would love find something similar for a key management servic...
New
unaware8150
Hello folks! So at work, we are seeing some situations where we have to define some “fixed” strings that are used across the codebase in...
New
Alvinkariuki
How Can I Optimise Compile Time Dependencies I have been building an elixir application for about 2 years now. Many modules and files ha...
New
leoschweizer
Hi, we run different LLM models across a cluster, with each node loading a configurable subset. Model-specific queues let nodes consume ...
New

Other Trending Topics Top

budgie
A little off-topic, but I feel like people here have a good head on their shoulders. I used to be quite good at making software. Was luc...
New
KristerV
Hey. Is there anyone here who creates agents in their apps? Not talking about using agents, but creating them. I’m finding it pretty diff...
New
mudasobwa
I fully migrated to my own harness from Anthropic/Gemini and I think it’s time to share it. Welcome DSH, the DeepSeek Harness, fully writ...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
juhalehtonen
There has been a thread to discuss the Stack Overflow Developer Survey on this forum every year since 2018, so here’s yet another one for...
New
budgie
I love Elixir. It’s one of 2 programming languages I’ve ever fallen in love with. But I don’t use it anymore. Serverless was the promis...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews