Verifying requests using HTTP signature verification (we need a better way)

Possible Solutions?

I’m not sure what would be the best way to go about solving this, but maybe a way to hook Plugs into the Plug.Parsers module (for certain routes, or route pipelines, or …?) where we can use the body before it is parsed?

Or maybe we could have another plug before Plug.Parsers and Plug.Parsers could check %Conn{} for (and use) a temporary “already fetched body attribute” in the struct, if the body has been read already?

I don’t know if those ideas are possible or feasible, so I’m going to do some digging around, I guess.

Any other thoughts, ideas, possible solutions, concerns?

Does anybody agree that this is a problem that should be solved in core libraries, or at least without having to override core modules or functionality?

I have not encountered systems requiring this kind of signature verification yet myself, which gives me the (subjective) opinion that it might nog be that general of a problem (right now). But there could definitely be a Plug that will do this.

Do you often integrate with other services, and use webhooks?

Although, it would be applicable to any use case where you’d want to verify a request body with a signature, the most common situation where you’d encounter this is in verifying 3rd-party webhook requests. (like the ones from GitHub, Stripe, etc that I mentioned earlier)

I have implemented this - and it was my first (and only!) “bad” experience in elixir world.

Something you assume is gonna be fast to do - but gets you into quite the rabbit hole - and you end up with code you are not really happy about.

dev happiness rock bottom - and worst case is if people implement these webhooks without verifying, because it’s quite steep to understand what’s going on.

My brain have suppressed the experience/specifics - so can’t talk of potential solutions - but if there is no easy solution(which I think is the case) a PR with documentation for plug and even (especially?) for Phoenix and a best/recommended way of doing it is probably the fastest way forward..
eg titled “Verifying webhooks”, and then it tells the story of why it’s troublesome, and gives a suggested implementation..

well I think a good pattern is /webhooks/provider - so you add multiple Plugs (which of course is a small performance hit) - and you can decode the json and pass it to the controller.. How to read request body multiple times during request handling? - #7 by outlog - but pretty it ain’t.

so totally share your frustration..

Yeah, I’m looking through right now and Plug.Conn.read_body/2 is not used until inside each specific parser’s parse/5 function, so I think that makes a solution more difficult.

I wonder if we used Plug.Conn.read_body/2 in Plug.Parsers and passed the body into parser.parse/5 instead, if that would be okay.

Too much breaking change, probably

How about a Plug that replaces the Conn.adapter with one that verifies the body when read? read_body can return an error tuple already, so it shouldn’t break Plug.Parsers.

We had this problem, and after talking about it at ElixirDaze Four other people approached me with the same. I think it’s common enough that it should be handled by something that is mean to be as general purpose as plug.

@ryanwinchester we copied and pasted the plug parses into our codebase and added a line the put the raw body in conn.assigns[:raw_body]. Here is the appropriate slide in that ElixirDaze talk about it Daze

However we don’t use this solution any more. The punchline is we don’t use plug on any service that validates signatures. We have one service on the front end still using phoenix because it serves assets and html and has to sign requests but not validate.

There is also this issue raised by someone else i know is doing signing
https://github.com/elixir-plug/plug/issues/691

Isn’t reading the body done with an adapter?

Maybe a toplevel plug could:

1. Read the body and then put_private(conn, :raw_body, raw_body)
2. Put a custom adapter that makes read_body use:raw_body instead.

This way when the parsers parse, they will use the adapter’s read_req_body instead of the one that can only be called once.

The custom adapter could otherwise use the original adapter’s implementation for anything else.

Where can I “plug” in my own Plug.Conn.adapter in the framework?