Intercept the raw request in the Phoenix controller

Hi @adriancarayol, welcome!

Did you create the CacheBodyReader like the docs mention? Plug.Parsers — Plug v1.20.2

You need to create that file as well as adding that snipped you posted. Please try that and let us know what issues you find then? Thanks!

Hi @pdgonzalez872 , thanks!

Yes, I have created the CacheBodyReader like the docs.

The problem is that it seems that if the content-type is not registered in the parsers, it does not work, for example:

curl -X POST http://localhost:4000 \
  -H “Content-Type: application/x-yaml” \
  -d “---
name: John Doe
age: 30
city: New York”

The assigns field in the conn struct it’s empty.

conn #=> %Plug.Conn{
  adapter: {Bandit.Adapter, :...},
  assigns: %{},
...
}

The controller it’s pretty simple:

defmodule InterceptorWeb.RequestController do
  use InterceptorWeb, :controller

  def receive(conn, _params) do
    dbg(conn)

    json(conn, %{message: "Hello, Interceptor!"})
  end
end

The idea is to be able to capture any type of request, regardless of its Content-Type.

It’s currently configured to only allow MIME types starting with “text/“:

Change that to: pass: ["*/*"]

Sorry @jswanner , it’s wrong in the OP, the configuration is as you say, but still, it does not work correctly.

I see. I tried it out and agree, it doesn’t seem to get parsed/ingested at all.

After a bit of digging, it turns out that it is the Content-Type in that curl request that is the problem, not the body. FYI - If you switch it to json and pass in valid json, it works out of the box.

To make your current setup (that curl script) “work” is very interesting and I hope you can see how the team building and working on these tools are excellent. I say that because they offer great defaults but also allow you to do whatever you want, like we see this person doing with xml (https://stackoverflow.com/a/36202176). To make it work, you’d need to implement a parser that takes in x-yaml and parses it correctly. You will see sane defaults, great docs and also these “scape hatches” throughout the libraries in the community.

To solve for your OP question (and maybe a follow up you may have), accepting ALL kinds of requests requires a parser that takes anything you give. Be careful, this sounds dangerous →

Disclaimer: The Phoenix team knows way more than I do so I’m sure there are GOOD reasons to not accept every kind of request and that these defaults are great ones to have. Since you are returning json, could you also make the curl/what sends you the posts send json as well? I’ve never worked anything other than the defaults and looked into this because I was curious about how it all worked. But hey, maybe you do have a valid use case. I’ll let folks that know more chime in about it.

In the discussion above, [*/*] Seems to have nudged you that it would accept everything and I think it’s the crux of the confusion. It DOES accept and respond to your request , what it doesn’t do is parse the body because the content-type was not defined (no x-yaml). That’s why you don’t see it in assigns. To see it, seems like we need to write a parser, like the person that wanted to do the xml one likely did. I won’t parse the yaml for you, that’s an exercise for the reader. So, let’s look at the parsers that come out of the box, here is the urlencoded one: plug/lib/plug/parsers/urlencoded.ex at v1.16.1 · elixir-plug/plug · GitHub

So, here is a naive implementation that shoves things into assigns:

defmodule CatchAllParserDanger do
  @behaviour Plug.Parsers

  @impl true
  def init(opts) do
    opts
  end

  @impl true
  @doc """
  https://hexdocs.pm/plug/Plug.Parsers.html#c:parse/5
  """
  def parse(conn, "application" = _type, _subtype, _params, opts) do
    {:ok, body, conn} = Plug.Conn.read_body(conn, opts)
    conn = Plug.Conn.assign(conn, :raw_body, body)
    {:ok, %{}, conn}
  end

  @impl true
  def parse(conn, _, _, _, _) do
    {:next, conn}
  end
end

Then we need to adjust our app, here are some diffs:

# endpoint.ex
-    parsers: [:urlencoded, :multipart, :json],
+    parsers: [:urlencoded, :multipart, :json, CatchAllParserDanger],

# router.ex
   # Other scopes may use custom stacks.
-  # scope "/api", EFWeb do
-  #   pipe_through :api
-  # end
+  scope "/api", EFWeb do
+    pipe_through :api
+
+    post "/adriancarayol", PageController, :receive
+  end

# page_controller.ex
+  def receive(conn, _params) do
+    IO.inspect(conn.assigns)
+
+    json(conn, %{message: "Welcome!", raw_body: conn.assigns.raw_body})
+  end

Using your curl, in a script:

#!/bin/bash

curl -X POST http://localhost:4000/api/adriancarayol \
  -H "Content-Type: application/x-yaml" \
  -d "---name: John Doe age: 30 city: New York"

Running the server and calling the script gives this in our logs:

[info] POST /api/adriancarayol
[debug] Processing with EFWeb.PageController.receive/2
  Parameters: %{}
  Pipelines: [:api]
%{raw_body: "---name: John Doe age: 30 city: New York"}
[info] Sent 200 in 21ms

and returns:

ef [main] $ ./curl_test.sh
{"message":"Welcome!","raw_body":"---name: John Doe age: 30 city: New York"}ef [main] $

Also, the person wanting to accept xml linked this: Way to Read Request Body As String · Issue #459 · phoenixframework/phoenix · GitHub in their accepted answer, pretty neat and pretty much what I showed above.

Hope this helps! But again, please read the disclaimer above and welcome to the community!

Thank you very much for your reply @pdgonzalez872 ! Definitely this use case is not something usual, in the years I’ve been using Phoenix I’ve never had to use something like this (and of course you have to use it carefully !).

Again, thanks and long life to Phoenix !