nikody

nikody

Cowboy Send Stream Chunks from a Separate Process

I’m testing server-side events with Plug’s send_chunked/2 and chunk/2 but I also receive data from a stream. Due to the latter detail I decided to implement the receiving of the streamed data in a GenServer, the request is made with HTTPoison and it is similar to what is mentioned in this StackOverflow post, although I receive the chunk data in a slightly different format (still a non-JSON string with a data key), and had to also implement a handle_info callback for messages with a %HTTPoison.AsyncEnd{} struct.

Other than that it’s working and the GenServer is started from my controller. So essentially I have a controller that receives a request, then spins up a GenServer that makes another request, receives streamed data and has to send the data parsed back to the original connection.

The issue comes when trying to send the data back from the GenServer. I tried passing the conn struct to the genserver and calling Plug.Conn.chunk/2 from it but it seems that Cowboy recognized that a separate process sends a messages, raises a warning and then the GenServer stops processing. The Cowboy warning raised is here in the Cowboy code: [warning] Received message #{message inspected} for unknown stream 1. (or another stream ID, but generally integers 1 to 3).

As Cowboy spawns a process for each request I assume it expects that responses should be sent from this process which is what is causing this issue. I’ll probably implement a receive loop without spawning a different process to handle this, but I was wondering if I’m not missing or misunderstanding something and if there isn’t a way to send a response with Plug (Cowboy) from a separate process, in this case a GenServer.

Most Liked

dimitarvp

dimitarvp

You can still do that but the abstraction doesn’t need be on a process level, it can just be functions in a separate module that get called directly in the context of the web requests’ process.

I too wouldn’t want a 100-line pattern-matching clauses soup capturing the state machine of calling an external API right inside my Phoenix controller function so good job there, you are on the right path.

I might have been misreading various blog posts and news but it seems the writing is on the wall that Cowboy will eventually be abandoned in favor of Bandit. Cowboy has legitimate problems with scaling, pooling and data copying (if I remember correctly!) and Bandit aims to fix (part of) those.

Where Next?

Popular in Questions Top

New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
mgjohns61585
Could someone help me? I’m making my first elixir program, number guessing game. I can’t figure out how to convert the user’s guess from ...
New
fireproofsocks
I’m working on defining a simple Ecto schema for a table (in PostGres), but I don’t see where I can define a column as NOT NULL. Conside...
New
vac
Hi, I’m quite new in Elixir and I’m trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and I...
New
earth10
Hi, I’m just starting to build a side-project with Elixir and Phoenix and doing some basic test with Elixir alone. What strikes me is th...
New
dokuzbir
I want to highlight html closing tags when i click a html tag. That works in .html files but doesnt work for html.eex templates. How can...
New
beno
I will often find my self writing things similar to: case some_value do nil -> something() "" -> something() _ -> somethi...
New
Emily
I have VueJS GUIs with the project generated using Webpack. I have Elixir modules that will need to be used by the VueJS GUIs. I forese...
New
itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: The documentation above suggests that while ...
New

Other popular topics Top

johnnyicon
Hi all, I’ve just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I’m trying to use Postgres...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
Lily
In templates/appointment/index.html.eex: <%= for appointment <- @appointments do %> <tr> <td><%= appoi...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
RisingFromAshes
I’ve read in another post that it may be possible with a router helper - but I couldn’t find an appropriate one, and tbh, I’m still just ...
New
AstonJ
Please see the new poll here: Which code editor or IDE do you use? (Poll) (2022 Edition) It’s been a while since we first asked this, I...
208 31142 143
New
boundedvariable
I am going through the kafka architecture. All the features what the kafka is providing are already in Erlang. I would like hear your opi...
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 records...
New
dogweather
I wrote this comment on r/haskell, and it’s not popular there. :wink: But I think I’m on to something… Haskell reminds me of Java, and e...
New
sergio
Kind of like when jquery came out, it was super necessary. Existing drag and drop libraries have a bunch of baggage to support old browse...
New

We're in Beta

About us Mission Statement