I am getting data from an endpoint comming in a list , assume they are unique id's I try attempt to start a GenServer process for each of them but

Phoenix Forum Questions / Help
,
1

I am getting data from an endpoint as list, assume they are unique id’s I try attempt to start a GenServer process for each of them but the processes are terminated as soon they are started but the error is unclear , i cant seem to find what is killing the process, here is my sample code below and error trying to debug.

def new(conn, %{"hashes" => [_ | _] = hashes}) do
    # Enum.each the list of hashes call hash fn on each hash
    # needs more debugging the function is glitching "some processes failing"
    hashes
    |>Enum.each(fn hash ->  call_hash(conn,hash) end)  #research more queing calls?
  end



  defp call_hash(conn,hash) do
    with {:ok, _} <- GenServer.start_link(TransactionSubscriptionHandler, %{hash: String.to_atom(hash)}, name: String.to_atom(hash))
        #  {:ok, %{status: 200}} <- BlockNative.subscribe_transaction(hash)
         do
         json(conn,%{status: "Ok"})
        else
     _ -> conn |> put_status(400) |> json(%{status: "Failed"})
  end

  end

input

"hashes": [
        "0x04b08ab13d51613975cd5035cebb52d5f574137c42902a5b1147f953f1895c6a",
        "0x2285dfeafe2eae5846fe66bd9631f2d258c1d5b32702649153c9073a3a4ec8ad"
    ]

output from the command line

[info] POST /api/transaction
[debug] Processing with VhsWeb.TransactionController.new/2
  Parameters: %{"hashes" => ["0x04b08ab13d51613975cd5035cebb52d5f574137c42902a5b1147f953f1895c6a", "0x2285dfeafe2eae5846fe66bd9631f2d258c1d5b32702649153c9073a3a4ec8ad"]}
  Pipelines: [:api]
{:ok, #PID<0.597.0>}
[info] Sent 200 in 2ms
{:ok, #PID<0.598.0>}
[info] Sent 200 in 2ms
[error] Ranch listener VhsWeb.Endpoint.HTTP had connection process started with :cowboy_clear:start_link/4 at #PID<0.594.0> exit with reason: {:function_clause, [{:cowboy_http, :commands, [{:state, #PID<0.471.0>, VhsWeb.Endpoint.HTTP, #Port<0.22>, :ranch_tcp, :undefined, %{env: %{dispatch: [{:_, [], [{:_, [], Phoenix.Endpoint.Cowboy2Handler, {VhsWeb.Endpoint, []}}]}]}, stream_handlers: [:cowboy_telemetry_h, :cowboy_stream_h]}, "", %{}, {{127, 0, 0, 1}, 34624}, {{127, 0, 0, 1}, 4000}, :undefined, #Reference<0.3145460136.2965635073.81437>, 2, {:ps_request_line, 0}, :infinity, 1, :done, 100, [{:stream, 1, {:cowboy_telemetry_h, {:state, {:cowboy_stream_h, {:state, :undefined, VhsWeb.Endpoint.HTTP, #PID<0.595.0>, :undefined, :undefined, :undefined, :undefined, 0, :fin, "", 180, :undefined, ...}}, #Function<0.122385210/1 in :cowboy_telemetry_h."-fun.metrics_callback/1-">, :undefined, %{body_length: 180, cert: :undefined, has_body: true, headers: %{"accept" => "*/*", "accept-encoding" => "gzip, deflate, br", "connection" => "keep-alive", "content-length" => "180", "content-type" => "application/json", "host" => "localhost:4000", "postman-token" => "ad48096e-c775-4f82-a860-a4d4b4286c9c", "user-agent" => "PostmanRuntime/7.26.8"}, host: "localhost", method: "POST", path: "/api/transaction", peer: {{127, 0, 0, ...}, 34624}, pid: #PID<0.594.0>, port: 4000, qs: "", ref: VhsWeb.Endpoint.HTTP, ...}, "200 OK", %{"cache-control" => "max-age=0, private, must-revalidate", "content-length" => "15", "content-type" => "application/json; charset=utf-8", "date" => "Mon, 03 May 2021 20:41:43 GMT", "server" => "Cowboy", "x-request-id" => "FnupLbYol1vbVZsAAABE"}, VhsWeb.Endpoint.HTTP, -576460686234141769, :undefined, -576460686234115433, -576460686234115433, -576460686203746015, -576460686203746015, %{#PID<0.595.0> => %{spawn: -576460686234129902}}, [], 180, ...}}, "POST", :"HTTP/1.1", :undefined, :undefined, 0, []}], [{:child, #PID<0.595.0>, 1, 5000, :undefined}]}, 1, [{:response, "200 OK", %{"cache-control" => "max-age=0, private, must-revalidate", "content-length" => "15", "content-type" => "application/json; charset=utf-8", "date" => "Mon, 03 May 2021 20:41:43 GMT", "server" => "Cowboy", "x-request-id" => "FnupLbYol1vbVZsAAABE"}, ["{\"", [[] | "status"], "\":", [34, [[] | "Ok"], 34], 125]}]], [file: '/home/edwin/dev/phoenix-projects/vhs/deps/cowboy/src/cowboy_http.erl', line: 922]}, {:cowboy_http, :loop, 1, [file: '/home/edwin/dev/phoenix-projects/vhs/deps/cowboy/src/cowboy_http.erl', line: 231]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 226]}]}
2

Calling GenServer.start_link from the controller means the newly-started processes will be terminated when the controller process terminates. Exactly how important this is depends on how long TransactionSubscriptionHandler processes are expected to live.

I can’t speak to any particular issues with your GenServer (not posted), but there are some issues with the controller:

  • json sends the response so it shouldn’t be called repeatedly. It will give a Plug.Conn.AlreadySentError if it is passed a conn that knows the response has already been sent
  • also, call_hash needs to return the updated conn after using it. This is likely the cause of the unusual error deep in the internals of cowboy_http: the second call to call_hash gets passed a conn which is out-of-sync with the underlying Cowboy machinery

To fix this, you’ll need to decide what getting an error partway through the list of hashes should do - stop and leave the existing processes? Clean up? Keep going? - and then whatever makes that decision can render a JSON response.

1 Like
3

I am still new to elixir can you demonstrate this in an example, lets say

["1","2"] |> Enum.each(fn call_hash(conn,hash)end)

how can I structure call_hash to return the updated conn and handle the error

4 
def new(conn, %{"hashes" => [_ | _] = hashes}) do
  results = Enum.map(hashes, &call_hash/1)

  if Enum.all?(results, & &1 == :ok) do
    json(conn, %{status: "Ok"})
  else
    conn
    |> put_status(400)
    |> json(%{status: "Failed"})
  end
end

defp call_hash(conn,hash) do
  with {:ok, _} <- GenServer.start_link(TransactionSubscriptionHandler, %{hash: String.to_atom(hash)}, name: String.to_atom(hash))
     {:ok, %{status: 200}} <- BlockNative.subscribe_transaction(hash) do
     :ok
  else
     _ -> :error
  end
end

The big changes:

  • split “doing the thing” apart from “rendering the response”; call_hash could be moved out of the controller if desired, and doesn’t need to know anything about conn
  • use Enum.map instead of Enum.each. Each always returns just the atom :ok, and we care about the results here.
2 Likes
5

Thanks@al2o3cr it worked