TELunus

TELunus

problem asynchronizing ecto calls

I seem to be having an issue making some code asynchronous. I’m running

    stream = Task.Supervisor.async_stream_nolink(
      Module.TaskSupervisor,
      enum,
      fn (item) -> my_func(item, 2) end,
      [ordered: false, max_concurrency: 1]
    )
    results = Enum.to_list(stream)

which gives me 12:31:22.290 [error] Postgrex.Protocol (#PID<0.678.0>) disconnected: ** (DBConnection.ConnectionError) owner #PID<0.1768.0> exited while client #PID<0.1771.0> is still running with: shutdown

part of the code that’s run by my_func is an ecto call

IO.puts("before")
Repo.get(schema, id)
IO.puts("after")

and I’m seeing the "before" get printed right before the error message, but I’m not seeing the "after"
On the other hand if I change my code to run synchronously as

    stream = Enum.map(enum, fn (item) -> my_func(item, 2) end)
    results = Enum.to_list(stream)

then everything works fine. There’s no error message, and both print statements get run.

Any thoughts on what I could be doing wrong or how I can fix this?

Most Liked

alco

alco

Just wanted to comment on this. The team working on Elixir does their best to keep Elixir versions backwards-compatible. The worst thing you should get after upgrading to a newer version is a bunch of new deprecation warnings.

I would actually encourage you to upgrade early because if something does get deprecated, you can stop doing that in your code and reduce the amount of work that you would otherwise have to do if you decided to upgrade months later. Dependencies also keep working as before as a rule, but since you may start getting new warnings, that can be something that prompts you to look into gradually upgrading your dependencies as well sooner rather than later. This is again going to be easier than skipping a few version of Elixir and getting a lot more issues with dependencies when you finally decide to upgrade.

TELunus

TELunus

It looks like the project is already doing that. The test file starts with

defmodule Project.Module do
    use Project.ModelCase

And in model_case.ex we’ve got

  setup tags do
    :ok = Ecto.Adapters.SQL.Sandbox.checkout(Project.Repo)

    unless tags[:async] do
      Ecto.Adapters.SQL.Sandbox.mode(Project.Repo, {:shared, self()})
    end

    :ok
  end

So since :async isn’t included in the use line I believe that means we’re using shared mode, so any process should be able to connect to the DB connection.

Last Post!

TELunus

TELunus

It does seem to be an issue in our test setup, as it looks like you were suggesting, since if I run the same code outside of the test environment it returns correctly, and the message from IO.puts("after") following the call to the Ecto.Repo.get callback shows up.

I suspect you’re right that the issue is outside of what I’ve shared, but I’m uncertain where to look next.

The test file looks roughly (modified to preserve confidentiality) like:

defmodule Project.ModuleTest do
  use Project.ModelCase

  alias Project.{Module, Factory, TestHelpers}

  @moduletag :module_tag

  setup do
    variables = values

    %{
      more_variables: more values,
      cleanup_func: cleanup_func
    } = TestHelpers.setup_stuff_we_need(variable, other_variable)

    on_exit(cleanup_func)

    built_variables = Factory.build(variables)

    %{
      names: some_variables
    }
  end

  describe "#description" do
    test "test description", %{
      names: some_variables
    } do
      assert {:ok, %{
        specific_thing_we_need_to_check: false,
        id: result_id
      }} = Module.function(thing_id, and, some, values)

      # wait for a different second so records that only have 1 second resolution don't collide
      Process.sleep(2000)

      assert {:ok, %{
        specific_thing_we_need_to_check: true,
        something: ^specific_value,
        thing_id: ^thing_id
      }} = Module.function_that_calls_my_func(thing_id, and, some, values) # <--- the line that actually creates the issue
    end

  end
end

And the file that defines Project.ModelCase looks like

defmodule Project.ModelCase do

  use ExUnit.CaseTemplate

  using do
    quote do
      alias Project.Repo
      import Ecto
      import Ecto.Query, only: [from: 2]
      import Project.ModelCase
    end
  end

  setup tags do
    :ok = Ecto.Adapters.SQL.Sandbox.checkout(Project.Repo)

    unless tags[:async] do
      Ecto.Adapters.SQL.Sandbox.mode(Project.Repo, {:shared, self()})
    end

    :ok
  end

  def errors_on(changeset) do
    Ecto.Changeset.traverse_errors(changeset, fn {message, opts} ->
      Enum.reduce(opts, message, fn {key, value}, acc ->
        String.replace(acc, "%{#{key}}", to_string(value))
      end)
    end)
  end
end

We’ve also got some stuff in config/test.exs, and taking some guesses as to what might be relevant we’ve got

use Mix.Config

config :project, Project.Repo
  adapter: Ecto.Adapters.Postgres,
  username: System.get_env("TEST_USERNAME"),
  password: System.get_env("TEST_PASSWORD") ,
  database: System.get_env("TEST_DATABASE"),
  pool_size: 3,
  pool: Ecto.Adapters.SQL.Sandbox,
  types: Project.PostgresTypes,
  ownership_timeout: 10 * 60 * 1000

Where Next?

Popular in Questions Top

lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
Lily
In templates/appointment/index.html.eex: &lt;%= for appointment &lt;- @appointments do %&gt; &lt;tr&gt; &lt;td&gt;&lt;%= appoi...
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
shijith.k
I am trying to start a new phoenix project with elixir 1.9, but mix phx.new does not work. It says that ** (Mix) The task "phx.new" could...
New
alice
Hey, Just curious what are the main benefits of Elixir compared to Clojure? When is Elixir more useful than Clojure and vice versa? Th...
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New

Other popular topics Top

JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
shijith.k
I am trying to start a new phoenix project with elixir 1.9, but mix phx.new does not work. It says that ** (Mix) The task "phx.new" could...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
jason.o
In the code below, if the create action is not set to accept “extra_key” as an input, it errors out with a message shown above. Is there ...
New

We're in Beta

About us Mission Statement