Warning: Ecto.Repo.insert_all/3 is undefined or private

(Please note: Noob here who just started learning Ecto. If you can, please ELI5, thanks.)

Snippet:

defmodule Wallet do
    use Ecto.Repo,
    otp_app: :arbit,
    adapter: Ecto.Adapters.Postgres
    alias Ecto.Repo
    #use Genserver

    require ArbitDB.WalletBalance

    def refresh_db do
        updated_balances = liquid_holdings()
        Repo.insert_all(WalletBalance, updated_balances, 
            on_conflict: :replace_all_except_primary_key, conflict_target: :value_USD)
    end



$ iex -S mix
Erlang/OTP 22 [erts-10.6.4] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]

Compiling 1 file (.ex)
warning: Ecto.Repo.insert_all/3 is undefined or private

What am I missing? Thanks for taking a sec to assist :slight_smile:

2 Likes

Ecto.Repo.insert_all/3 is a callback, so typically you will be calling insert_all/3 from a module that has the Ecto.Repo behaviour. Ecto.Repo itself doesn’t have it. I believe you’ll want Wallet.insert_all..., since the use Ecto.Repo drags in a bunch of boilerplate and turn the module that invokes use into one that implements Ecto.Repo behaviour (be ware that not all use statements do this sort of thing as a general rule, look up the code or docs for each module’s __using__ macro to understand what’s going on on a case-by-case basis).

1 Like

Thanks for the insight! :slight_smile:

defmodule Wallet do
	use Ecto.Repo,
	otp_app: :arbit,
	adapter: Ecto.Adapters.Postgres
	#use Genserver

	require ArbitDB.WalletBalance
	alias ArbitDB.WalletBalance

	require ArbitDB.Repo
	alias ArbitDB.Repo

	
	def refresh_db do
		updated_balances = liquid_holdings()
		Repo.insert_all(WalletBalance, updated_balances, 
			on_conflict: :replace_all_except_primary_key, conflict_target: :value_USD)
	end
iex(1)> recompile
Compiling 1 file (.ex)
:ok

iex(2)> Wallet.refresh_db
** (RuntimeError) could not lookup Ecto repo ArbitDB.Repo because it was not started or it does not exist
    (ecto 3.2.2) lib/ecto/repo/registry.ex:19: Ecto.Repo.Registry.lookup/1
    (ecto 3.2.2) lib/ecto/repo/schema.ex:35: Ecto.Repo.Schema.do_insert_all/6

Arbit/lib/repo.ex

defmodule ArbitDB.Repo do
	use Ecto.Repo,
	otp_app: :arbit,
	adapter: Ecto.Adapters.Postgres

end

Why isn’t it seeing the repo?

read the docs for ecto config, I think you missed something in there.

It looks like it is in your dependencies.

Normally you would have your own Repo module - something like this:

defmodule MyApp.Repo do
use Ecto.Repo,
    otp_app: :arbit,
    adapter: Ecto.Adapters.Postgres
end

If you have used generators to build your app skeleton you should already have this - look for repo.ex.

You would then alias that rather than Ecto.Repo

defmodule Wallet do
    alias MyApp.Repo
    alias ArbitDB.WalletBalance

    def refresh_db do
        updated_balances = liquid_holdings()
        Repo.insert_all(WalletBalance, updated_balances, 
            on_conflict: :replace_all_except_primary_key, conflict_target: :value_USD)
    end
end
1 Like

Not sure why I didn’t see earlier replies…

To understand all the pieces required probably the easiest way is to create a new Phoenix application in a separate directory with the defaults - see https://hexdocs.pm/phoenix/up_and_running.html

Once you have done that, these are the main pieces that come together to make Ecto work…

In config/config.exs you will see configuration for the Ecto repository. In lib/[app name]/application.ex you will see the bootstrap code for the Ecto repository (in the start function the child “applications” are started, one of which is the Ecto repository). In lib/[app name]/repo.ex you will see the macro invocation to create your application specific Ecto Repo (use Ecto.Repo is injecting a whole lot of implementation code into your repo module).

What you don’t need is use Ecto.Repo anywhere except your one application-specific Repo module. You also shouldn’t need the require ArbitDB... statements in addition to the alias statements unless you are doing something funky with macros in your application code. And if you are doing something funky with macros, you probably shouldn’t until you have everything else working nicely!

1 Like

Thanks so much for your detailed and patient walk-through. Here’s the refactored code:

defmodule Wallet do

	alias ArbitDB.{WalletBalance, Repo}


	def refresh_db do
		updated_balances = liquid_holdings()
		Repo.insert_all(WalletBalance, updated_balances, 
			on_conflict: :replace_all_except_primary_key, conflict_target: :value_USD)
	end

The config.exs:

import Config

config :arbit, ArbitDB.Repo,
  adapter: Ecto.Adapters.Postgres,
  database: "ArbitDB",
  username: "nexus",
  password: "nexus",
  hostname: "localhost",
  migration_timestamps: [type: :utc_datetime_usec]

config :arbit,
  ecto_repos: [ArbitDB.Repo]

Verifying with Postgres:

$ psql ArbitDB
psql (12.2 (Ubuntu 12.2-1.pgdg18.04+1), server 11.7 (Ubuntu 11.7-1.pgdg18.04+1))
Type "help" for help.

ArbitDB=# \dt
             List of relations
 Schema |       Name        | Type  | Owner 
--------+-------------------+-------+-------
 public | schema_migrations | table | nexus
 public | wallet_balance    | table | nexus
(2 rows)

Bootstrap code:

defmodule Arbit.Application do
  # See https://hexdocs.pm/elixir/Application.html
  # for more information on OTP Applications
  @moduledoc false

  use Application

  def start(_type, _args) do
    # List all child processes to be supervised
    children = [
      ArbitDB.Repo
    ]

    # See https://hexdocs.pm/elixir/Supervisor.html
    # for other strategies and supported options
    opts = [strategy: :one_for_one, name: Arbit.Supervisor]
    Supervisor.start_link(children, opts)
  end
end

Schema:

defmodule ArbitDB.WalletBalance do
	use Ecto.Schema
	import Ecto.Changeset

	schema "wallet_balance" do
		field :asset_id, :string, null: false
		field :name, :string, null: false
		field :amount, :float, null: false
		field :value_WAVES, :float
		field :value_USD, :float		
		timestamps(type: :utc_datetime)
	end

	def changeset(wallet_balance, params \\ %{}) do 
		wallet_balance
		|> Ecto.Changeset.cast(params, [:asset_id, :name, :amount, :value_WAVES, :value_USD])
		|> validate_required([:asset_id, :name, :amount, :value_WAVES, :value_USD])
	end


end

Error message:

iex(3)> Wallet.refresh_db
** (RuntimeError) could not lookup Ecto repo ArbitDB.Repo because it was not started or it does not exist
    (ecto 3.2.2) lib/ecto/repo/registry.ex:19: Ecto.Repo.Registry.lookup/1
    (ecto 3.2.2) lib/ecto/repo/schema.ex:35: Ecto.Repo.Schema.do_insert_all/6

What am I overlooking?

Does your mix.exs in the root directory of the project have an application function? Have a read of the docs here - it may be that your application module isn’t being invoked on startup. https://hexdocs.pm/elixir/Application.html#module-the-application-callback-module

If you have any more issues it would be good to understand how you created your project. Most of the wireup is done by the project generators normally.

defmodule Arbit.MixProject do
	use Mix.Project

	def project do
		[
			app: :arbit,
			version: "0.1.0",
			elixir: "~> 1.10",
			start_permanent: Mix.env() == :prod,
			deps: deps()
		]
	end


	def application do
		[
			extra_applications: [:logger],
			mod: {Arbit.Application, []}
		]
	end


	defp deps do
		[
			{:httpoison, "~> 1.6"},
			{:poison, "~> 4.0"},
			{:postgrex, "~> 0.15.1"},
			{:ecto_sql, "~> 3.2.0"},	
		]
	end
end
iex(1)> recompile
Compiling 1 file (.ex)
:ok
iex(2)> Wallet.refresh_db
** (RuntimeError) could not lookup Ecto repo ArbitDB.Repo because it was not started or it does not exist
    (ecto 3.2.2) lib/ecto/repo/registry.ex:19: Ecto.Repo.Registry.lookup/1
    (ecto 3.2.2) lib/ecto/repo/schema.ex:35: Ecto.Repo.Schema.do_insert_all/6

If memory serves, I created it using $ mix new arbit. Where else should I look?

How does your supervision tree look like? Have you followed the ecto docs how to integrate ecto in an applications supervision tree?

defmodule Arbit.Application do
  # See https://hexdocs.pm/elixir/Application.html
  # for more information on OTP Applications
  @moduledoc false

  use Application

  def start(_type, _args) do
    # List all child processes to be supervised
    children = [
      ArbitDB.Repo
    ]

    # See https://hexdocs.pm/elixir/Supervisor.html
    # for other strategies and supported options
    opts = [strategy: :one_for_one, name: Arbit.Supervisor]
    Supervisor.start_link(children, opts)
  end
end

What happens if you do Application.ensure_all_started(:arbit)

$ iex -S mix
Erlang/OTP 22 [erts-10.6.4] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]

Interactive Elixir (1.10.1) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> Application.ensure_all_started(:arbit)
{:ok, []}
iex(2)> Wallet.refresh_db                     

11:01:27.299 [debug] QUERY ERROR db=8.1ms queue=1.3ms
INSERT INTO "wallet_balance" ("amount","asset_id","name","value_USD","value_WAVES") VALUES ($1,$2,$3,$4,$5),($6,$7,$8,$9,$10),($11,$12,$13,$14,$15),($16,$17,$18,$19,$20),($21,$22,$23,$24,$25),($26,$27,$28,$29,$30),($31,$32,$33,$34,$35),($36,$37,$38,$39,$40),($41,$42,$43,$44,$45),($46,$47,$48,$49,$50),($51,$52,$53,$54,$55),($56,$57,$58,$59,$60),($61,$62,$63,$64,$65),($66,$67,$68,$69,$70),($71,$72,$73,$74,$75),($76,$77,$78,$79,$80),($81,$82,$83,$84,$85) ON CONFLICT ("value_USD") DO UPDATE SET "asset_id" = EXCLUDED."asset_id","name" = EXCLUDED."name","amount" = EXCLUDED."amount","value_WAVES" = EXCLUDED."value_WAVES","value_USD" = EXCLUDED."value_USD","inserted_at" = EXCLUDED."inserted_at","updated_at" = EXCLUDED."updated_at" [6.0e-6, "51LxAtwBXapvvTFSbbh4nLyWFxH6x8ocfNvrXxbTChze", "TORCorp", 4.4477100000000006e-9, 1.7442000000000002e-9, 0.05218729, "474jTeYx2r2Va35794tCScAXWJG9hU2HcgxzMowaZUnu", "WETH", 25.83157508840351, 10.13002944643275, 1.0, "81HnQ1PWViWUiv39NGkPjbfDehW1edjeJe6XxVhw1TPZ", "MGOLD", 0.0255, 0.01, 5.0e-8, "EbLVSrAi6vS3AkLwBinzZCvAXP2yYiFJEzj1MBVHcwZ5", "ASIMI", 3.952499999999999e-9, 1.5499999999999998e-9, 250.0, "BS1KFNR8zrXKBEWdUUvpaP6G57Hic3aESkwK7qQKdLpB", "Marquise Museum", 0.01036575, 0.004065, 2.176e-5, "HZk1mbfuJpmxU1Fs4AX5MWLVYtctsNcg6e2C6VKqK8zk", "Litecoin", 0.0029255365569734394, 0.0011472692380287998, 1.6444e-4, "62LyMjcr2DtiyF5yVXFhoQ2q414VPPJXjsNYp72SuDCH", "Bitcoin SV", 0.08026650180903983, 0.0314770595329568, 21.08167447, "8Yw4QmskrQauQeNjgh2fTQ4swmkNm85GTQzdHEf6QdUU", "ABOT", 0.09669069940484006, 0.0379179213352314, 0.01, "DHgwrRvVyqJsepd32YbBqUeDH4GJ1N984X8QoekjgH8J", "WavesCommunity", 0.0011225153549999998, 4.402021e-4, 47.92501839, "4LHHvYGNKJUg5hj65aGD5vgScvCBmLpdRFtjokvCjSL8", "Vostok", 5.3405256463776185, 2.0943237828931838, ...]
** (Postgrex.Error) ERROR 42P10 (invalid_column_reference) there is no unique or exclusion constraint matching the ON CONFLICT specification
    (ecto_sql 3.2.0) lib/ecto/adapters/sql.ex:629: Ecto.Adapters.SQL.raise_sql_call_error/1
    (ecto_sql 3.2.0) lib/ecto/adapters/sql.ex:538: Ecto.Adapters.SQL.insert_all/8
    (ecto 3.2.2) lib/ecto/repo/schema.ex:54: Ecto.Repo.Schema.do_insert_all/6
iex(2)>

Seems like it’s working now!

Yup, yup! Thanks everyone for your kind assistance.

Now onto the next bug! :slight_smile: