Questions & Help>Questions

Confusing behavior of `optional` deps in `mix.exs`

#mix
david_ex

david_ex

I’m struggling with optional: true dependencies in one of my projects. This project will wrap an API (complete code here), and users will be able to provide their own functions/modules to handle the HTTP interaction and response body decoding if they wish. However, default implementations of these will ship with the library.

These default implementations depend on :hackney and Jason. Having learned about optional dependencies, they seemed like exactly what I was looking for (docs):

:optional - marks the dependency as optional. In such cases, the current project will always include the optional dependency but any other project that depends on the current project won’t be forced to use the optional dependency. However, if the other project includes the optional dependency on its own, the requirements and options specified here will also be applied.

(The file is here if you want to poke around.)

If I change

  defp deps do
    [
      {:hackney, "~> 1.14", only: [:dev, :test]},
      {:jason, "~> 1.1", only: [:dev, :test]},
      ...
    ]
  end

to

  defp deps do
    [
      {:hackney, "~> 1.14", optional: true},
      {:jason, "~> 1.1", optional: true},
      ...
    ]
  end

I get strange results. First, dialyzer (used via dialyxir) no longer runs successfully and I get these errors:

:0:unknown_function
Function Jason.decode/1 does not exist.
________________________________________________________________________________
:0:unknown_function
Function Jason.encode/2 does not exist.
________________________________________________________________________________
:0:unknown_function
Function :hackney.body/1 does not exist.
________________________________________________________________________________
:0:unknown_function
Function :hackney.request/5 does not exist.
________________________________________________________________________________
done (warnings were emitted)

Then, 2 tests using hackney fail, although others tests also using hackney still pass. The tests failing are these.

What’s going on?

Per my understanding of the docs, using an optional dependency shouldn’t change any of these results since the dependencies would be included in my project. They simply wouldn’t be forced on the users of my package, and would only make the version requirements apply if those dependencies were already included in their own project’s mix file.

11 comments
#mix
4022 11

Showing Posts 1 to 10

ericmj

ericmj

Elixir Core Team

I cannot reproduce these errors. When I set the hackney and jason dependencies as optional tests pass cleanly and I get no errors from dialyzer.

What Elixir version are you using? Try deleting the _build and deps directories and do a clean mix deps.get and retry.

2
Post #1
david_ex

david_ex OP

Thanks for your help!

It seems my issues are caused by the :hackney and :jason OTP apps not getting started when configured as optional dependencies. It appears the application inference introduced in Mix 1.4 isn’t handling this case correctly, as it can be fixed by ensuring :hackney and :jason are started (by adding them to :extra_applications). Is this a bug in Mix, or is there some special handling I should be doing for optional OTP apps that should get started if available?

Reproduction

I have pushed a clean branch reproducing the issue.

Env info:

$ elixir -v
Erlang/OTP 21 [erts-10.0.5] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]

Elixir 1.7.2 (compiled with Erlang/OTP 20)

Steps to reproduce (when on commit e4a85376def2ca2bcb1671ea812714f83b02e132):

$ rm -rf _build/
$ rm -rf deps/
$ mix deps.get
# output omitted
$ mix test
# compilation messages omitted
==> scrapy_cloud_ex
Compiling 23 files (.ex)
Generated scrapy_cloud_ex app
Starting HTTParrot on port 8080
Starting HTTParrot on port 8433 (SSL)
Starting HTTParrot on unix socket httparrot.sock
..................................................................................................................................................................

  1) test request/1: POST requests add included body (ScrapyCloudEx.HttpAdapters.DefaultTest)
     test/http_adapters/default_test.exs:113
     ** (ArgumentError) argument error
     code: test_included_body(request)
     stacktrace:
       (stdlib) :ets.lookup_element(:hackney_config, :mod_metrics, 2)
       /home/david/Documents/dev/scrapy_cloud_ex/deps/hackney/src/hackney_metrics.erl:27: :hackney_metrics.get_engine/0
       /home/david/Documents/dev/scrapy_cloud_ex/deps/hackney/src/hackney_connect.erl:71: :hackney_connect.create_connection/5
       /home/david/Documents/dev/scrapy_cloud_ex/deps/hackney/src/hackney_connect.erl:39: :hackney_connect.connect/5
       /home/david/Documents/dev/scrapy_cloud_ex/deps/hackney/src/hackney.erl:328: :hackney.request/5
       (scrapy_cloud_ex) lib/http_adapters/default.ex:52: ScrapyCloudEx.HttpAdapters.Default.make_request/6
       (scrapy_cloud_ex) lib/http_adapters/default.ex:26: ScrapyCloudEx.HttpAdapters.Default.request/1
       test/http_adapters/default_test.exs:66: ScrapyCloudEx.HttpAdapters.DefaultTest.CommonTests.test_included_body/1
       test/http_adapters/default_test.exs:114: (test)

.

  2) test request/1: PUT requests add included body (ScrapyCloudEx.HttpAdapters.DefaultTest)
     test/http_adapters/default_test.exs:121
     ** (ArgumentError) argument error
     code: test_included_body(request)
     stacktrace:
       (stdlib) :ets.lookup_element(:hackney_config, :mod_metrics, 2)
       /home/david/Documents/dev/scrapy_cloud_ex/deps/hackney/src/hackney_metrics.erl:27: :hackney_metrics.get_engine/0
       /home/david/Documents/dev/scrapy_cloud_ex/deps/hackney/src/hackney_connect.erl:71: :hackney_connect.create_connection/5
       /home/david/Documents/dev/scrapy_cloud_ex/deps/hackney/src/hackney_connect.erl:39: :hackney_connect.connect/5
       /home/david/Documents/dev/scrapy_cloud_ex/deps/hackney/src/hackney.erl:328: :hackney.request/5
       (scrapy_cloud_ex) lib/http_adapters/default.ex:52: ScrapyCloudEx.HttpAdapters.Default.make_request/6
       (scrapy_cloud_ex) lib/http_adapters/default.ex:26: ScrapyCloudEx.HttpAdapters.Default.request/1
       test/http_adapters/default_test.exs:66: ScrapyCloudEx.HttpAdapters.DefaultTest.CommonTests.test_included_body/1
       test/http_adapters/default_test.exs:122: (test)

...................

Finished in 0.8 seconds
184 tests, 2 failures

Randomized with seed 511043

Here’s the root issue:

$ iex -S mix
# compilation messages omitted
iex(1)> ScrapyCloudEx.Endpoints.App.Comments.get("foobar", "1/2/3")
# note: `:hackney.request("https://www.example.com")` would also fail
# debug data omitted

** (ArgumentError) argument error
    (stdlib) :ets.lookup_element(:hackney_config, :mod_metrics, 2)
    /home/david/Documents/dev/scrapy_cloud_ex/deps/hackney/src/hackney_metrics.erl:27: :hackney_metrics.get_engine/0
    /home/david/Documents/dev/scrapy_cloud_ex/deps/hackney/src/hackney_connect.erl:71: :hackney_connect.create_connection/5
    /home/david/Documents/dev/scrapy_cloud_ex/deps/hackney/src/hackney_connect.erl:39: :hackney_connect.connect/5
    /home/david/Documents/dev/scrapy_cloud_ex/deps/hackney/src/hackney.erl:328: :hackney.request/5
    (scrapy_cloud_ex) lib/http_adapters/default.ex:52: ScrapyCloudEx.HttpAdapters.Default.make_request/6
    (scrapy_cloud_ex) lib/http_adapters/default.ex:26: ScrapyCloudEx.HttpAdapters.Default.request/1
    (scrapy_cloud_ex) lib/endpoints/helpers.ex:46: ScrapyCloudEx.Endpoints.Helpers.make_request/1

iex(1)> Application.ensure_all_started(:hackney)
{:ok,
 [:unicode_util_compat, :idna, :mimerl, :certifi, :ssl_verify_fun, :metrics,
  :hackney]}

iex(2)> ScrapyCloudEx.Endpoints.App.Comments.get("foobar", "1/2/3")
# debug data omitted

{:error, %{message: "Authentication failed", status: 403}}
# this is the expected response from the API given the dummy data

If I add :hackney to the :extra_applications in mix.exs, I can run the above mix example correctly and the tests also execute properly.

If I then also add :jason to :extra_applications, the dialyzer errors also go away.

1
Post #2
ericmj

ericmj

Elixir Core Team

The issue here is that optional dependencies are not started by default (this may change in Elixir 1.8) so you need to start the, manually when running tests or doing local development.

The original reasoning for this behavior was that you wouldn’t be able to test how your application functioned without the dependencies if mix always started them.

3
Post #3
david_ex

david_ex OP

Is this documented somewhere, or would it be helpful for me to open a PR to add this info to the docs?

2
Post #4
ericmj

ericmj

Elixir Core Team

No, it’s not documented afaict but the behavior is changed in Elixir master so we can only document on the v1.7 branch.

0
Post #5
david_ex

david_ex OP

I found this commit making the change on master. I assume when you said “this may change” in 1.8, it was in the spirit of “you can’t be sure until it’s released”, but that this change is likely to ship in 1.8?

Also I didn’t understand whether it would be helpful to open PR clarifying the docs for :optional in v 1.4 - 1.7, or if in your opinion it would just add noise?

In any case, thanks a lot for your help with my issue!

1
Post #6
ericmj

ericmj

Elixir Core Team

I said “may” because we have noticed that it causes issues when compiling mix projects from rebar3. Others may also rely on the original behavior so there is a chance we will roll it back.

1
Post #7
yordisprieto

yordisprieto

Ran into this issue. I had to remove the optional flag from Plug to make the dialyzer happy.

Any idiomatic way to fix such a problem?

1
Post #8
mtrudel

mtrudel

Creator of Bandit

I’ve solved this by selectively starting extra applications as necessary based on Mix.env():

https://github.com/mtrudel/websock_adapter/blob/main/mix.exs#L25-L29

4
Post #9
mtrudel

mtrudel

Creator of Bandit

An even better solution to this is to use the plt_add_apps arg to dialyzer in your mix.exs file:

defp dialyzer do
    [plt_add_apps: [:cowboy], ...]
end

See Use `plt_add_apps` arg to dialyzer to bring cowboy into analysis · phoenixframework/websock_adapter@507229d · GitHub for an example!

3
Post #10

Where Next? Top

View thread on forum (has 11 replies!)
mix
Home Questions & Help>Questions
20 3852 11
Last post
REPLY VIEW THREAD WITH 11 POSTS

Trending in Questions Top

katta
Questions & Help>Questions

Phoenix Ecto realistic benchmarks
I having some trouble figuring out if I have set myself too strict of standards for my production server. Currently I can handle 75% of r...
/phoenix#ecto#troubleshooting
2 1850 12
New
achenet
Questions & Help>Questions

No Tailwind binary for FreeBSD
Hello, I’m trying to build a basic Phoenix web-app, and I’d like to use Tailwind. However, when I launch mix phx.server, I get an error...
/phoenix#freebsd#tailwindcss#tailwind
11 1073 14
New
kpanic
Questions & Help>Questions

"Mentoring" match maker? maybe MVP
Hi everyone, I am toying with the idea of building a “match maker” for giving personal help to people that wants to start coding. I sta...
#helpers#programming
8 940 4
New
Cxx-mlr
Questions & Help>Questions

Using try/catch with update_in/3
I’m working on a small exercise involving update_in/3, and I came up with this solution: data = %{ name: "Periodic Table", category:...
#learning-elixir
9 273 12
New
ChrisAmelia
Questions & Help>Questions

Need help understanding order of function calls from Phoenix auth
I’ve got trouble wrapping my head around the order in which functions are called in this snippet (from Phoenix’s authentication): toke...
/phoenix#learning-elixir
0 76 2
New
dillonoconnor
Questions & Help>Questions

Hologram compiler and iex -S mix
Is there any way to avoid the Hologram compiler running when using iex? It seems like the front-end code could potentially be disregarded...
/hologram#iex
2 386 3
New
thiagogsr
Questions & Help>Questions

Oban Worker max_attempts doesn't compile with a constant
** (ArgumentError) expected :max_attempts to be a positive integer, got: {:@, [line: 10, column: 19], [{:max_attempts, [line: 10, column:...
/oban
0 66 1
New

Other Trending Topics Top

GenericJam
News>Announcing

Mob - Native BEAM / Elixir on Native Mobile
Edit: 2026 May 15 - This post is archived. Mob is alive!! Main docs: mob v0.7.11 — Documentation A bit of explanation for the slightly c...
/mob#library#beam#mobile#liveview#react-native#mobile-app#mobile-development
259 8396 111
New
mudasobwa
Chat & Discussions>Discussions

Cure - a new language compiled to BEAM
I am happy to introduce the very α version of the new programming language compiled to BEAM. Welcome Cure. It has literally three kille...
#static-types#type-checking#types#fsm
171 6315 70
New
garrison
News>Announcing

Hobbes - a low-level distributed database for the Elixir programming language
Hobbes is a low-level distributed database for the Elixir programming language. Hobbes provides a simple, safe, and scalable storage lay...
#library#database#distributed#hobbes
306 6242 88
New
budgie
Chat & Discussions>Discussions

I genuinely hate programming now
A little off-topic, but I feel like people here have a good head on their shoulders. I used to be quite good at making software. Was luc...
#ai#career#rant
460 3763 113
New
KristerV
Chat & Discussions>AI / LLMs

State of developing agents with Elixir (not coding agents)
Hey. Is there anyone here who creates agents in their apps? Not talking about using agents, but creating them. I’m finding it pretty diff...
#agent#ai#llm#dev-tools#ai-tools
31 3653 21
New
mcass19
News>Announcing

ExRatatui - Elixir bindings for the Rust ratatui terminal UI library
ExRatatui lets you cook up rich terminal UIs in Elixir, powered by Rust’s ratatui via Rustler NIFs. Build interactive terminal applicatio...
#library#rust#terminal#tui
60 3370 16
New
Back to top

Questions & Help>Questions

Questions Questions ❯

Latest on Elixir Forum

Elixir Forum ❯

Sponsor Spotlight

AppSignal

Catch errors, track performance, monitor hosts and more.

Our Sponsors

Dashbit

Build Elixir applications with speed and confidence.

Coders51

We build reliable cloud platforms for business-critical systems.

AppSignal

Catch errors, track performance, monitor hosts and more.

Honeybadger

Error tracking, logs, uptime, and Just Enough APM. Built for developers.

Code Sync

Elixir, Erlang & BEAM conferences from Code Sync - join us in 2026!

Manning

Master modern tools, write cleaner code, and tackle real-world challenges.

Defacto

Bringing learning, work and compliance together since 1987.

WyeWorks

**Building the future of software with AI, trust, and human connection.**

Software Mansion

The team behind Membrane, Popcorn, LiveDebugger. Available for hire.

Pragmatic Studio

Courses that'll move you from confusion to "Aha, now I get it!"

View all our sponsors ❯

Categories:

Sub Categories:

Forums

Popular Tags

Our Sponsors

Dashbit

Build Elixir applications with speed and confidence.

Coders51

We build reliable cloud platforms for business-critical systems.

AppSignal

Catch errors, track performance, monitor hosts and more.

Honeybadger

Error tracking, logs, uptime, and Just Enough APM. Built for developers.

Code Sync

Elixir, Erlang & BEAM conferences from Code Sync - join us in 2026!

Manning

Master modern tools, write cleaner code, and tackle real-world challenges.

Defacto

Bringing learning, work and compliance together since 1987.

WyeWorks

**Building the future of software with AI, trust, and human connection.**

Software Mansion

The team behind Membrane, Popcorn, LiveDebugger. Available for hire.

Pragmatic Studio

Courses that'll move you from confusion to "Aha, now I get it!"

View all our sponsors ❯

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews