Dialyzer gets nested map wrong and errors out ([number] vs. map)

Hi there fellow Elixir friends,

in the latest Benchee released I introduced structs and type specs, all worked fine until I wanted to release the last plugin. It worked fine when running against master of the core library through a github dependency build but when I switched to the released version it started failing although there are almost no code changes on master save removing a type alias

Needless to say, all tests are running, the samples work flawlessly…, the typespecs in all sister projects pass. The problem seems to be typespec only.

The error message (build) is the following:

$ if ! ([[ "$TRAVIS_ELIXIR_VERSION" == "1.4"* ]] && [[ "$TRAVIS_OTP_RELEASE" == "18"* ]]); then mix dialyzer; fi
Checking PLT...
[:benchee, :benchee_json, :compiler, :deep_merge, :elixir, :kernel, :logger,
 :poison, :stdlib]
PLT is up to date!
Starting Dialyzer
dialyzer args: [check_plt: false,
 init_plt: '/home/travis/build/PragTob/benchee_html/_build/dev/dialyxir_erlang-19.2_elixir-1.4.2_deps-dev.plt',
 files_rec: ['/home/travis/build/PragTob/benchee_html/_build/dev/lib/benchee_html/ebin'],
 warnings: [:unknown]]
done in 0m3.35s
lib/benchee/formatters/html.ex:129: The created fun has no local return
lib/benchee/formatters/html.ex:135: Function reports_for_input/5 has no local return
lib/benchee/formatters/html.ex:154: Function job_reports/4 has no local return
lib/benchee/formatters/html.ex:155: The call 'Elixir.Benchee.Formatters.HTML':merge_job_measurements(input_stats@1::#{'__struct__':='Elixir.Benchee.Statistics', 'average':=float(), 'ips':=float(), 'maximum':=number(), 'median':=number(), 'minimum':=number(), 'sample_size':=integer(), 'std_dev':=float(), 'std_dev_ips':=float(), 'std_dev_ratio':=float()},input_run_times@1::[number()]) will never return since it differs in the 2nd argument from the success typing arguments: (map(),map())
done (warnings were emitted)

The problem seems to be that dialyzer thinks an array of numbers ends up as the second argument, while it really is a map with keys that point to arrays of numbers.

Here is the call that causes the error:

  defp job_reports(input, input_stats, input_run_times, system) do
    merged_stats = merge_job_measurements(input_stats, input_run_times)
    # other stuff
  end

  # called from...

  defp reports_for_input(input, input_stats, input_run_times, system, filename) do
    # other stuff
    job_reports = job_reports(input, input_stats, input_run_times, system)
    # other stuff
  end

  # called from (this call removes one nesting level but there is still another one)...

  defp input_job_reports(statistics, run_times, system, filename) do
    Enum.map statistics, fn({input, input_stats}) ->
      input_run_times = Map.fetch! run_times, input
      reports_for_input(input, input_stats, input_run_times, system, filename)
    end
  end

  # and finally from...

  @spec format(Benchee.Suite.t) :: %{Benchee.Suite.key => String.t}
  def format(%{statistics: statistics, run_times: run_times, system: system,
               configuration: %{
                 formatter_options: %{html: %{file: filename}}}}) do
    statistics
    |> input_job_reports(run_times, system, filename)
    |> #more pipe
  end  

And Benchee.Suite is defined as:

  @type optional_map :: map | nil
  @type key :: atom | String.t
  @type benchmark_function :: (() -> any) | ((any) -> any)
  @type t :: %__MODULE__{
    configuration: Benchee.Configuration.t | nil,
    system: optional_map,
    run_times: %{key => %{key => [integer]}} | nil,
    statistics: %{key => %{key => Benchee.Statistics.t}} | nil,
    jobs: %{key => benchmark_function}
  }

→ Both run_tmes and statistics are defined as being nested with 2 keys before the actual [integer] (or well number) comes. The code above just removes the first of those levels (which is the input name, the second is the job name), still dialyzer says it has a type mismatch between map and [number].

I’ve tried:

• adding more type specs in the indermediate function calls and the function itself
• replacing Map.fetch! with a pattern match
• tried reinstating the type alias that was removed

All to no avail And after a couple of hours now I gave up, still published the package and am looking for your help The project is benchee_html

Thank you kind stranger for reading this far - I genuinely hope it’s a dumb oversight on my part.