Dialyzer gets nested map wrong and errors out ([number] vs. map)

Questions / Help
,
1

Hi there fellow Elixir friends,

in the latest Benchee released I introduced structs and type specs, all worked fine until I wanted to release the last plugin. It worked fine when running against master of the core library through a github dependency build but when I switched to the released version it started failing although there are almost no code changes on master save removing a type alias

Needless to say, all tests are running, the samples work flawlessly…, the typespecs in all sister projects pass. The problem seems to be typespec only.

The error message (build) is the following:

$ if ! ([[ "$TRAVIS_ELIXIR_VERSION" == "1.4"* ]] && [[ "$TRAVIS_OTP_RELEASE" == "18"* ]]); then mix dialyzer; fi
Checking PLT...
[:benchee, :benchee_json, :compiler, :deep_merge, :elixir, :kernel, :logger,
 :poison, :stdlib]
PLT is up to date!
Starting Dialyzer
dialyzer args: [check_plt: false,
 init_plt: '/home/travis/build/PragTob/benchee_html/_build/dev/dialyxir_erlang-19.2_elixir-1.4.2_deps-dev.plt',
 files_rec: ['/home/travis/build/PragTob/benchee_html/_build/dev/lib/benchee_html/ebin'],
 warnings: [:unknown]]
done in 0m3.35s
lib/benchee/formatters/html.ex:129: The created fun has no local return
lib/benchee/formatters/html.ex:135: Function reports_for_input/5 has no local return
lib/benchee/formatters/html.ex:154: Function job_reports/4 has no local return
lib/benchee/formatters/html.ex:155: The call 'Elixir.Benchee.Formatters.HTML':merge_job_measurements(input_stats@1::#{'__struct__':='Elixir.Benchee.Statistics', 'average':=float(), 'ips':=float(), 'maximum':=number(), 'median':=number(), 'minimum':=number(), 'sample_size':=integer(), 'std_dev':=float(), 'std_dev_ips':=float(), 'std_dev_ratio':=float()},input_run_times@1::[number()]) will never return since it differs in the 2nd argument from the success typing arguments: (map(),map())
done (warnings were emitted)

The problem seems to be that dialyzer thinks an array of numbers ends up as the second argument, while it really is a map with keys that point to arrays of numbers.

Here is the call that causes the error:

  defp job_reports(input, input_stats, input_run_times, system) do
    merged_stats = merge_job_measurements(input_stats, input_run_times)
    # other stuff
  end

  # called from...

  defp reports_for_input(input, input_stats, input_run_times, system, filename) do
    # other stuff
    job_reports = job_reports(input, input_stats, input_run_times, system)
    # other stuff
  end

  # called from (this call removes one nesting level but there is still another one)...

  defp input_job_reports(statistics, run_times, system, filename) do
    Enum.map statistics, fn({input, input_stats}) ->
      input_run_times = Map.fetch! run_times, input
      reports_for_input(input, input_stats, input_run_times, system, filename)
    end
  end

  # and finally from...

  @spec format(Benchee.Suite.t) :: %{Benchee.Suite.key => String.t}
  def format(%{statistics: statistics, run_times: run_times, system: system,
               configuration: %{
                 formatter_options: %{html: %{file: filename}}}}) do
    statistics
    |> input_job_reports(run_times, system, filename)
    |> #more pipe
  end

And Benchee.Suite is defined as:

  @type optional_map :: map | nil
  @type key :: atom | String.t
  @type benchmark_function :: (() -> any) | ((any) -> any)
  @type t :: %__MODULE__{
    configuration: Benchee.Configuration.t | nil,
    system: optional_map,
    run_times: %{key => %{key => [integer]}} | nil,
    statistics: %{key => %{key => Benchee.Statistics.t}} | nil,
    jobs: %{key => benchmark_function}
  }

–> Both run_tmes and statistics are defined as being nested with 2 keys before the actual [integer] (or well number) comes. The code above just removes the first of those levels (which is the input name, the second is the job name), still dialyzer says it has a type mismatch between map and [number].

I’ve tried:

  • adding more type specs in the indermediate function calls and the function itself
  • replacing Map.fetch! with a pattern match
  • tried reinstating the type alias that was removed

All to no avail :’( And after a couple of hours now I gave up, still published the package and am looking for your help :smiley: The project is benchee_html

Thank you kind stranger for reading this far :green_heart: - I genuinely hope it’s a dumb oversight on my part.

2

The question here is how dialyzer derived the type [number]? Poking around your code a bit, I see that you’re passing input_run_times to JSON.format_measurements/2. That function has the following signature:

@spec format_measurements(Benchee.Statistics.t, [number]) :: String.t
def format_measurements(statistics, run_times) do

Therefore, I’d guess that dialyzer uses this spec to conclude that run_times is a list of numbers, and subsequently treats it that way. Commenting out the invocation to JSON.format_measurements results in no errors, which should confirm my suspicion. I can’t tell whether the spec in JSON.format_measurements is wrong, or there’s some deeper problem, but I’ll leave this investigation to you :slight_smile:

3 Likes
3

Thank you a thousand times Sasa that you took the time and helped me :green_heart: :heart: :purple_heart:

It has been fixed (the typespec at format_measurements was plain wrong…) and benchee_json 0.3.1 and benchee_html 0.3.1 have been released.

The worst part is that I half remembered that I put a [number] somewhere where I wasn’t too sure but I only searched in benchee/benchee_html :see_no_evil: Worst part is, the doctests of format_measurements clearly show that the type signature was wrong. :see_no_evil:
Wouldn’t have thought that dialyzer picks up type information from calling other functions and then assumes that must be the type. That makes a lot of sense though and is pretty clever.

Thanks again, you saved my day! :tada: :thumbsup:

2 Likes