Understanding improper lists usage in elixir-plug

:wave:t3:

I want to understand the reason of constructing an improper list usage in the elixir-plug repository, Line 256

First Question: Is the usage of an improper list necessary here?:

# encoder fallback
  defp encode_pair(field, value, encoder) do
    [field, ?= | encode_value(value, encoder)]
  end

I think improper lists do not have many uses; also there are many functions that will error when given an improper list.

for example Dialyzer would raise a warning for constructing an improper list, and they got this -dialyzer({no_improper_lists, g/0}). to suppress the warning.

Second Question: why this construct actually exists in Erlang?

3 Likes

I don’t think this does create an improper list. It’s a recursive function, and that function has termination clauses that return [], which means it’ll construct a proper list.

As to this:

I think it’s better to approach this by understanding that the cons operator | is, at least in erlang, more foundational than even lists. Its most common use is with lists, but it has uses for low memory btrees, and gets use in IOlists as well.

2 Likes

Oh! i guess i was wrong about this! if it is not going to create an improper list why would dialyze raise these two warning to me for that function on line 255?

List construction (cons) will produce an improper list, because its second argument is nonempty_improper_list(61, binary()).
Warning: List construction (cons) will produce an improper list, because its second argument is binary().

where is the improper list that its getting warned about?

You might not be wrong about this function, it just isn’t super obvious to me that Dialyzer is right either. Either way, check out this line: plug/lib/plug/conn/query.ex at 274e44f9a149b922099bf60029d8267afe494968 · elixir-plug/plug · GitHub

The encode_pair function is a defp and the output of that function is passed immediately to IO.iodata_to_binary. iodata is one of the real world uses for improper lists, because it reduces the overhead of building out a list front to back. The iodata_to_binary is able to handle improper lists and turn them into a single binary.

Dialyzer warns about improper lists because most of the time you do indeed want to warn for them. If the Plug core team used dialyzer they would probably put a no-warn invocation in there to tell dialyzer that this is intentional.

4 Likes

smart, it makes sense now! i will make sure to read more about iodata

thank you for your time.

Sorry for resurrecting an old thread, but while looking into this I also stumbled upon this issue which is relevant I think:

The OTP team seems to be considering disabling these warnings, since there are legit use cases for improper lists:

  • IO data as already explained above
  • as a low-level performance optimization for libraries: [a | b] takes less memory than {a, b}
2 Likes