Error when zipping files whose names contain unicode codepoints above 255

I think I may have found an error in :zip related to zipping files whose names contain unicode codepoints above 255, but I would appreciate it if somebody was interested in double checking and helping get it fixed.

Recreating the problem is simple enough.

Erlang/OTP 21 [erts-10.0] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]

Interactive Elixir (1.8.1) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> filename = [256]
[256]

iex(2)> cwd = '/tmp/'
'/tmp/'

iex(3)> File.write!(cwd ++ filename, "")
:ok

iex(4)> :zip.zip('tmp.zip', [filename], [:memory, cwd: cwd])
{:error,
 {:EXIT,
  {:badarg,
   [
     {:erlang, :iolist_size, [[256]], []}, 
     {:zip, :binary_io, 2, [file: 'zip.erl', line: 1754]}, 
     {:zip, :put_z_files, 6, [file: 'zip.erl', line: 859]},
     {:zip, :do_zip, 3, [file: 'zip.erl', line: 454]},
     {:zip, :zip, 3, [file: 'zip.erl', line: 443]},
     {:erl_eval, :do_apply, 6, [file: 'erl_eval.erl', line: 677]},
     {:elixir, :eval_forms, 4, [file: 'src/elixir.erl', line: 258]},
     {IEx.Evaluator, :handle_eval, 5, [file: 'lib/iex/evaluator.ex', line: 257]}
   ]}}}

iex(5)> :erlang.iolist_size(filename)
** (ArgumentError) argument error
    :erlang.iolist_size([256])

iex(5)> :erlang.iolist_size([255])   
1

The stacktrace from :zip.zip points first to here

binary_io({write, Data}, {Pos, B}) ->
    {Pos + erlang:iolist_size(Data), pwrite_binary(B, Pos, Data)};

Where iolist_size is used to advance Pos when writing, from here

    Out2 = Output({write, FileName}, Out1),

Where the filename is written.

I think this means filenames cannot contain codepoints over 255, but the Erlang documentation for file:name says

If VM is in Unicode filename mode, string() and char() are allowed to be > 255.

Which leads me to believe that the use of iolist_size is a problem.

1 Like

I suggest you report this to the issue tracker for Erlang (http://bugs.erlang.org) and it will be checked (and corrected) if necessary) by the OTP team.
/Kenneth

2 Likes

Reported: https://bugs.erlang.org/browse/ERL-1003

2 Likes