pavel-sev

pavel-sev

IO problems - (IO.StreamError) error during streaming: unknown POSIX error

Hello

I am trying to deal with stdio and face strange (for me) behavior when reading bytes. To reproduce it I did simple one line bash script:

elixir -e ':binary.list_to_bin( for n <- 0..255, into: [], do: n) |> IO.binwrite()' | elixir -e 'IO.binstream(:stdio, 1) |> Enum.each(&(&1))'

and the error:

** (IO.StreamError) error during streaming: unknown POSIX error
    (elixir 1.14.5) lib/io.ex:752: IO.each_binstream/2
    (elixir 1.14.5) lib/stream.ex:1625: Stream.do_resource/5
    (elixir 1.14.5) lib/enum.ex:4307: Enum.each/2
    nofile:1: (file)
    (stdlib 4.3) erl_eval.erl:748: :erl_eval.do_apply/7

Actually it works without Enum.each but I need this data )
Did I missed some thing or maybe there is better way to read from stdio?

Elixir 1.14.5-otp-25
Erlang 25.3
OS Ubuntu 22.4

#io

Marked As Solved

al2o3cr

al2o3cr

There are two problems with this code:

  • the code that writes doesn’t actually produce the bytes you expect

  • the code that reads is trying to read binary data from a device set to UTF8 encoding


The first issue, as the documentation for IO.binwrite mentions:

Important: do not use this function on IO devices in Unicode mode as it will write the wrong data. In particular, the standard IO device is set to Unicode by default, so writing to stdio with this function will likely result in the wrong data being sent down the wire.

The actual data that the first half of that pipeline prints out is 384 bytes long, because each byte of the second half of the data is translated to the corresponding Unicode character (so 128<<194, 128>>

Calling :io.setopts(:standard_io, [encoding: :latin1]) before binwrite avoids this behavior and writes exactly the 256 bytes expected.


The second issue is that IO.binread also behaves unexpectedly with a Unicode device. Reading a single byte from it will fail if bytes farther along in the stream are invalid Unicode. so:

echo $'\x01\x02' | elixir -e ':file.read(:standard_io, 1) |> IO.inspect; :file.read(:standard_io, 1) |> IO.inspect'

# prints
{:ok, <<1>>}
{:ok, <<2>>}


# but this doesn't work even though the first two bytes are both OK
echo $'\x01\x02\xFF' | elixir -e ':file.read(:standard_io, 1) |> IO.inspect; :file.read(:standard_io, 1) |> IO.inspect'

# prints
{:error, :collect_chars}
:eof

This {:error, :collect_chars} is swallowed by higher levels of the IO machinery and produces the “unknown POSIX error” explanation you’re seeing.

As in the first case, setting the encoding correctly produces the correct result:

echo $'\x01\x02\xFF' | elixir -e ':io.setopts(:standard_io, [encoding: :latin1]); :file.read(:standard_io, 1) |> IO.inspect; :file.read(:standard_io, 1) |> IO.inspect'

# prints
{:ok, <<1>>}
{:ok, <<2>>}

(note: this is the same collect_chars error shape as in this ticket that @josevalim just filed, but I don’t believe it’s directly related)


Putting both together does what you want:

elixir -e ':io.setopts(:standard_io, [encoding: :latin1]); :binary.list_to_bin( for n <- 0..255, into: [], do: n) |> IO.binwrite()' | elixir -e ':io.setopts(:standard_io, [encoding: :latin1]); IO.binstream(:stdio, 1) |> Enum.each(&IO.inspect/1)'

# prints
<<0>>
...
<<255>>

Also Liked

al2o3cr

al2o3cr

What specifically do you mean by “it”?

What specifically do you mean by “did not work”?

You can see the code you’re trying and the error message you’re getting, but the rest of us can’t.

Also, consider opening a new thread if you have a new problem.

Where Next?

Popular in Questions Top

gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New
fireproofsocks
I’m working on defining a simple Ecto schema for a table (in PostGres), but I don’t see where I can define a column as NOT NULL. Conside...
New
shahryarjb
Hello, I get Persian date from my client and convert it to normal calendar like this: def jalali_string_to_miladi_english_number(persi...
New
ycv005
I have followed this StackOverflow post to install the specific version of Erlang. And When I am running mix ecto.setup then getting fol...
New
vegabook
I’m brand new to Phoenix and I have stripped one of the demo applications to the bone. I just want to get an svg up on the screen. Here i...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New

Other popular topics Top

Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID&lt;0.412.0&gt; terminating ** (Postgrex.Error) FATAL...
New
JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
chrismccord
Phoenix 1.4.0 released Phoenix 1.4 is out! This release ships with exciting new features, most notably with HTTP2 support, improved deve...
688 30877 112
New
vegabook
I’m brand new to Phoenix and I have stripped one of the demo applications to the bone. I just want to get an svg up on the screen. Here i...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
KronicDeth
Elixir plugin for JetBrain’s IntelliJ Platform (including Rubymine) This is a plugin that adds support for Elixir to JetBrains IntelliJ...
289 36128 110
New
klo
Got a question about when to concat vs. prepending items to list then reversing to achieve appending. So i know lists boil down to [1 | ...
New
AstonJ
Seen any cool LiveView demos, sample apps or examples? Please post them here! :003:
New

We're in Beta

About us Mission Statement