While working on the ICFP Contest this weekend, I struggled to find a good way to periodically read content not delimited by newlines. Let me show some examples of what I mean.
Here’s a Ruby script that produces messages:
$stdout.sync = true
loop do
message = rand(1_000_000)
$stdout.write "<#{message}>"
sleep rand(3) + 1
end
I can think of multiple ways, using Ruby, to read these messages as they come in. For example, I can use non-blocking I/O:
loop do
begin
raw = $stdin.readpartial(1_024)
puts raw[/\d+/]
rescue EOFError
sleep 0.1
end
end
Or I can read what’s available:
require "io/wait"
loop do
$stdin.wait_readable
raw = $stdin.read($stdin.nread)
puts raw[/\d+/]
end
There are other options too, like using IO::select(). Here’s how the above examples work in practice:
$ ruby producer.rb | ruby read_nonblocking.rb
319187
122221
30420
…
$ ruby producer.rb | ruby read_ready.rb
640243
971582
366808
…
I haven’t found a good way to do similar work with Elixir. The best I’ve come up with for the same input is to read character by character:
defmodule MessageReader do
def read_message(device, buffer \\ "") do
new_buffer = buffer <> IO.read(device, 1)
if String.first(new_buffer) == "<" and String.last(new_buffer) == ">" do
String.slice(new_buffer, 1..-2)
else
read_message(device, new_buffer)
end
end
def read_messages(device, handler) do
read_message(device)
|> handler.()
read_messages(device, handler)
end
end
MessageReader.read_messages(:stdio, &IO.puts/1)
This does work:
$ ruby producer.rb | elixir read_chars.exs
963609
378034
387827
…
However, that would be pretty inefficient with long messages and I can’t find a way to read ahead. Am I missing a useful trick?
Thanks in advance!
