I have a process where I’m creating and accumulating a string into a StringIO file, the creation and aggregation, it is simple, I open the string IO and send the PID with process state.

# process init    
{:ok, pid} = StringIO.open("stream_buffer")

# call proccess to aggregation 
IO.write(pid, text <> " ")

The problem is, when I’m finishing the process, I want to send it directly to a stream. I use to save it to a file, but when the buffer is too big (1 or 2 GBs), the process to send it to a file and get it back takes more time than I wish to wait.

My first try to send it directly into a stream was something like this:

    pid
    |> IO.binstream(:line)

The problem with this try (and I don’t understand if it is intentional or not) is that this instead of getting the content written on this IO returns the content I’ve used on open. So instead of transform, my content aggregated trough IO.write it always transforms the content I’ve used as the “stream name”, example:

{:ok, pid} = StringIO.open("buffer")
IO.write(pid, "And here we should have the buffer content")

# will evaluate to {"buffer", "And here we should have the buffer content"}
pid
|> StringIO.describe()
|> IO.inspect()

# will evaluate "buffer", but I expect it evaluate the whole buffer content, not only the initial value
pid
|> IO.binstream(:line)
|> Stream.map(&IO.inspect/1)
|> Stream.run()

What I’m doing now, it’s creating a new StringIO with the old one content, what I’m feeling it’s really wrong:

{:ok, other_pid} = pid
|> StringIO.flush()
|> StringIO.open() 

So I yank the content from the PID where I was aggregating to a new device, the problem is:
1 - It really feels wrong.
2 - When the String is huge this yank operation from the old to the new device still takin a lot of time (lass than create and read a file from disk, but still slow)

What I’m doing wrong here? What are the other options to process it on the fly using streams, are other possible options besides StringIO?