Streaming xml file over ftp fails if I don't convert it into a list first

I’m trying to stream an xml file over ftp with Stream.resource/3 and SweetXml.stream_tags!/3, but I’m running into a strange issue where sweet_xml raises an error, before chunk arrives, unless I call Enum.to_list/1.

# Function definitions

def stream_file!(client) do
  Stream.resource(fn -> start_file_stream(client) end, &next_file_chunk/1, &stop_file_stream/1)
end

defp start_file_stream(client) do
  Logger.debug("Requesting to stream file")

  with :ok <- :ftp.recv_chunk_start(client.conn, to_charlist(client.credentials.file_path)) do
    client
  end
end

defp next_file_chunk(%{conn: conn} = client) do
  case :ftp.recv_chunk(conn) do
    {:ok, binary} ->
      Logger.debug("Stream file chunk #{byte_size(binary)} bytes)")
      {[binary], client}

    :ok ->
      Logger.debug("Transfer completed successfully")
      {:halt, client}

    error ->
      Logger.error(
        credentials_id: client.credentials.id,
        message: "Unable to stream file",
        reason: error
      )

      {:halt, error}
  end
end

defp next_file_chunk(error), do: {:halt, error}
defp stop_file_stream(_), do: []

# Code that is executed

# conn is the pid from :ftp.open/2
# Credentials holds infos about the file and the credentials for the ftp server
client = %{conn: conn, credentials: credentials}

client
|> stream_file!()
# For debugging
|> Stream.each(&IO.inspect(&1, label: :chunk))
# |> Enum.to_list()
|> SweetXml.stream_tags!(:Mitglied, discard: [:Mitglied])
|> Stream.map(fn {:Mitglied, elem} -> 
  # parse data here
end)
|> Stream.run()

Calling this code produces the following output. If I uncomment |> Enum.to_list() it will work fine and print all the chunks as they come. Any idea what might cause this?

[debug] Requesting to stream file
[error] 3917- fatal: :expected_element_start_tag

** (SweetXml.XmerlFatal) :expected_element_start_tag
    (sweet_xml 0.7.3) lib/sweet_xml.ex:539: anonymous fn/1 in SweetXml.stream!/2
    (elixir 1.14.4) lib/stream.ex:1619: Stream.do_resource/5
    (elixir 1.14.4) lib/stream.ex:1813: Enumerable.Stream.do_each/4
    (elixir 1.14.4) lib/stream.ex:689: Stream.run/1

For reference, I have a separate implementation for streaming files over sftp with sftp_client, which works perfectly. I’ve looked into their implementation, but couldn’t find any meaningful difference.
https://github.com/tlux/sftp_client/blob/v1.4.7/lib/sftp_client/stream.ex#L26-L51