How may we generalize code across OS platforms, that use different `end-of-line`

I do my elixir dev on windows hence I always assume \r\n

For example:

item_codes =!("path/to/item-codes.txt") |> String.split("\r\n")

This might not work on other OS that default to \n

Does (or can) Elixir offer any normalization to eliminate the need for a devs to worry about end of line formats on different OS?

You could use!/3 which by default reads the file one line at a time.
This might also be better than! and String.split because it doesn’t require you to load the full file into memory right away.

Check out the docs for!/3 here:!/3

File stream does assume LF only.

Normalisation of line endings has already happened a long time ago. But windows folks do it their own way…

Most win devs I know do use LF only, because it’s how many tools want to see it. Go does even consider CRLF a parse error…

That not quite right.!/3 relies on Erlang’s :io.read_line and does in fact normalize line endings. I’ve just tried it with a sample file:!("test.file")
# => "Line 1\r\nLine 2\r\nLine 3"!("test.file") |> Enum.take(3)
# => ["Line 1\n", "Line 2\n", "Line 3"]

For comparison, the same code but with just \n line endings yields the same result with!:!("test.file")                  
# => "Line 1\nLine 2\nLine 3"
iex(11)>!("test.file") |> Enum.take(3)
# => ["Line 1\n", "Line 2\n", "Line 3"]
1 Like

Cool, maybe it was an older erts where I had the trouble becAuse of crlf…

Well I have experienced problems when using code that was written without consideration for \r\n on windows.

It used macros to expand a text file that was split on \n - took me a while too see why it would not work when compiled on my laptop.

Shouldn’t pure Elixir code just run on any OS without consideration for end-of-line differences?

One would think it should.

It sounds like the macro author was not considering eol differences, rather than Elixir itself?

One solution is to String.split on ~r/\R/.

That handles both \n and \r\n