List append (++) does not return a list

[polovy]

I am going through some tutorials, and one of the tasks is to create a list of items by taking first two elements of the existing item and appending them to the end of said list. The problem I have is that for some numerical values in the list, I get a strange response returned.

The Cos:

defmodule Tutorial do 
  def mirror_row(row) do
    [first, second | _tail] = row
    row ++ [second, first]
  end
end

the results I get are:

iex> Maps.mirror_row([1, 2, 3])
[1, 2, 3, 2, 1]
iex> Maps.mirror_row([121, 12, 123])
~c"y\f{\fy"

The 1st result is as expected, but the 2nd I would expect [121, 12, 123, 12, 121].
What is actually happening here and why?
I chacked this for different values and it does not make sense to me.

Some other results:

iex> Maps.mirror_row([123, 122, 121])
~c"{zyz{"
iex> Maps.mirror_row([123, 22, 12])
[123, 22, 12, 22, 123]
iex> Maps.mirror_row([121, 122, 123])
~c"yz{zy"
iex> Maps.mirror_row([121, 12, 123])
~c"y\f{\fy"
iex> Maps.mirror_row([121, 12, 13])
~c"y\f\r\fy"
iex> Maps.mirror_row([121, 22, 13])
[121, 22, 13, 22, 121]

Thanks.

[LostKobrakai]

What you’re seeing are charlists using the ~c sigil. They are the exact data you expected, but printed as text to allow interoperability with erlang, which uses charlists as their default string type: List — Elixir v1.17.3

[polovy]

I suspected that this may be something like that. Is there a way of ‘forcing’ a standard list view in the console?
I assume that this will make no difference for the program (ie. I can pass the list to another function and it will work).

[LostKobrakai]

The link I posted shows how to configure the inspect protocol that way. IEx.configure can be used to configure iex with custom inspect options. Indeed the application itself doesn’t care. This is just something relevant to printing the data for human consumption.

[polovy]

Thanks.

[dimitarvp]

IEx.configure(inspect: [charlists: :as_lists])

Run this manually inside iex or put it in a file called .iex.exs at the root of your project and it will be ran automatically on iex startup in that directory.

[sbuttgereit]

As an aside, I completely forgot this was out here…

Elixir Gotchas and Common Issues Wiki

Preamble This Wiki is intended to be a community-maintained (see the Contribution Guidelines if interested) resource of common “gotchas” (unexpected issues) that beginners to Elixir might experience. The main intended audience is programmers used to other languages, who come to Elixir and find themselves confused by some particularities of the language and its ecosystem. We also hope it can be used by experienced Elixir programmers as a tool for answering questions and guiding beginners - a qui…

Which, naturally, includes this very issue. I’m guessing this post is not very visible and only appears in searches… which in a case like this is more likely to not appear in a search since you’d kinda got to know what the issue was in the first place to hit the right search terms for it.

It’s a good idea… but I wonder if there’s a way to call this out for special attention so that it isn’t forgettable for those that might add to it or discoverable for those that might need it.

[arcanemachine]

Cool, TIL. But this just flips the confusion the other way around, no?

Erlang/OTP 25 [erts-13.2.2.7] [source] [64-bit] [smp:20:20] [ds:20:20:10] [async-threads:1] [jit:ns]

Interactive Elixir (1.15.7) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> [97, 98, 99]                                  
~c"abc"
iex(2)> ~c"abc"
~c"abc"
iex(3)> IEx.configure(inspect: [charlists: :as_lists])
:ok
iex(4)> [97, 98, 99]                                  
[97, 98, 99]
iex(5)> ~c"abc"                                       
[97, 98, 99]

I’m guessing the tradeoff is that one is less likely to find themself surprised when a charlist is represented as a list of integers, compared to the more-common case where a list of integers is rendered as a charlist, which is a very common surprise for beginners. I suppose that if you’re deliberately working with charlists, you’re probably deep enough into the Elixir rabbit-hole to understand what is happening behind the scenes.

---

To confirm my obvious suspicion, the behaviour can be reverted to the default with this line:

IEx.configure(inspect: [charlists: :as_charlists])

Thanks again for another excellent tip. I’m gonna dogfood this one for a bit…

[dimitarvp]

@arcanemachine

I’m guessing the tradeoff is that one is less likely to find themself surprised when a charlist is represented as a list of integers, compared to the more-common case where a list of integers is rendered as a charlist, which is a very common surprise for beginners.

Yes, I am guessing the same. I don’t like the status quo either but unless we’re willing to go contribute to OTP itself (or maybe only to Elixir? not sure actually) then it’s going to stay that way and we should make the crutches obvious. At least they restore things to a bit saner state (hopefully).

[arcanemachine]

Hey, I’m in no position to complain while I’m standing on the shoulders of giants. But I have not encountered this tip before, and I think it is definitely worth knowing.

It’s funny too, because it’s one of those things that’s obvious in retrospect. I’ve used a line like that to temporarily change the charlist rendering style. I just never though to put it in my IEx config.

[dimitarvp]

I agree. It’s not obvious. I’d even argue this config should be the default; I really don’t see who would derive an actual tangible value out of seeing some integer lists as charlists.

[JEG2]

@dimitarvp

I really don’t see who would derive an actual tangible value out of seeing some integer lists as charlists.

The folks calling Erlang functions that return “strings” that wouldn’t be readable.

I provide a detailed breakdown of the reasoning behind this feature and how to alter the behavior in Livebookisms. Here’s an exert:

[polovy]

I think what is really confusing to a newbie is that the function will return either the listo OR charlist, depending on the values supplied. This has confused me more than the return of the charlist.

[dimitarvp]

That’s just list still.

[sbuttgereit]

To expand on what @dimitarvp said. A charlist is just a list:

iex(2)> [64,65,66] === ~c"@AB"
true

The only thing that differentiates any list from a charlist is this definition (Binaries, strings, and charlists — Elixir v1.17.3):

A charlist is a list of integers where all the integers are valid code points.

Once you have a defined list of integers which matches that definition you have both a list of integers and a valid charlist. At that point they’re indistinguishable… which is why they can be confusing and why any display in a REPL will catch someone up… the REPL can only (reasonably) display one representation. If chosen representation doesn’t match your intent/expectation you might very well think it’s returning the wrong thing when it’s just an unexpected representation: not a “different thing” at all.

[LostKobrakai]

@JEG2

The folks calling Erlang functions that return “strings” that wouldn’t be readable.

It doesn’t even need to be an erlang function directly. E.g. Application.started_applications/0 or Application.loaded_applications/1 return [{atom, charlist, charlist}]. Also many error tuples include charlists, especially around e.g. gen_tcp/ssl.

[dimitarvp]

None of which is so important for actual production development – we’re talking inspecting stuff in iex. Whoever truly needs it should opting in for lists to be shown as charlists. IMO Elixir got the setting the wrong way around: lists should have been lists regardless if they are printable or not.

Some of us here, yourself included, are around for a long time and it’s been apparent that this is something newcomers regularly get confused of.

[LostKobrakai]

@dimitarvp

None of which is so important for actual production development – we’re talking inspecting stuff in iex.

I’d not just iex. The inspect protocol is used in all manner ot places to turn arbitrary data into a string representation. That includes logs, tracing, error handling system and yes you iex shell. I would consider those very much important for production usage. And no I don’t want to need to transform a list of integersin my error tracking to a string to be able to figure out what the error is about.

I also don’t consider changing a subset of those places to different defaults. That would be even more confusing.

[dimitarvp]

@LostKobrakai

And no I don’t want to need to transform a list of integersin my error tracking to a string to be able to figure out what the error is about.

I have solved this for myself with helpers that convert charlists to strings at the edges. From then on it’s business as usual. Charlists are an Erlang relic and the rest of the arguments in their favor I view as post-hoc rationalization.

Regardless of my opinion however, people like myself have made their own inspect wrappers and put them in the right places and we have no surprising visual outputs. I still wonder however: since it’s been obvious for a long time that this is tripping up newcomers, why was it not deemed important enough to reverse the default? Are people’s businesses going to crash and burn if 1% of their inspected values in logs suddenly don’t show strings but lists of integers?

[LostKobrakai]

@dimitarvp

Are people’s businesses going to crash and burn if 1% of their inspected values in logs suddenly don’t show strings but lists of integers?

In my experience lists with integers in valid ascii range are much rarer that seeing charlists around code interacting with erlang. Also I consider the information lost for a string not being represented as a string larger than the information lost for a very specific subsets of lists, which would be represented as a charlist.

In the end I also don’t think changing the default would mean new people are no longer tripped up by charlists. It’ll just happen in different circumstances.