Matthew

Matthew

Newbie with UnicodeConversionError

Hello, I am new to Elixir and so far finding it awesome. However I have just stumbled across a problem with non ascii strings. Please see below to see the error message I am getting:

iex(4)> "하이"
** (UnicodeConversionError) invalid encoding starting at <<199, 207, 192, 204, 34, 10>>
    (elixir) lib/string.ex:1801: String.to_charlist/1
iex(4)>

I note that I am using Windows 10 and running iex from a Cmder shell (the same error occurs in windows cmd and in powershell).

Any suggestions for how I can fix this? (Besides using developing on a linux system :wink: )

Most Liked

kip

kip

ex_cldr Core Team

Actually you can: String.codepoints/1 will decode what it can and show you the code points it can’t.

iex> x = <<160, 73, 100, 111, 119, 117, 32, 84, 97, 121, 108, 111, 114, 32, 83, 60, 47,
  99, 111, 110, 116, 97, 99, 116, 45, 97, 100, 100, 114, 101, 115, 115, 62, 10>>
iex> String.codepoints x
[
  <<160>>,
  "I",
  "d",
  "o",
  "w",
  "u",
  " ",
  "T",
  "a",
  "y",
  "l",
  "o",
  "r",
  " ",
  "S",
  "<",
  "/",
  "c", 
  "o",
  "n",
  "t",
  "a",
  "c",
  "t",
  "-",
  "a",
  "d",
  "d",
  "r",
  "e",
  "s",
  "s",
  ">",
  "\n"
]
michalmuskala

michalmuskala

While I have very little windows experience, I would expect this error to be caused by shell being not in unicode encoding.

In general any unicode issues on windows related to the shell were solved by using the graphical iex --werl shell.

Marcus

Marcus

You can try to use the raw representation of the string.

iex(30)> i "하이"
...
Raw representation
  <<237, 149, 152, 236, 157, 180>>
...
iex(31)> i <<237, 149, 152, 236, 157, 180>>
Term
  "하이"
...
Raw representation
  <<237, 149, 152, 236, 157, 180>>
...
iex(32)> str = <<237, 149, 152, 236, 157, 180>>
"하이"
iex(33)>

As you can see your string has just the wrong encoding.

iex(43)> String.to_charlist(<<199, 207, 192, 204, 34, 10>>)
** (UnicodeConversionError) invalid encoding starting at <<199, 207, 192, 204, 34, 10>>
    (elixir) lib/string.ex:1801: String.to_charlist/1
iex(43)>

Where Next?

Popular in Questions Top

chrisalley
ExUnit now has describe blocks which is a welcome addition coming from RSpec. In the docs, it states that nested hierarchies of describe ...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New
shahryarjb
Hello, I get Persian date from my client and convert it to normal calendar like this: def jalali_string_to_miladi_english_number(persi...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
JulienCorb
I am trying to implement my new.html.eex file to create new posts on my website. new.html.eex: &lt;h1&gt;Create Post&lt;/h1&gt; &lt;%= ...
New
Lily
In templates/appointment/index.html.eex: &lt;%= for appointment &lt;- @appointments do %&gt; &lt;tr&gt; &lt;td&gt;&lt;%= appoi...
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
srinivasu
How to handle excepions in elixir? Suppose i have A, B, C ,D, E modules. and each module has get() function. A.get() method will call t...
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New

Other popular topics Top

malloryerik
Hi, this is for people who, like me, have had some friction using .html.heex templates in VSCode. The solution seems to be, in a hyphena...
New
9mm
I am constructing a JSON object (map) and I need to conditionally set a field. I’m trying to write proper elixir-way code… and I’m at a l...
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
AstonJ
Please see the new poll here: Which code editor or IDE do you use? (Poll) (2022 Edition) It’s been a while since we first asked this, I...
208 31142 143
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
dogweather
I wrote this comment on r/haskell, and it’s not popular there. :wink: But I think I’m on to something… Haskell reminds me of Java, and e...
New
sergio
Kind of like when jquery came out, it was super necessary. Existing drag and drop libraries have a bunch of baggage to support old browse...
New
vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New

We're in Beta

About us Mission Statement