Matthew
Hello, I am new to Elixir and so far finding it awesome. However I have just stumbled across a problem with non ascii strings. Please see below to see the error message I am getting:
iex(4)> "하이"
** (UnicodeConversionError) invalid encoding starting at <<199, 207, 192, 204, 34, 10>>
(elixir) lib/string.ex:1801: String.to_charlist/1
iex(4)>
I note that I am using Windows 10 and running iex from a Cmder shell (the same error occurs in windows cmd and in powershell).
Any suggestions for how I can fix this? (Besides using developing on a linux system
)
Trending in Questions
Hi all,
I’m working on a Phoenix project where I need to make external HTTP requests and I’m trying to choose between Req and HTTPoison....
New
I’m very new to Elixir, so sorry if this is a silly question:
I am wondering what happens to the mailbox of a process when the machine i...
New
How do I tell Hologram not to escape HTML when substituting into templates?
{Textile.render(@product.long_description)}
escapes the HTM...
New
Background
When trying to execute mix release on a Windows 11 machine for a Phoenix project I get the following error:
* assembling mark...
New
I’d like to create a file with 600 permissions (rw-------), but there’s a race condition if I create it and then change the permissions. ...
New
Is there a way to import record definitions in the iex shell for pretty printing (similar to the rr command in the erl shell)? This woul...
New
When I use Mix.install with a git dependency, using git@github.com:... how do I change the SSH identity used?
I’ve got two Github accoun...
New
Other Trending Topics
Hey there,
I haven’t seen a list of useful Elixir specific skills for the popular AI tools yet so I’m starting one here with my first.
...
New
Hi everyone :waving_hand:
I wanted to share something I’ve been building — NowBlind, a random chat app designed to help people discover ...
New
I fully migrated to my own harness from Anthropic/Gemini and I think it’s time to share it. Welcome DSH, the DeepSeek Harness, fully writ...
New
(I just needed to vent somewhere and LinkedIn is full of hope, or hype, I’m not sure which exactly)
AI dream has many faces, but general...
New
Localize is the next generation localisation library for Elixir. Think of it as ex_cldr version 3.0. The first version will be released ...
New
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #code-sync
- #discussion
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #ai
- #elixirconf-us
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #blog-post
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #elixirconf-eu
- #processes
- #security
- #api
- #forms
- #performance










Showing Posts 1 to 10- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
Marcus
You can try to use the raw representation of the string.
As you can see your string has just the wrong encoding.
michalmuskala
While I have very little windows experience, I would expect this error to be caused by shell being not in unicode encoding.
In general any unicode issues on windows related to the shell were solved by using the graphical
iex --werlshell.CharlesO
This is not always the case. For example i see the same / similar issues reading data from SQL… i’m not printing to the console… my solution is generating xml files from SQL data, and i run into the UnicodeConversionError as well.
For example:
%UnicodeConversionError{encoded: " <serial-no>1</serial-no>\n <full-name>UBA Pensions Custodian Limited </full-name>\n <contact-address>3rd Floor, 22B,", message: "invalid encoding starting at <<160, 73, 100, 111, 119, 117, 32, 84, 97, 121, 108, 111, 114, 32, 83, 60, 47, 99, 111, 110, 116, 97, 99, 116, 45, 97, 100, 100, 114, 101, 115, 115, 62, 10>>"}code:
kip
That looks like an UTF-8 encoding error (which is what the exception is reporting).
<< 160 >>is ISO8859 encoding for anon breaking space. For UTF-8 the encoding is<< 0xc2, 160 >>(or<< 0xc2, 0xa0 >>). The exception isn’t showing the full binary I know, but I assume that the previous byte to the referenced binary is not<< 0xc2 >>.I wonder if your database connection is set to
ISO8859instead ofUTF-8?CharlesO
I totally did not check I just went with the defaults
kip
If you have access to
psqltry:Also would be helpful to see the full binary (if possible) in order to see if its maybe the wrong encoding, or whether its really an encoding error.
The default encoding for postgresql is, iirc, utf8. So although a possibility it would be a little surprising if the database encoding was something else. But definitely possible.
CharlesO
Oh, i’m connecting to MS-SQL not postgresql
kip
Ahhhhhhhhhhh. No idea about the defaults there but since I think Windows has a predilection for ISO-8859, checking the encoding would definitely be worth doing first.
CharlesO
The safe bet would be to have a simple function that outputs just printable chars on String … like
to_printable/1that would eliminate guess work (… in may case)kip
Actually you can:
String.codepoints/1will decode what it can and show you the code points it can’t.