shotleybuilder

shotleybuilder

Unicode for emoji - string concatenation

Hi,

I’m not understanding the behaviour when including unicode inside a string which is then string concatenated. The resulting string is not known when streamed into a file.

Here’s something that works:
"❔ foobar" -> "❔ foobar"
This works
"\u2754 foobar" -> "❔ foobar"

This doesn’t work:
"❔" <> "foobar" -> �
Neither does this
"\u2754" <> "foobar" -> �

What am I missing?

Most Liked

NobbZ

NobbZ

+U274c is represented by these 3 bytes in utf8: 0xE2 0x9D 0x8C.

If your terminal doesn’t support utf8, then you should probably inspect on a byte level rather than printed representation.

As you have not specified your terminals encoding, basically everything can be triggered by these bytes.

Assuming latin-X, which is often used in the windows world, then the 8x and 9x bytes are not used and can cause weirdnesses.

kip

kip

ex_cldr Core Team
iex> << 0x1F517 :: utf8 >>
"🔗"

iex> << 0xF0, 0x9F, 0x94, 0x97 >>
"🔗"

iex> << 0xF0, 0x9F, 0x94, 0x97>> == << 0x1f517 :: utf8 >>
true

d8 3d dd 17 is the UTF16 representation. Elixir is a UTF8 language where the encoding is f0 9f 94 97.

kip

kip

ex_cldr Core Team

With a little bit of erlang magic:

iex> :erlang.binary_to_list << 0x1f517 :: utf8 >>       
[240, 159, 148, 151]

I have a unicode library that examines code points. For example:

iex> Unicode.category << 0x1F49A :: utf8 >>             
[:So]
iex> Unicode.properties << 0x1F49A :: utf8 >>                                                                          
[[:emoji, :grapheme_base]]

Unfortunately the Regex module doesn’t support the Unicode character class [:So:]. Or any other Unicode character classes. I have another lib unicode_set that has some support for Unicode character classes. But regexes not yet - soon though.

Where Next?

Popular in Questions Top

greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
vrod
I am using the Starship cross-shell prompt – it seems pretty nice, but I get some errors: [WARN] - (starship::utils): Executing command ...
New
alice
Hey, Just curious what are the main benefits of Elixir compared to Clojure? When is Elixir more useful than Clojure and vice versa? Th...
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New
nobody
Hi! In PHP: $SERVER['SERVERADDR'] - in Elixir? Searched the docs for ip address and the web, no good results. Thanks!
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
JDanielMartinez
Hi! May someone helps me, please! I have two apps into an umbrella project: the first one is Database, which manages queries, and the se...
New

Other popular topics Top

vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
Darmani72
If I have a post route which an argument: post /my_post_route/:my_param1, MyController.my_post_handler How would get the post params ...
New
JakeBecker
TL;DR: I’ve just released an implementation of Microsoft’s IDE-independent Language Server Protocol for Elixir. It adds language support ...
1144 53578 245
New
Emily
I have VueJS GUIs with the project generated using Webpack. I have Elixir modules that will need to be used by the VueJS GUIs. I fore...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
PeterCarter
There are pre-rolled solutions for other frameworks that do work. However, Phoenix does not seem to have these. Have people had good expe...
New
sergio
Kind of like when jquery came out, it was super necessary. Existing drag and drop libraries have a bunch of baggage to support old browse...
New

We're in Beta

About us Mission Statement