abtrapp

abtrapp

Regular expressions in Elixir with unicode characters

Hi!

Can somebody please explain this:

iex(9)> Regex.match?(~r/\p{Lu}/, "Ö")
true
iex(10)> Regex.match?(~r/\p{Lu}/, "ö")
true
iex(11)> Regex.match?(~r/\p{Lu}/, "o")
false
iex(12)> Regex.match?(~r/\p{Lu}/, "O")
true

Why are all unicode characters that are not from the english alphabet uppercase characters in Elixir (in contrast to every other language I know and the definition of \p{Lu})?

Elixir 1.7.4 (compiled with Erlang/OTP 21)

Marked As Solved

chriseyre

chriseyre

There is the/u option to enable Unicode support on regex.

Also Liked

michalmuskala

michalmuskala

I will agree that the elixir regexes should include the u flag by default and require explicit opt-out. We generally should prefer correctness over speed with ways to opt-in into faster, but potentially incorrect results.

Unfortunately, the soonest we could be able to change this would be 2.0 (and there are no plans for 2.0 at the moment). This is because that would be a breaking change since many functions would start returning different results.

kip

kip

ex_cldr Core Team

i would challenge the “unicode is not usually needed” assumption. Unless you have complete confidence in your knowledge of the cultural context of your audience, Latin-1 is a poor assumption. And in an Elixir context, the String.t type is unicode so its not surprising that developers are caught out by the default behaviour of Regex being Latin-1 (even though I understand why the choice, I think its hard to justify as the default).

Latin-1 doesn’t even include the Euro symbol. @michalmuskala’s native language can’t be represented in Latin-1 (Polish is Latin-2). And given the global participation of this forum I’d warrant there are many members building applications for a global audience and therefore should not make limited assumptions on character input or output.

I suppose I may be an outlier on this (Australian, living in Singapore, and having worked in several Asian countries for quite a while). It even bugs me that we don’t consistently address José with the name his mother gave him :slight_smile:

Elixir is a Unicode-centric language. I think that the :re option "u" should be the default.

Nicd

Nicd

With unicode flag:

iex(1)> Regex.match?(~r/\p{Lu}/u, "Ö")
true
iex(2)> Regex.match?(~r/\p{Lu}/u, "ö")
false

Where Next?

Popular in Questions Top

lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
mgjohns61585
Could someone help me? I’m making my first elixir program, number guessing game. I can’t figure out how to convert the user’s guess from ...
New
fireproofsocks
I’m working on defining a simple Ecto schema for a table (in PostGres), but I don’t see where I can define a column as NOT NULL. Conside...
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New
vac
Hi, I’m quite new in Elixir and I’m trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and I...
New
ycv005
I have followed this StackOverflow post to install the specific version of Erlang. And When I am running mix ecto.setup then getting fol...
New
stefanluptak
Hello everybody, usually, I use a 29" ultra-wide monitor for VSCode which can easily accomodate explorer (files panel) + file with code ...
New
RisingFromAshes
I’ve read in another post that it may be possible with a router helper - but I couldn’t find an appropriate one, and tbh, I’m still just ...
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New

Other popular topics Top

chrismccord
As promised, the first release candidate of Phoenix 1.3.0 is out! This release focuses on code generators with improved project structure...
New
JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
aesmail
Hello guys, I have finally made it. I created an admin interface for a framework. It’s been on my todo list for years and with the curre...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
Emily
I have VueJS GUIs with the project generated using Webpack. I have Elixir modules that will need to be used by the VueJS GUIs. I forese...
New
AstonJ
Please see the new poll here: Which code editor or IDE do you use? (Poll) (2022 Edition) It’s been a while since we first asked this, I...
208 31142 143
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
hariharasudhan94
Lets say I have map like this fetching from my database %{"_id" => #BSON.ObjectId<58eb1a7a9ad169198c3dXXXX>, "email" => ...
New

We're in Beta

About us Mission Statement