abtrapp

abtrapp

Regular expressions in Elixir with unicode characters

Hi!

Can somebody please explain this:

iex(9)> Regex.match?(~r/\p{Lu}/, "Ö")
true
iex(10)> Regex.match?(~r/\p{Lu}/, "ö")
true
iex(11)> Regex.match?(~r/\p{Lu}/, "o")
false
iex(12)> Regex.match?(~r/\p{Lu}/, "O")
true

Why are all unicode characters that are not from the english alphabet uppercase characters in Elixir (in contrast to every other language I know and the definition of \p{Lu})?

Elixir 1.7.4 (compiled with Erlang/OTP 21)

Marked As Solved

chriseyre

chriseyre

There is the/u option to enable Unicode support on regex.

Also Liked

michalmuskala

michalmuskala

I will agree that the elixir regexes should include the u flag by default and require explicit opt-out. We generally should prefer correctness over speed with ways to opt-in into faster, but potentially incorrect results.

Unfortunately, the soonest we could be able to change this would be 2.0 (and there are no plans for 2.0 at the moment). This is because that would be a breaking change since many functions would start returning different results.

kip

kip

ex_cldr Core Team

i would challenge the “unicode is not usually needed” assumption. Unless you have complete confidence in your knowledge of the cultural context of your audience, Latin-1 is a poor assumption. And in an Elixir context, the String.t type is unicode so its not surprising that developers are caught out by the default behaviour of Regex being Latin-1 (even though I understand why the choice, I think its hard to justify as the default).

Latin-1 doesn’t even include the Euro symbol. @michalmuskala’s native language can’t be represented in Latin-1 (Polish is Latin-2). And given the global participation of this forum I’d warrant there are many members building applications for a global audience and therefore should not make limited assumptions on character input or output.

I suppose I may be an outlier on this (Australian, living in Singapore, and having worked in several Asian countries for quite a while). It even bugs me that we don’t consistently address José with the name his mother gave him :slight_smile:

Elixir is a Unicode-centric language. I think that the :re option "u" should be the default.

Nicd

Nicd

With unicode flag:

iex(1)> Regex.match?(~r/\p{Lu}/u, "Ö")
true
iex(2)> Regex.match?(~r/\p{Lu}/u, "ö")
false

Where Next?

Popular in Questions Top

chokchit
** (DBConnection.ConnectionError) connection not available and request was dropped from queue after 2733ms. You can configure how long re...
New
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
shahryarjb
Hello, I get Persian date from my client and convert it to normal calendar like this: def jalali_string_to_miladi_english_number(persi...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: The documentation above suggests that while ...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
PeterCarter
There are pre-rolled solutions for other frameworks that do work. However, Phoenix does not seem to have these. Have people had good expe...
New

Other popular topics Top

aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
Darmani72
If I have a post route which an argument: post /my_post_route/:my_param1, MyController.my_post_handler How would get the post params ...
New
AstonJ
Posting this to see if we can make things easier for people to get into Neovim. If you use Neovim and have a favourite distro please let ...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID<0.412.0> terminating ** (Postgrex.Error) FATAL...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
vrod
I am using the Starship cross-shell prompt – it seems pretty nice, but I get some errors: [WARN] - (starship::utils): Executing command ...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New
komlanvi
Hi everyone, I was playing with phoenix liveView but I run into an issue. I have a form and want to validate each input text when the te...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New

We're in Beta

About us Mission Statement