9mm

Help converting regex (to remove emojis)

I’m trying to make a regex to remove emojis based on this thread:

This causes an error:

Regex.replace(~r/[\u{1F600}-\u{1F6FF}]/, "💰 Monies! 💲", "")
# (Regex.CompileError) PCRE does not support \L, \l, \N{name}, \U, or \u at position 2

I tried using \x{FFFF} syntax instead according to this: Regex Tutorial: Introduction to Unicode Regular Expressions

But I get another error:

Regex.replace(~r/[\x{1F600}-\x{1F6FF}]/, "💰 Monies! 💲", "")
# (Regex.CompileError) character value in \x{} or \o{} is too large at position 9

Apparently I also need u to enable unicode but that doesnt seem to work either:

Regex.replace(~r/[\x{1F600}-\x{1F6FF}]/u, "💰 Monies! 💲", "")
"💰 Monies! 💲"

23 comments

#regex

2 5716 23

2023-04-12 20:00:42 UTC

Most Liked

kip

ex_cldr Core Team

You might have some better luck using the unicode character class So (other symbol). For example:

iex> x = "💰 Monies! 💲"                                
"💰 Monies! 💲"
iex> String.replace x, ~r/\p{So}/u, ""
" Monies! "

BTW, the “correct” approach to this would be:

iex> String.replace x, ~r/\p{Emoji}/u, ""

But erlang’s re module doesn’t support that character class.

Post #1

mudasobwa

Creator of Cure

There are many issues here. Both characters above are not in the range you specified in the first place.

▶ to_charlist "💰" 
#⇒ [128176]
▶ to_charlist "💲"
#⇒ [128178]
▶ 0x1F600
#⇒ 128512

To make it work for the range with \u, just interpolate the literals (I voluntarily changed the starting value for the range to what it probably should be):

▶ Regex.replace(~r/[#{"\u{1F000}"}-#{"\u{1F6FF}"}]/u, "💰 Monies! 💲", "")
" Monies! "

\x will work out of the box with a proper range:

Regex.replace(~r/[\x{1F000}-\x{1F6FF}]/u, "💰 Monies! 💲", "")
" Monies! "

Post #7

kip

ex_cldr Core Team

Try String.codepoints/1. Where the code point can’t be decoded to UTF-8 you’ll see a number.

iex> String.codepoints x
["a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p",
 "q", "r", "s", "t", "u", "v", "w", "x", "y", "z", ".", ".", ".", ".", "0", "1",
 "2", "3", "4", "5", "6", "7", "8", "9", " ", "不", "極", "，", "物", "片",
 "類", "書", "車", "裡", ...]

If you want to get the numeric value, add <<0>> to the string:

iex> x <> <<0>>
<<97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112,
  113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 46, 46, 46, 46, 48, 49, 50,
  51, 52, 53, 54, 55, 56, 57, 32, 228, 184, 141, 230, 165, 181, 239, 188, 140,
  ...>>

If you want the raw integers underneath use to_charlist/1:

iex> to_charlist x
[97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112,
 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 46, 46, 46, 46, 48, 49, 50,
 51, 52, 53, 54, 55, 56, 57, 32, 19981, 26997, 65292, 29289, 29255, 39006,
 26360, 36554, 35041, ...]

Post #14

Last Post!

sergio

Here’s what I ended up doing in my changeset to remove emojis from my schema’s name field:

def changeset(my_schema, attrs) do
  attrs = Map.update(attrs, :name, "", &String.replace(&1, ~r/\p{So}/u, ""))

  my_schema
  |> cast(attrs, [
    :name,
    # snip

Post #23