Cldr.Number.Parser.parse quite slow?

@Phillip, I’ll take a look (I’m the author) and see if there is anything I can do to improve the performance. Feel free to add an issue so I can track it properly as well.

Note however that Cldr.Number.Parser.parse/2 is probably not the best tool for this job. If you know you are only receiving data that has no localisations in it (no separators, no localised decimal digits, …) then using the standard library tools would be better. For example the following code parsers both integers and floats that have no formatting in them (no localisations) in about 1.46 μs versus 1.01 μs for Float.parse/1:

def parse_number(x) do
  case Integer.parse(x) do
    {integer, ""} -> integer
    other -> case Float.parse(x) do
      {float, ""} -> float
      _other -> x
    end
  end
end

``Cldr.Number.Parser.parse/2` is designed to be quite resilient in the face of localised and formatted numbers and that means there is definitely more work going on. Its typical use is to enhance user experience when parsing user-provided text input.

I did some more digging here and the issue isn’t primarily number parsing. Its related to repeatedly parsing the default locale which happens here if one is not supplied and if one wasn’t set with Cldr.put_locale/1.

You can probably get a speed up of 40x with the following addition to your original code:

  # Assumes you are only using one locale
  @locale Cldr.Locale.new!("en", MyApp.Cldr)

  def parse_value(value) do
    case Cldr.Number.Parser.parse(value, locale: @locale) do
      {:ok, number} -> number
      _ -> value
    end
  end

I will fix the underlying issue here and publish a new version of ex_cldr.

Nevertheless, my suggestions in the previous message still hold.

I have published ex_cldr version 2.17.1 which improves your example code by about 40x. The changelog entry is:

Bug Fixes

• Significantly improve the performance of Cldr.default_locale/0. In previously releases, the default locale was being parsed on each access. In this release it is parsed once and cached in the application environment. This improves performance by about 40x. Thanks to @Phillipp who brought this to attention in Elixir Forum

Wouldn‘t the process dict fit better here? It‘s not external to the computation and should therefore not experience race conditions.

In this specific case it’s the system wide default so the process dictionary isn’t really suitable. The system wide default has always been in the app environment, but as a binary and therefore parsed on each access. The only change is to now also store the parsed version as well, which does save about 1ms per access which a really big win.

Hey @kip,

thanks for your investigation and recommendations. You are right that it is probably better to use the standard lib for this job. I’ve tested the new version of the package and my code execution went from around 15 seconds to around 1,1 seconds (including 2 http requests over local network + xml parsing).

Glad that this led to a performance improvement in your (awesome) library.

Phillipp