Ex_cldr - Common Locale Data Repository (CLDR) functions for Elixir

ex_cldr_numbers version 2.14.0 is released today. This release makes it easy to parse numbers and currencies in a localised manner. Some examples:

Parsing

# Interpret grouping and decimal marker in
# a localised way
iex> Cldr.Number.Parser.parse("+1.000,34", locale: "de")
{:ok, 1000.34}

# Using underscore as a separator is supported
iex> Cldr.Number.Parser.parse("-1_000_000.34")
{:ok, -1000000.34}

# The type of number can be specified: integer, float, Decimal
iex> Cldr.Number.Parser.parse("1.000", locale: "de", number: :integer)
{:ok, 1000}

# If the string doesn't fit the type an error is returned
iex> Cldr.Number.Parser.parse("+1.000,34", locale: "de", number: :integer)
{:error, "+1000.34"}

Scanning

Scan the string and separate numbers and string
iex> Cldr.Number.Parser.scan("£1_000_000.34")
["£", 1000000.34]

# In any arbitrary string
iex> Cldr.Number.Parser.scan("I want £1_000_000 dollars")
["I want £", 1000000, " dollars"]

iex> Cldr.Number.Parser.scan("The prize is 23")
["The prize is ", 23]

iex> Cldr.Number.Parser.scan("The lottery number is 23 for the next draw")
["The lottery number is ", 23, " for the next draw"]

iex> Cldr.Number.Parser.scan("The loss is -1.000 euros", locale: "de", number: :integer)
["The loss is ", -1000, " euros"]

Resolving currencies from strings

# Try to find the currency that reflects 
# string description
iex> Cldr.Number.Parser.scan("100 US dollars")
...> |> Cldr.Number.Parser.resolve_currencies
[100, :USD]

# Even with a fuzzy match
iex> Cldr.Number.Parser.scan("100 eurosports")
...> |> Cldr.Number.Parser.resolve_currencies(fuzzy: 0.8)
[100, :EUR]

# and in a locale-aware way
iex> Cldr.Number.Parser.scan("100 dollars des États-Unis")
...> |> Cldr.Number.Parser.resolve_currencies(locale: "fr")
[100, :USD]
1 Like

Ex_cldr_calendars version 1.9.0 published

The is minor release that primarily fixes several bugs in duration calculations. Did someone mention data and time are hard? The changelog is here.

Enhancements

Three relevant features are added:

  1. Cldr.Calendar.Duration.new/1 support for Date.Range.t and for CalendarInterval.t. CalendarInterval is defined by @wojtekmach’s excellent calendar_interval library.

  2. Cldr.Calendar.plus/2 to support adding a duration to a Calendar.date. Support for adding a duration to a Calendar.datetime is not yet provided.

  3. String.Chars support for Cldr.Calendar.Duration makes it easy to output localised strings representing a duration.

Examples

# Create a duration, then add it back to the beginning date
# and check the round trip is correct.
iex> {:ok, duration} = Cldr.Calendar.Duration.new(~D[2019-01-01], ~D[2019-12-31])
{:ok,
 %Cldr.Calendar.Duration{
   day: 30,
   hour: 0,
   microsecond: 0,
   minute: 0,
   month: 11,
   second: 0,
   year: 0
 }}

iex> Cldr.Calendar.plus ~D[2019-01-01], duration
~D[2019-12-31]

# Create a duration from a CalendarInterval
iex> use CalendarInterval
iex> Cldr.Calendar.Duration.new ~I"2020-01/12"
{:ok,
 %Cldr.Calendar.Duration{
   day: 30,
   hour: 0,
   microsecond: 0,
   minute: 0,
   month: 11,
   second: 0,
   year: 0
 }}

# Support the String.Chars protocol
to_string Cldr.Calendar.Duration.new!(~D[2020-01-01], ~D[2020-03-01]) ==
"2 months"

iex> to_string Cldr.Calendar.Duration.new!(~D[2020-01-01], ~D[2020-03-04]) ==
"2 months and 3 days"

iex> to_string Cldr.Calendar.Duration.new!(~D[2020-01-01], ~D[2021-03-04]) ==
"1 year, 2 months and 3 days"

iex> to_string Cldr.Calendar.Duration.new!(~D[2020-01-01], ~U[2021-03-04 01:02:03.0Z]) ==
"1 year, 2 months, 3 days, 1 hour, 2 minutes and 3 seconds"
4 Likes

This weeks release bring you ex_cldr_dates_times version 2.5.0. As usual here is the changelog and the github repo.

This release reinforces my emerging view that developing these CLDR-based libraries is:

  • 30% understanding the structure of the CLDR data, the nuances of which there are many and marvelling at how smart the people that build and maintain CLDR are.

  • 30% distilling a functionally very rich set of data into an api design that is approachable and easy to use for 90% of cases with enough flexibility to use the another 8% in a straight forward manner (there are always a few small parts that I omit for my own sanity, and maybe the library users too).

  • 30% on documentation and tests

  • 10% on writing code.

Date/Time Interval formatting

This release adds formatting of date, time and datetime intervals. Intervals can be two date/time/datetimes or a Date.Range or if you use the fabulous calendar_interval library then a CalendarInterval can also be used.

The cool thing about formatting intervals is collapsing common date/time elements into a readable format for every configured locale. This is helpful whether you use one locale or many.

Date Examples

# Same month and year
iex> Cldr.Interval.to_string ~D[2020-01-01], ~D[2020-01-31]
{:ok, "Jan 1 – 31, 2020"}

# Same year but different months
iex> Cldr.Interval.to_string ~D[2020-01-01], ~D[2020-02-11]
{:ok, "Jan 1 – Feb 11, 2020"}

# Standard formats :short, :medium and :long are defined
iex> Cldr.Interval.to_string ~D[2020-01-01], ~D[2020-01-31], format: :long
{:ok, "Wed, Jan 1 – Fri, Jan 31, 2020"}

# Of course localised ....
iex> Cldr.Interval.to_string ~D[2020-01-01], ~D[2020-01-31], locale: "th" 
{:ok, "1–31 ม.ค. 2020"}

# Including other number systems
iex> Cldr.Interval.to_string ~D[2020-01-01], ~D[2020-01-31], locale: "th", number_system: :thai
{:ok, "๑–๓๑ ม.ค. ๒๐๒๐"}

# Date ranges can be used:
iex> Cldr.Interval.to_string Date.range(~D[2020-01-01], ~D[2020-02-11])                        
{:ok, "Jan 1 – Feb 11, 2020"}

# As can the very cool calendar intervals
iex> use CalendarInterval
iex> Cldr.Interval.to_string ~I"2020-02-01/06-30"                        
{:ok, "Feb 1 – Jun 30, 2020"}

iex> Cldr.Interval.to_string ~I"2020-01-01/31"                           
{:ok, "Jan 1 – 31, 2020"}

iex> Cldr.Interval.to_string ~I"2020-02-01 00:00/10:05"
{:ok, "Feb 1, 2020, 12:00:00 AM – 10:05:00 AM"}
3 Likes

Introducing ex_cldr_calendars_composite which allows composition of multiple calendars at different points of the time continuum. A small but interesting project for a Saturday afternoon.

The Gregorian Proleptic calendar is the most common calendar implementation (including Elixir’s Calendar module) and its behaviour is correct for the majority of contemporary uses. It is explicitly required by ISO8601.

However the fact that it represents dates before the introduction of the Gregorian calendar introduces inaccuracies in dates expressed before its introduction in a given territory. The Gregorian calendar was introduced starting on Friday, 15 October 1582 in the Papal States and was still being introduced in the 20th century with Russia adopting it (for non religious purposes) between 1918-1920 with parts of Romania, Greece, Serbia and Yugoslavia around the same period.

ex_cldr_calendars_composite supports composing two or more calendars to more correctly specify dates before the introduction of the Gregorian calendar.

Example composite Calendar for England

England (it is then colonies) introduced the Gregorian calendar on September 14th, 1752. This can be configured as:

defmodule England do
  use Cldr.Calendar.Composite,
    calendars: ~D[1752-09-14],
    # The :base_calendar is optional and this is the default
    base_calendar: Cldr.Calendar.Julian. 
end

As a result we can now treats dates as normal and have them respect the transition.

Composed calendar examples

Using our example for England, dates now reflect the Julian calendar before September 14th, 1752 and the Gregorian calendar on or after it. This has some expected but unusual results. For example:

# September 1752 is 11 days "shorter" at the transition
iex> Cldr.Calendar.England.days_in_month 1752, 9  
19

# And the full year is also shorter as a result
iex> Cldr.Calendar.England.days_in_year 1752    
355

In all other respects, composite calendars implement the Calendar and Cldr.Calendar behaviours.

A more complex example

More complex compositions are possible. Consider that Egypt used the Coptic calendar from about 238 BCE. The Julian calendar was introduced in approximately 30 BCE. And the Gregorian calendar in 1875. This can be approximated with:

defmodule Cldr.Calendar.Composite.Egypt do
  use Cldr.Calendar.Composite,
    calendars: [
      ~D[-0045-01-01 Cldr.Calendar.Julian],
      ~D[1875-09-01]
    ],
    base_calendar: Cldr.Calendar.Coptic
end

A Coptic calendar implementation is also available as ex_cldr_calendars_coptic.

4 Likes

@kip I’ve restarted working on my translation library based ICU messages. I’ve found that your implementation of the CLDR message format works great. I’ll publish it as soon as I have something useful, which should include the following:

  • Translations should be cached in an efficient way, like compiling translations into raw functions (which would be the most efficient way possible) or using parsed ICU messages and storing them in something like :persistent_term
  • It should be possible to edit translations at runtime
  • Based on the above, for Phoenix apps, there should be a plugin that allows you to click on an untranslated/translated string and edit it in place (possibly opening a new web page), and the new translation should be available on the next page reload 8or even forcing a page reload after the translation)
  • Unlike you, I’m not too keen on using an external web service to edit translations (unless is something you can self-host in development so that you can use it in the way I described above)

Glad you’re back on this :slight_smile:

My current implementation only serves to prove out the parser and renderer. Definitely not a full implementation as you point out. It why I was advocating for splitting out the .pot parsing separately from Gettext so it could be a general message serialisation format.

My original thinking was that a backend could be one in which messages are baked into code (like Gettext), or kept at runtime (like :persistent_term) or managed in a web service and cached. I think different use cases suggest different approaches.

Anyway, more than happy to adjust ex_cldr_messages to fit in with the work you’re doing. I’ve been “aggressively monitoring” the MessageFormat Working Group which is a collaboration to define the evolution of message localisation and all of the big players are participating. But recent pace of progress suggests thats going to take some time to mature.

1 Like

I already have a .po parser that’s implemented outside of gettext and can parse a .po file into a list of %Translation{} which can be used by my implementation. You can then use Poedit or a similar program to edit those files.

I published a small update to ex_cldr_messages primarily to fix a warning on Elixir 1.11 but also to support later versions of ex_cldr_units and friends.

Back to your original message and design objectives and how ex_cldr_messages might fit in:

  • Cldr.Message.format/2 operates totally at runtime, parsing and interpreting the format and returning a string

  • Cldr.Message.Parser.parse/1 parses a format and returns list that can be serialised safely and interpreted at runtime. Translations could be stored internally in this format.

  • Cldr.Message.Interpreter.format/3 takes the output from Cldr.Message.Parser.parse/1 and interprets it in the context of a binding (a Keyword list) and returns an io_list. Which of course can easily be processed into a binary if required.

There is a macro form of format/1 defined in a backend CLDR module that parses at compile time and performs some basic checks on bindings (where possible). This is an optimisation of limited use at the moment since this doesn’t support runtime localisation.

1 Like

You can explore the latest version of Mezzofanti here: GitHub - tmbb/ex_i18n: Experimental translation library for Elixir

Currently it supports only a gettext backend (that it, it supports translations supplied as gettext files, even though the messages themselves use the ICU message format).

Currently I don’t have much use for this, but I might ad dit in the future. My rationale is that it’s not possible to translate a string that isn’t a static string marked as translatable in the source, so all translatable strings should be wrapped ina ,macro and available at compile-time. This is different from gettext, where one can feed it dynamic strings, which will be translated according to the .po files. This is a requirement for packages like ecto, which can’t access the dependent application’s Gettext backend. However, because my Mezzofanti library is able to translate strings in dependencies (this was actually the main design criterion for my library), being able to handle dynamic strings is not as useful.

That is what I’m doing now. I’m currently embedding these lists in the code, but I can add them to a persistent term with little work and probably no relevant performance impact. I’m not doing this right now because I’m trying to reduce the amount of moving parts, but I have plans for adding it in the future.

I’m supplying the locale through the process dictionary.

This module illustrates a some interesting features (explained in the comments):

defmodule Mezzofanti.Fixtures.ExampleModule do
  use Mezzofanti
  # Not that I don't need to require or impor a Mezzofanti backend here.
  # I just use the Mezzofanti library, and once a backend is configured
  # it will automatically become aware of these messages
  # (even if the messages exist in a different application)

  def f() do
    # A simple static translation
    translate("Hello world!")
  end

  def g(guest) do
    # A translation with a variable.
    # This translation contains a context, possibly to disambiguate it
    # from a similar string which should be translated in a different way.
    # Mezzofanti will keep equal strings with different contexts separate.
    translate("Hello {guest}!", context: "a message", variables: [guest: guest])
  end

  def h(user, nr_photos) do
    # A more complex translation with two variables and plural forms.
    # It also defines a different domain.
    translate("""
    {nr_photos, plural,
      =0 {{user} didn't take any photos.}
      =1 {{user} took one photo.}
      other {{user} took # photos.}}\
    """,
      domain: "photos",
      variables: [
        user: user,
        nr_photos: nr_photos
      ])
  end
end

The following commands will create a priv/mezofanti/ dir for the locale data:

mix mezzofanti.extract
mix mezzofanti.new_locale pt-PT
mix mezzofanti.merge

This will result in:

priv/
  mezzofanti/
    pt-PT/
      LC_MESSAGES/
        default.po
        photos.po
    default.pot
    photos.pot

The Mezzofanti and Cldr backends should be created and added to the apps’ config. Mezzofanti expects a single global Mezzofanti backend, so that you can centralize the translations of several applications/dependencies in the same place (this is a quastionable choice, but again, it was the main design goal of this library).

Like gettext, Mezzofanti reads the locale from the process dictionary. There are Mezzofanti.set_locale() and Mezzofanti.get_locale() for that. And also my favourite function for testing, the function Mezzofanti.with_locale(locale, func), which sets the locale, executes the func and then resets the old locale.

For example:

iex> alias Mezzofanti.Fixtures.ExampleModule
Mezzofanti.Fixtures.ExampleModule

iex> ExampleModule.f()
["Hello world!"]

iex> ExampleModule.f() |> to_string()
"Hello world!"

iex> ExampleModule.g("tmbb") |> to_string()
"Hello tmbb!"

iex> ExampleModule.h("kip", 2) |> to_string()
"kip took 2 photos."

iex> ExampleModule.h("kip", 1) |> to_string()
"kip took one photo."

iex> ExampleModule.h("kip", 0) |> to_string()
"kip didn't take any photos."

iex> Mezzofanti.with_locale("pt-PT", fn -> ExampleModule.h("kip", 0) |> to_string() end)
"kip não tirou fotografias nenhumas."

iex> Mezzofanti.with_locale("pt-PT", fn -> ExampleModule.h("kip", 1) |> to_string() end)
"kip tirou uma fotografia"

iex> Mezzofanti.with_locale("pt-PT", fn -> ExampleModule.h("kip", 2) |> to_string() end)
"kip tirou 2 fotografias"

Mezzofanti also contains two “fake” locales for pseudoloalization. Pseudolocalization is a process of automatically replacing latin characters by similar foreign characters (like characters that are visually similar or contain diacritics not present in English) and extending words with extra characters so that they ar longer than the originals. This produces still readable strings, helps detect places where strings in your application haven’t been translated and helps detect places where you don’t have space for longer strings.

The “pseudo” locale provides pseudolocalization for simple text:

iex> Mezzofanti.with_locale("pseudo", fn -> ExampleModule.h("kip", 2) |> to_string() end)
"ǩıƥ~ ťøøǩ~ 2 ƥȟøťøš~~."

iex> Mezzofanti.with_locale("pseudo", fn -> ExampleModule.h("kip", 1) |> to_string() end)
"ǩıƥ~ ťøøǩ~ øñê~ ƥȟøťø~."

The “pseudo_html” provides pseudolocalization for HTML text. It preserves HTML tags and HTML entities, while localizing the rest of the text. In the first line below, you can see how “normal” pseudolocalization is not sufficient for HTML:

iex> Mezzofanti.with_locale("pseudo", fn -> ExampleModule.i() |> to_string() end)        
"Ťȟıš~ ɱêššàğê~~ ċøñťàıñš~~ <šťȓøñğ>ȟťɱĺ~~~~ ťàğš</šťȓøñğ>~~~~ &àɱƥ~; ñàšťÿ~ šťüƒƒ~..."

iex> Mezzofanti.with_locale("pseudo_html", fn -> ExampleModule.i() |> to_string() end)
"Ťȟıš~ ɱêššàğê~~ ċøñťàıñš~~ <strong>ȟťɱĺ~ ťàğš~</strong> &amp; ñàšťÿ~ šťüƒƒ~..."

Thats all very cool. ex_cldr and friends also use the process dictionary for the locale too. Do you have any thoughts on how we might synchronise the gettext, mezzofanti and ex_cldr locales? In ex_cldr I built in awareness of gettext locales to make interoperability easier but that’s not a very scaleable solution.

ex_cldr_messages needs to know the locale for when it localises number formats, dates, units, … By default it uses the locale in the process dictionary (or a system default) but that risks being out of sync with Mezzofanti.

Looks like I know what I’m doing this weekend!

We don’t need to synchronize anything. Your Cldr.Message.format/3 function accepts a locale as an option in the options list. I simply read my :mezzofanti_locale key from the process dictionary and feed that as an argument to Cldr.Message.format/3, which is totally unaware I’m getting the locale from the process dictionary. Everything is decoupled and Cldr doesn’t need to be aware of anything.

The key module is this:

defmodule Mezzofanti.Translator do
  @moduledoc false
  alias Mezzofanti.Config

  @doc false
  def __translate__(hash, variables, translation) do
    locale = Mezzofanti.get_locale() || Cldr.default_locale().cldr_locale_name

    case Config.backend() do
      nil ->
        # Interpolate the message with the default locale
        Cldr.Message.format_list(translation.parsed, variables, locale: locale)

      module ->
        module.translate_from_hash(hash, locale, variables, translation)
    end
  end
end

All translations in Mezzofanti go through the __translate__() function. This function reads the locale from the process and feeds it to the module.translate_from_hash/4 function, which then feeds it into Cldr.Message.format() (the code where this happens is not as nice to show because it’s built inside a macro, but you get the idea)

It’s October and release time for CLDR version 38. With some adjustments to the ex_cldr build process I can now get new releases of ex_cldr out the door at the same time.

Updated ex_cldr library versions:

4 Likes

This week, downloads of ex_cldr hits 500,000.

A little over 4 years ago I decided to follow @josevalim to Elixir and having spent a bit of time hanging around on the edges of the Ruby i18n Gem I thought working on an I18n project for Elixir would be a good way to spend a weekend getting to know the language.

That weekend turned into a 4 year journey. Fast forward and now there have been

Many thanks to @lostkobrakai, @danschultzer, @szTheory, @KronidDeth, @michalmuskala, @zaack, @tcitworld, @ jueberschlag, @ Schultzer, @Zurga, @yuchunc, @kwando, @bglusman, @kianmeng, @Qqwy, @NickNeck and @dbernheisel for their support. It’s been a wild ride and there’s still more to come!

To celebrate what is, for me, a pretty big milestone I’ve started a blog on i18n with Elixir with the first two articles at Adventures in Internationalization and Localization.

Yeah, creative it isn’t and my writing is really rusty. Like software, I assume it will get better with practise!

20 Likes

Congratulations @kip and thank you for the extensive contributions to the ecosystem!

The blog looks great, I’ve already learned about Cldr.Chars, which I found pretty neat. :slight_smile:

3 Likes

thank you @kip for all the great work on cldr :raised_hands:

1 Like

Congratulation @kip, this is a big milestone! And reading your blog is like reading your code, its easy to read and well thought out! Thanks again for all your contributions!

1 Like

Today is the launch of a small addition to the ex_cldr family, ex_cldr_strftime. Its only purpose is to provide a keyword list of options to Calendar.strftime/3 that was introduced in Elixir 1.11.

This makes it easy to use the localised date/time/datetime formats of CLDR for over 500 locales but using the format strings of Calendar.strftime/3.

Examples

These examples use the %x, %X and %c format placeholders to delegate to the preferred formats defined by CLDR. The CLDR format strings are translated to the strftime/3 format at compile time so runtime performance is on par with other format strings.

Calendar.strftime ~U[2019-08-26 13:52:06.0Z], "%x", Cldr.Strftime.strftime_options!()
"Aug 26, 2019"

iex> Calendar.strftime ~U[2019-08-26 13:52:06.0Z], "%X", Cldr.Strftime.strftime_options!()
"13:52:06 PM"

iex> Calendar.strftime ~U[2019-08-26 13:52:06.0Z], "%c", Cldr.Strftime.strftime_options!()
"Aug 26, 2019, 13:52:06 PM"

iex> Calendar.strftime ~U[2019-08-26 13:52:06.0Z], "%c", Cldr.Strftime.strftime_options!("ja")
"2019/08/26 13:52:06"

iex> Calendar.strftime ~U[2019-08-26 13:52:06.0Z], "%c", Cldr.Strftime.strftime_options!("ja", format: :long)
"2019年08月26日 13:52:06 +0000"

iex> Calendar.strftime ~U[2019-08-26 13:52:06.0Z], "%c", Cldr.Strftime.strftime_options!("pl")
"26 sie 2019, 13:52:06"

iex> Calendar.strftime ~U[2019-08-26 13:52:06.0Z], "%c", Cldr.Strftime.strftime_options!("pl", format: :long)
"26 sierpnia 2019 13:52:06 +0000"
6 Likes

How is the ICU message parsing and rendering going? My internationalization library is pretty good now. It uses .PO and .POT files for the backend, which means translations can be edited with programs like Poedit. The limiting factor is only the ICU message handling, and I’m not sure exactly which features are supported :slight_smile:

CLDR version 39 was released today. With an updated tool chain and release process I can now update the ex_cldr libraries and release them on the same day. The following is a summary of the main updates to the family of libraries. Happy localising!

ex_cldr version 20.0.0

  • Updates to CLDR version 39 data.

  • Add Cldr.Locale.parent/1 to return the parent locale according. This is not exactly the same as the CLDR locale inheritance rules

  • Add Cldr.Locale.parents/2 to return a list of parents up to and including the root locale. It is a recursive use of Cldr.Locale.parent/1.

  • Add locale display name data to the locale files. This data can be used to format a locale for UI usage.

  • Add subdivision translations to the locale files. This data can be used to format subdivision names for UI usage. Thanks to @mskv. Closes #144.

  • Add grammatical features to the repository. This data is used in ex_cldr_units. See also Cldr.Config.grammatical_features/0.

  • Add grammatical gender to the repository. This data is used in ex_cldr_units. See also Cldr.Config.grammatical_gender/0.

  • Make Cldr.Locale.first_match/2 a public function. This function is useful for other CLDR-based libraries to help resolve the files of localised content in CLDR.

  • Add :add_fallback_locales to the backend configuration. When true, the fallback locales of the configured backend locales is also added to the configuration. The default is false and therefore by default there is no change to behaviour from previous releases. Setting this option to true enables means that data that is stored in parent locales rather than child locales can be processed. This applies particularly to rules-based number formats and subdivision data. These data aren’t stored in all locales - generally they are stored in the base language locale.

  • Add Cldr.Config.fallback_chain/1 which takes a locale name and returns a list of locales from which this locale inherits up to but not including the root locale.

  • Add Cldr.Config.fallback/1 which takes a locale name and returns the direct parent of the locale name.

  • Rename alias key subdivisionAlias to subdivision

  • Fix Cldr.Substitution.substitute/2 when the template has no substitutions. Thanks to @jarrodmoldrich. Closes ex_cldr_units #20.

ex_cldr_currencies 2.9.1

  • Depends upon ex_cldr version 2.20 which embodies CLDR39 data.

  • Add Cldr.Currency.display_name/2 that returns a localized display name suitable for use in UI applications.

  • Add implementation of String.Chars and Cldr.Chars protocols for t:Cldr.Currency structs.

ex_cldr_units version 3.5.0

Overview

In this release the Cldr.Unit.to_string/{1, 2, 3} function has been rewritten and the concrete impementation is now in Cldr.Unit.Format. The primary reasons for rewriting are:

  1. Improves performance by 20% over the old implementation.
  2. Supports grammatical case and grammatical gender. These allow for better sentence formation in a localised fashion. Only are few locales have the required data for now (for example, fr and de) however more locales will have data in upcoming CLDR releases.

Note that full testing of grammatical case and grammatical gender variations is not yet complete.

Soft Deprecation

  • The function Cldr.Unit.to_iolist/{1, 2, 3} is soft deprecated. It is still available and no deprecation warning is emitted. It will however be removed from the public API in a future release. This function is primarily used to support implementation of Cldr.Unit.to_string/3

  • As of this release, argument checking in Cldr.Unit.to_iolist/3 is less rigorous in order to avoid the relatively expensive argument normalization process happening twice (once in Cldr.Unit.to_string/3 and then again in Cldr.Unit.to_iolist/3).

Bug Fixes

  • The new string formatter correctly assembles units with an SI prefix (ie millimeter) in languages such as German where the noun is capitalized.

  • Fixes calculating the base unit when the unit is a complex compound unit.

  • Remove double parsing when calling Cldr.Unit.new/2 and the unit is not in Cldr.Unit.known_units/0

  • Ensure Cldr.Unit.unit_category/1 returns an error tuple if the category is unknown

Enhancements

  • Updated to require ex_cldr version 2.20 which includes CLDR 39 data.

  • Add Cldr.Unit.validate_grammatical_gender/2

  • Add Cldr.Unit.known_grammatical_cases/0

  • Add Cldr.Unit.known_grammatical_genders/0

  • Add Cldr.Unit.known_measurement_system_names/0

  • Add Cldr.Unit.invert/1 to invert a “per” unit. This allows for increased compatibility for conversions. For example, “liters per 100 kilometers” is a measure of consumption, as is “miles per gallon”. However these two units are not convertible without inverting one of them first since one is “volume per length” and the other is “length per volume”.

  • Add Cldr.Unit.conversion_for/2 to return a conversion list used when converting one unit to another.

  • Add Cldr.Unit.grammatical_gender/2 to return the grammatical gender for a given unit and locale

  • Add Cldr.Unit.conversion_for/2 to return a conversion list used when converting one unit to another.

  • Add support for grammatical cases for Cldr.Unit.to_string/2 and Cldr.Unit.to_iolist/2. Not all locales support more than the nominative case. The nominative case is the default. Any configured “Additional Units” in a backend module will need to be modified to put the localisations a map with the key :nominative. See the readme for more information on migrating additional units. On example is:

defmodule MyApp.Cldr do
  use Cldr.Unit.Additional

  use Cldr,
    locales: ["en", "fr", "de", "bs", "af", "af-NA", "se-SE"],
    default_locale: "en",
    providers: [Cldr.Number, Cldr.Unit, Cldr.List]

  unit_localization(:person, "en", :long,
    nominative: %{
      one: "{0} person",
      other: "{0} people"
    },
    display_name: "people"
  )
end
  • Support conversions where one of the base units is the inverted conversion of the other. This allows conversion between, for example, mile per gallon and liter per 100 kilometer. These are both compound units of length and volume but are inverse representations from each other.
4 Likes

@tmbb To the best of my knowledge the full ICU message format is supported for parsing (with some modest extensions that leverage Elixir protocols) and for formatting. The only area I haven’t fully explored is how well the compile-time integration with mezzofanti operates.

Still very motivated to support your efforts - I think its a huge step forward for Elixir I18n and L10n. Let me know where you see issues or opportunities and I’ll jump on it. Issues very welcome - now that I have CLDR39 updates done and launched I have a little more capacity to work on this.