Yes, definitely looks like a bug. Would you mind opening an issue on ex_cldr_dates_times?
kip
ex_cldr Core Team
kip
ex_cldr Core Team
Some updates to the Elixir CLDR libraries:
-
Improved locale parsing for language tags including complete parsing of the -u- and -t- extensions in ex_cldr version 2.23.0. For a more accessible description see this presentation by Mark Davis from Google. It also adds the
Cldr.Language.Sigilmodule definingsigil_lfor defining language tags. -
Added the ex_cldr_locale_display library to - you guessed it - render a language tag in a human readable format suitable for presentation in a UI.
-
Updated ex_cldr_html to support producing localised
Phoenix.HTML.Form.select/4select tags.
Locale display with ex_cldr_locale_display
For common locales the output is as you would expect - and perhaps you’re thinking why even have a library for these?
iex> Cldr.display_name ~l"en"u
"English"
iex`> Cldr.display_name ~l"fr"u
"French"
iex`> Cldr.display_name ~l"zh"u
"Chinese"
Things get more interesting with combinations of languages, scrips and territories:
iex> Cldr.display_name ~l"en-GB"u
"British English"
iex> Cldr.display_name ~l"fr-CA"u
"Canadian French"
iex> Cldr.display_name ~l"zh-Hant"u
"Traditional Chinese"
iex> Cldr.display_name ~l"zh-Hant"
"Traditional Chinese (Taiwan)"
iex> Cldr.display_name ~l"zh-Hans"
"Simplified Chinese (China)"
iex> Cldr.display_name ~l"nl-BE"u
"Flemish"
And if you really want to get into the power of language tags then this example``` can be a bit mind-bending:
iex> Cldr.display_name ~l"fr-CA-u-ca-gregory-nu-arab-cu-usd-cf-account-ms-uksystem-t-hi-h0-hybrid"u
"Canadian French (Hybrid: Hindi, Gregorian Calendar, Accounting Currency Format, Currency: US$, Imperial Measurement System, Arabic-Indic Digits)"
iex> Cldr.display_name ~l"fr-CA-u-ca-gregory-nu-arab-cu-usd-cf-account-ms-uksystem"u, locale: "zh-Hant"
"法文（加拿大）（公曆，會計貨幣格式，貨幣：US$，英制度量單位系統，阿拉伯-印度數字）"
iex> Cldr.display_name ~l"fr-CA-u-ca-gregory-nu-arab-cu-usd-cf-account-ms-uksystem"u, locale: "ar"
"الفرنسية الكندية (التقويم الميلادي، تنسيق العملة للحسابات، العملة: US$، نظام القياس البريطاني، الأرقام العربية الهندية)"
iex> Cldr.display_name ~l"fr-CA-u-ca-gregory-nu-arab-cu-usd-cf-account-ms-uksystem"u, locale: "he"
"צרפתית (קנדה) (לוח השנה הגרגוריאני, תבנית מטבע למטרות חשבונאות, מטבע: US$, מערכת מדידה אימפריאלית, ספרות הודיות-ערביות)"
iex> Cldr.display_name ~l"fr-CA-u-ca-gregory-nu-arab-cu-usd-cf-account-ms-uksystem"u, locale: "th"
"ฝรั่งเศส - แคนาดา (ปฏิทินเกรกอเรียน, รูปแบบสกุลเงินบัญชี, สกุลเงิน: US$, ระบบการวัดอิมพีเรียล, ตัวเลขอารบิก-อินดิก)"
ex_cldr supports most of the power of language tags and during localisation will respect regional overrides, subdivisions, currencies, currency formats, unit measurement systems and so on.
HTML select tags for locales with ex_cldr_html
Providing users with a means to select their preferred locale is important for good UX and DX. With Cldr.HTML.Locale.select/3 thats now easy:
# Select from the locales configured in `Cldr.default_backend/0` and localize them
# using the locale `Cldr.default_locale/0`
iex> Cldr.HTML.Locale.select(:my_form, :locales) |> Phoenix.HTML.safe_to_string()
Produces, when formatted:
<select id="my_form_locales" name="my_form[locales]">
<option value="ar">Arabic</option>
<option value="zh-Hans">Chinese (Simplified)</option>
<option value="zh-Hant">Chinese (Traditional)</option>
<option value="en">English</option>
<option value="he">Hebrew</option>
<option value="th">Thai</option>
</select>
# Select from the locales configured in `Cldr.default_backend/0` and localize them
# using their own locale`
iex> Cldr.HTML.Locale.select(:my_form, :locales, locale: :identity) |> Phoenix.HTML.safe_to_string()
Produces, when formatted:
<select id="my_form_locales" name="my_form[locales]">
<option value="en">English</option>
<option value="he">עברית</option>
<option value="ar">العربية</option>
<option value="th">ไทย</option>
<option value="zh-Hans">简体中文</option>
<option value="zh-Hant">繁體中文</option>
</select>
8
kip
ex_cldr Core Team
CLDR release 40 was release today by the Unicode Consortium.
CLDR 40 Changes
- Adds locales
sc,kgp,dsb,hsb,yrl - Add grammatical features (gender and case) to an additional 29 locales. In many languages, forming grammatical phrases requires dealing with grammatical gender and case. Without that, it can sound as bad as “on top of 3 hours” instead of “in 3 hours”. The overall goal for CLDR is to supply building blocks so that implementations of advanced message formatting can handle gender and case. ex_cldr_units allows specifying grammatical gender and case when formatting units of measure.
- Added or modified over 140,000 data fields
Ex_CLDR library updates
-
ex_cldr version 2.24.0 is updated to use CLDR 40 data and has a lot of internal refactoring to make it easier to update to new CLDR releases in the future.
-
ex_cldr_numbers version 2.23.0 allows using any of the rules based number formats when calling
Cldr.Number.to_string/3. -
ex_cldr_calendars version 1.17.0 adds support for Chinese, Japanese and Korean lunisolar calendars that will be released in the coming weeks. Early adopters can try the library now from GitHub. (documentation and testing is not complete).
-
ex_cldr_dates_times version version 2.10.0 adds localization in support of Chinese, Japanese and Korea calendars (era, month names, cyclic year, related gregorian year)
-
ex_cldr_units version 3.8.0 adds support for grammatical features (gender and case) in the newly-supported locales.
-
ex_cldr_territories version 2.3.0 by @Schultzer is released to support
ex_cldrversion 2.24.0. -
ex_cldr_languages version 0.3.0 by @LostKobrakai is release to support
ex_cldrversion 2.24.0.
4
kip
ex_cldr Core Team
Its release Sunday and today ex_cldr_units version 3.9.0 is released with the following enhancements.
Enhancements
- Add support for currency-based units. This allows for calculations and formatting of units such as “$2 per gallon”. For this example, the unit would be created with
Cldr.Unit.new(2, "curr-usd-per-gallon"). The inverse is also possible, for example:
iex> MyApp.Cldr.Unit.to_string(Cldr.Unit.new!(2, "curr-usd-per-gallon"))
{:ok, "$2.00 per gallon"}
iex> MyApp.Cldr.Unit.to_string(Cldr.Unit.new!(2, "gallon-per-curr-usd"))
{:ok, "2 gallons per US dollar"}
- Add support for binary factor prefixed units. These units are factors of 1024 and include “kibi”, “mebi”, “gibi”, “tebi”, “pebi”, “exbi”, “zebi” and “yobi”. For example:
iex> MyApp.Cldr.Unit.to_string Cldr.Unit.new!(3, :gibibyte)
{:ok, "3 gibibytes"}
- Add support for integer prefixes for units. This is useful for units like “liters per 100 kilometers” or “25 calories per 100 grams”. For example:
iex> MyApp.Cldr.Unit.to_string Cldr.Unit.new!(25, "calorie_per_100-gram")
{:ok, "25 calories per 100 grams"}
4
slouchpie
This is so crazy! I was playing with it and I noticed that not all units will work (probably “per” is being parsed). For example:
iex [8] > Gaming.Cldr.Unit.to_string(Cldr.Unit.new!(2, "curr-usd-per-mile-per-gallon"))
** (Cldr.UnknownUnitError) Unknown unit was detected at "per_gallon"
(ex_cldr_units 3.9.0) lib/cldr/unit.ex:637: Cldr.Unit.new!/3
“per” occurs twice. Although it’s strange because this works fine:
iex [9] > Gaming.Cldr.Unit.to_string(Cldr.Unit.new!(2, "curr-usd-per-meter-per-second"))
{:ok, "$2.00 per meter per second"}
1
kip
ex_cldr Core Team
Thanks for trying it out! The reason the first example works, and the second doesn’t parse, is that meter-per-second is a built-in unit and mile-per-gallon isn’t. The current implementation doesn’t support multiple instances of per in a compound unit definition. But … now I’ll look to making that work.
You can certainly do some crazy stuff, like:
iex> u = Cldr.Unit.new!(2, "curr-usd-foot-pound-per-ampere-light-year")
#Cldr.Unit<"USD_pound_foot_per_light_year_ampere", 2>
iex> MyApp.Cldr.Unit.to_string u
{:ok, "$2.00 pound-feet per light year-ampere"}
iex> MyApp.Cldr.Unit.to_string u, style: :short, fractional_digits: 0
{:ok, "$2 lb⋅ft/ly⋅A"}
4
kip
ex_cldr Core Team
I’ve published ex_cldr_units version 3.9.1 that now supports multiple -per- phrases in a unit definition.
iex> u = Cldr.Unit.new!(2, "curr-usd-per-mile-per-gallon")
#Cldr.Unit<"curr_usd_per_gallon_mile", 2>
But you’ll notice that this resolves to curr_usd_per_gallon_mile since per the specification:
- Multiplication binds more tightly than division, so kilogram-meter-per-second-ampere is interpreted as (kg ⋅ m) / (s ⋅ a).
- Thus if -per- occurs multiple times, each occurrence after the first is equivalent to a multiplication:
- kilogram-meter-per-second-ampere ⩧ kilogram-meter-per-second-per-ampere.
And then the canonical unit sort order is applied. When output as a string:
iex> MyApp.Cldr.Unit.to_string u
{:ok, "$2.00 per gallon-mile"}
Which is logically consistent but maybe not what you expect. That why there are pre-defined units like meter-per-second. Unfortunately, It is predefined but as a compound conversion, which results in the name being resolved as above. mile-per-gallon is not currently predefinedmeter-per-second is predefined with a single conversion factor which is why its name remains unchanged.
Thanks for the gentle push @slouchpie to get this fixed, even if it doesn’t necessarily serve your purpose.
2
slouchpie
I didn’t even have a purpose! 
I was just playing with it. For some reason, mile_per_gallon was returned by known_units in one of my projects so I tried it.
I think the solution is perfect though. Multiple “per” is, ultimately, linguistically ambiguous.
Example: x per y per z can be either x / (y /z) = xz / y or (x/y) / z = x /yz .
So even an implementation that rigorously adheres to the specification will be controversial for somebody’s use case.
1
kip
ex_cldr Core Team
Today ex_cldr hit one million downloads on hex.pm.
I wrote the first message on this topic in 2018 so it’s been a while. And maybe it’s three enthusiasts with a non-caching CI/CD pipeline only 
But it’s the most popular (by downloads) OSS library I’ve ever written. And because it’s Elixir it’s still fun!
Thanks to everyone who has used it, tried it, reported an issue or found a bug. The support and encouragement is much appreciated.
For early 2022 I’ll be launching:
- Chinese, Japanese and Korean Lunisolar calendars
- Updated ICU Message formatter with Gettext integration (due primarily to the work of @maennchen)
And then in the background I’m still working on transformations and collations.
Here’s to an exciting 2022!
14
elimark1
You can explore the latest version of Mezzofanti here: https://github.com/tmbb/mezzofanti
Currently it supports only a gettext backend (that it, it supports translations supplied as gettext files, even though the messages themselves use the ICU message format).
Currently I don’t have much use for this, but I might ad dit in the future.
My rationale is that it’s not possible to translate a string that isn’t a static string marked as translatable in the source, so all translatable strings should be wrapped ina ,macro and available at compile-time.
This is different from gettext, where one can feed it dynamic strings, which will be translated according to the .po files.
This is a requirement for packages like ecto, which can’t access the dependent application’s Gettext backend.
However, because my Mezzofanti library is able to translate strings in dependencies (this was actually the main design criterion for my library), being able to handle dynamic strings is not as useful.
That is what I’m doing now. I’m currently embedding these lists in the code, but I can add them to a persistent term with little work and probably no relevant performance impact.
I’m not doing this right now because I’m trying to reduce the amount of moving parts, but I have plans for adding it in the future.
I’m supplying the locale through the process dictionary.
This module illustrates a some interesting features (explained in the comments):
defmodule Mezzofanti.Fixtures.ExampleModule do
use Mezzofanti
# Not that I don't need to require or impor a Mezzofanti backend here.
# I just use the Mezzofanti library, and once a backend is configured
# it will automatically become aware of these messages
# (even if the messages exist in a different application)
def f() do
# A simple static translation
translate("Hello world!")
end
def g(guest) do
# A translation with a variable.
# This translation contains a context, possibly to disambiguate it
# from a similar string which should be translated in a different way.
# Mezzofanti will keep equal strings with different contexts separate.
translate("Hello {guest}!", context: "a message", variables: [guest: guest])
end
def h(user, nr_photos) do
# A more complex translation with two variables and plural forms.
# It also defines a different domain.
translate("""
{nr_photos, plural,
=0 {{user} didn't take any photos.}
=1 {{user} took one photo.}
other {{user} took # photos.}}\
""",
domain: "photos",
variables: [
user: user,
nr_photos: nr_photos
])
end
end
The following commands will create a priv/mezofanti/ dir for the locale data:
mix mezzofanti.extract
mix mezzofanti.new_locale pt-PT
mix mezzofanti.merge
This will result in:
priv/
mezzofanti/
pt-PT/
LC_MESSAGES/
default.po
photos.po
default.pot
photos.pot
The Mezzofanti and Cldr backends should be created and added to the apps’ config.
Mezzofanti expects a single global Mezzofanti backend, so that you can centralize the translations of several applications/dependencies in the same place (this is a quastionable choice, but again, it was the main design goal of this library).
Like gettext, Mezzofanti reads the locale from the process dictionary.
There are Mezzofanti.set_locale() and Mezzofanti.get_locale() for that.
And also my favourite function for testing, the function Mezzofanti.with_locale(locale, func), which sets the locale, executes the func and then resets the old locale.
For example:
iex> alias Mezzofanti.Fixtures.ExampleModule
Mezzofanti.Fixtures.ExampleModule
iex> ExampleModule.f()
["Hello world!"]
iex> ExampleModule.f() |> to_string()
"Hello world!"
iex> ExampleModule.g("tmbb") |> to_string()
"Hello tmbb!"
iex> ExampleModule.h("kip", 2) |> to_string()
"kip took 2 photos."
iex> ExampleModule.h("kip", 1) |> to_string()
"kip took one photo."
iex> ExampleModule.h("kip", 0) |> to_string()
"kip didn't take any photos."
iex> Mezzofanti.with_locale("pt-PT", fn -> ExampleModule.h("kip", 0) |> to_string() end)
"kip não tirou fotografias nenhumas."
iex> Mezzofanti.with_locale("pt-PT", fn -> ExampleModule.h("kip", 1) |> to_string() end)
"kip tirou uma fotografia"
iex> Mezzofanti.with_locale("pt-PT", fn -> ExampleModule.h("kip", 2) |> to_string() end)
"kip tirou 2 fotografias"
Mezzofanti also contains two “fake” locales for pseudoloalization.
Pseudolocalization is a process of automatically replacing latin characters by similar foreign characters (like characters that are visually similar or contain diacritics not present in English) and extending words with extra characters so that they ar longer than the originals.
This produces still readable strings, helps detect places where strings in your application haven’t been translated and helps detect places where you don’t have space for longer strings.
The “pseudo” locale provides pseudolocalization for simple text:
iex> Mezzofanti.with_locale("pseudo", fn -> ExampleModule.h("kip", 2) |> to_string() end)
"ǩıƥ~ ťøøǩ~ 2 ƥȟøťøš~~."
iex> Mezzofanti.with_locale("pseudo", fn -> ExampleModule.h("kip", 1) |> to_string() end)
"ǩıƥ~ ťøøǩ~ øñê~ ƥȟøťø~."
The “pseudo_html” provides pseudolocalization for HTML text. It preserves HTML tags and HTML entities, while localizing the rest of the text.
In the first line below, you can see how “normal” pseudolocalization is not sufficient for HTML:
iex> Mezzofanti.with_locale("pseudo", fn -> ExampleModule.i() |> to_string() end)
"Ťȟıš~ ɱêššàğê~~ ċøñťàıñš~~ <šťȓøñğ>ȟťɱĺ~~~~ ťàğš</šťȓøñğ>~~~~ &àɱƥ~; ñàšťÿ~ šťüƒƒ~..."
iex> Mezzofanti.with_locale("pseudo_html", fn -> ExampleModule.i() |> to_string() end)
"Ťȟıš~ ɱêššàğê~~ ċøñťàıñš~~ <strong>ȟťɱĺ~ ťàğš~</strong> & ñàšťÿ~ šťüƒƒ~..."
0