I have published ex_cldr version 2.29.0 that is primarily driven by the extraction of the Cldr.Plug.* modules into their own library. That library, ex_cldr_plugs has also been published. This re-packaging will require a configuration change to the dependencies of applications that use Cldr.Plug.* modules. See the Migration section below.
Migration
The plugs Cldr.Plug.SetLocale, Cldr.Plug.AcceptLanguage and Cldr.Plug.PutSession have been extracted to their own library, ex_cldr_plug. Therefore adding {:ex_cldr_plug, "~> 1.0"} to the deps of any application using these plugs is required.
Bug Fixes
Fixes resolving the RBNF locale name for locales that inherit the RBNF locale from a parent. This is true for at least the ânbâ locale which in previous releases had its own RBNF locale data but now inherits from ânoâ. Thanks to @juanperi for the report. Closes #175.
Country is a political term, not a geographic one. And there are around 105 disputed territories around the world.
Therefore I recommend the CLDR practise of using the terms âRegionâ or âTerritoryâ instead of âCountryâ in applications.
I also recommended against using national flags to indicate language preferences (its fine to associate a flag with a territory) for a similar reason.
In IOS 16, Apple is also removing flags from its input menus, following (or so it appears) the same underlying principle.
I was asked during the talk if there was a recommended alternative. Apart from using the locale name itself, I havenât found a more visual and compelling means to communicate language selection.
Iâd be most interested in any ideas the community might have.
ex_cldr version 2.33.0 is published to support Elixir 1.14. All deprecation warnings are removed. As a result, support for Elixir 1.10 is discontinued. Elixir versions 1.11 through Elixir 1.14 are supported.
There was one issue with Elixir 1.14 causes a compilation error. The error was in generated code that uses the function Cldr.Math.mod/2 which in earlier releases did not need to be specifically imported. For some reason in Elixir 1.14 it does need to be imported. Thankfully the change required for Elixir 1.14 also works perfectly well on Elixir 1.11 and later.
It is recommended to also update gettext to version 0.20.0 as well since that also removes Elixir 1.14 deprecation warnings.
The changelog entry for ex_cldr 2.33.0 is:
Bug Fixes
Fixed a bug whereby Cldr would not compile due to a change to module alias resolution in generated code. The root cause of the change has not been established but the fix, by specifically importing Cldr.Math, works on Elixir 1.14 and earlier releases back to Elixir 1.11.
Enhancements
Removes warnings for Elixir 1.14. As a result ex_cldr now supported with Elixir 1.11 and later only (support for Elixir 1.10 has been discontinued).
Allow either ratio 2.x or ratio 3.x depdendencies to be configured. This library is only used during the development of ex_cldr and is not normally used by library consumers. However ex_cldr_units does use ratio so this flexibility helps downstream maintenance, especially when ratio is updated to avoid Elixir 1.14 deprecation warnings.
Thanks @kip! I took a look at why things break, and this is the root cause:
We changed how we build the import metadata. It is probably best to avoid setting metadata by hand. My suggestion would be to expand the code above to the remote form:
Using flags is much more practical to scan the options visually, so I would still use them. IMHO the best solution would use geolocalisation to determine which flag to show for each language, e.g. an American would see the US flag for English whereas someone from England would see the UK flag, etc. There will always be people complaining though, as with everything.
I understand your perspective. But as someone who travels quite a lot I really dislike geolocalisation. Where I am currently located is way down the list of signals that express my preferences.
I agree that a visual cue is really helpful and pleasing. But flags arenât very respectful of many situations:
Iâm in Switzerland which has four national languages. Based upon my location, what would you show? If Iâm in Zurich then it may be that my normal daily language is âSwiss Germanâ (ICU locale gsw_CH) which is a dialect of high German, but quite different too. I donât think I would expect a German flag to represent my preference.
Iâm in Taiwan. Not very respectful for people in Taiwan to show the flag of the PRC.
Iâm in HK, part of the PRC. But I speak Cantonese and use Traditional Chinese characters. Which flag am I showing?
There are lots more examples, these are just a couple. I donât think its a matter of complaining so much as being respectful of the preferences your users want to express.
That might work for the single language multiple countries case, but what about the single country, multiple languages case? E.g. in Switzerland there four official state languages (German, French, Italian, Romansh). Would you consider the swiss flag an appropiate flag to represent the language of Romansh? No other country has more Romansh speaking people, but itâs also not the most commonly used language in Switzerland.
People who travel a lot are a small, small percentage of the population so unless your website is targeting them especially I wouldnât say itâs worth giving the majority a subpar experience.
As for the examples, thatâs the beauty about geolocalisation:
letâs say the Swiss donât like to use the French/German/Italian flags, you could display the locales names instead of a flag just for them. Maybe they do like seeing those flags and recognize their practicality, in which case we can use those 3 flags and a regional flag for Romansh.
in Taiwan, why not show the Taiwanese flag? Otherwise Iâm sure they have some historic or regional flag which would do well.
I donât know the specifics for HK, but you could have a look at how they handle things in their websites and do something similar (maybe there is a Cantonese flag even!).
@LostKobrakai again thatâs the beauty of geolocalisation. It solves 90% of the edge cases, and for the rest itâs always possible to fall back to the locales names. I like this much better than removing flags for everyone.
In the Romansh case, maybe this flag would do : https://www.eurominority.eu/index.php/en/romansh-people/ (we donât have to restrict ourselves to country flags). The idea is that by assuming the user origin (and thus culture even though we will miss sometimes), we can find the most culturally appropriate way to signify the linguistic options that are available.
Given the number of specialisations youâre suggesting could be required to reflect local customs and cultures, it still feels to me that simply using the language/locale name in the script of that locale is the most consistent, âcorrectâ and straight forward. And given Unicodeâs most recent stance on new emoji flags, a strategy on regional flags will be more complex to implement too.
Nevetheless I fully expect that the use of national flags to represent languages and locales will be popular for the very reasons you mention.
Interesting link. I can understand how much of a pain it would be to implement all the ever-changing local flags. It would have been good to have a descriptive way to define flags instead, similar to what has been in use in heraldry for many centuries, but it would go against our modern definition of âpixel-perfectâ design. Itâs a shame because emojis themselves use similar combinations and are not supposed to be rendered the same in different versions nor devices.
An update on whatâs next for ex_cldr and friends
A very occasional update on whatâs coming next for ex_cldr and related libraries.
In October 2022 (Proleptic Gregorian)
CLDR 42 is in beta and due for release in October. In line with usual practise, updated ex_cldr and related libraries will be available on the day of release. As usual, some formatted results may chance between releases but the API and semantics do not change.
Update ex_cldr_routes to support Phoenix 1.7 and to introduce a new sigil to localise the new verified_routes capability. Probably going to be sigil_q (q being after p) - alternative suggestions welcome!
By Year End 2022
Introduce ex_cldr_person_names. With CLDR 42 comes data to support person name formatting. ex_cldr_person_names will leverage that data to provide person name formatting. Input on user requirements are especially welcome.
By March 2023
Implement the Unicode Message Format version 2.0 as a new library. ex_cldr_messages implements the ICU Message Format and thanks to great work by @maennchen itâs fully integrated with Elixir gettext making for a seamless developer experience for what I believe is a better message format. The new Message Format 2.0 syntax is designed to be more widely adopted and address known issues with the 1.0 format. All the heavy-weight players from industry have been part of formulating this specification.
In Calendar Year 2023
Collaborate on writing book about localizing applications. There is very little literature on this topic let alone practical guides.
Implement some Liveview components on Phoenix 1.7+. Input, thoughts, ideas, suggestions very welcome. My initial list:
Locale selector. A panel to select an available locale and customize it (calendar, currency, etc etc)
Calendar picker. There are some great javascript calendars and they should be the preferred UI mechanism. But in the liveview world there is room for a calendar too. And, well, I just like calendars. This implementation will work on any calendar that can be defined with the Cldr.Calendar protocol and be completely localized so it will have different capabilities to the normal JS ones.
Implement an interactive notepad like numpad. ex_cldr and friends have a lot of inbuilt knowledge of numbers, dates, times, units of measure and it would be fun to wrap this in a notebook style interface. Iâm hoping this can be done on top of Livebook - I donât think there is a way to override the evaluator loop but maybe there is a way to put this all together. Definitely interested in ideas, feedback and suggestions.
One thing that could be extremely useful is some kind of guideline on storing names. Iâm likely not the only one guilty here of :first_name/:last_name columns, but what would be really amazing would be something similar to what youâve done with Money.Ecto: some kind of composite name storage that takes localization into account and that can be passed to display functions like short_name or formal_name (spitballing) when rendering in various circumstances.
Yep, definitely not as simple as I made it out to be! Another potential blocker/challenge: insertion/parsing, which is explicitly outside the scope of the CLDR release. Weâd need some way to get name input into the proper shape, and as the document pointed out, thereâs a huge amount of ambiguity there (unless you are going to offer a ton of fields which will likely have the effect of confusing many people.)
Perhaps something else to think about along those lines is using locale/preferences to drive name input.
Edit: again re: confusion, but I wonder if a + icon next to given/surname to add a given2 and surname2 would be a âdonât let the perfect be the enemy of the goodâ kind of solution that gets 95% of the way there.
Edit2: a final note, from the document: How names get placed into fields to be formatted is beyond the scope of CLDR PersonName formats; this document just lays out the assumptions the formatting code makes when formatting the names. Given this, I think itâs unreasonable to expect ExCLDR to address this directly, but helpers/etc. that make thoughtful name input possible to implement could be very useful. E.g. an easy way to get the âstandard partsâ and âstandard orderâ of a name from a given locale, so that the appropriate number of fields could be rendered in the expected order.
Incorporates all the updated locale data for a total of 586 locales available
A new -u extension key to the language tag is added to provide a preferred unit of measurement for temperature: Celsius, Fahrenheit, and Kelvin. (An effort has also been started to provide syntax for other unit preferences in future releases.)
Two new number systems are available, corresponding to new Unicode 15.0 scripts: Kawi and Nag Mundari.
A new short timezone ID is available, tz-uaiev, for Europe/Kyiv
Some locales now have higher coverage levels: Igbo (ig), yo (Yoruba), Chuvash (cv), Xhosa (xh), Haryanvi (bgc), Bhojpuri (bho), Rajasthani (raj), Tigrinya (ti)
Data normalization: There was an extensive normalization of different kinds of spaces (normal, non-breaking, thin, etc.) for consistency of behavior - CLDR-14032
Plural rules Additions: Added âmanyâ category for Asturian, Catalan. They only affect messages with large numbers. Maltese now has the âtwoâ category.
Plural Rules Removals: The âmanyâ plural category for Hebrew (CLDR-14634) was removed; it is unnecessary in modern practice.
Plural Rules Changes: There were a few changes to the rules that affect how numbers are assigned to categories.
Formatting Person Names: Added data and structure for formatting peopleâs names. For more information on why this feature is being added and what it does, see the background document. A new library, ex_cldr_person_names will be developed to expose and exploit this data by year end 2022.
Two new values for pattern elements used for currency format elements:
alpha_next_to_number: A pattern to use when the currency symbol would result in letter characters being adjacent to the numeric value; typically this adds a no-break space between the currency symbol and numeric value, if the standard currency format pattern does not already have a space. This provides an improved alternative to the currency spacing patterns.
no_currency: A pattern to use when currency-style formats are desired but without the actual symbol (as in a table of currency values all fo the same currency).
For the currency formats element, a new element currency_pattern_append_ISO containing a pattern that shows how to append an ISO currency symbol (€€) to a currency pattern using a standard currency symbol (€); this is needed for certain types of currency display.
Although these data changes are incorporated, the formatting functions do not expose them. They will be added in an update to ex_cldr_numbers by year end 2022.
The new -u extension key to the language tag is added to provide a preferred unit of measurement for temperature: Celsius, Fahrenheit, and Kelvin. This extension is parsed but is not yet utilised by ex_cldr_units. It will be applied in an update before year end 2022.
The length of a light-year has been adjusted to the IAU value (which uses a Julian year of 365.25 days).
The unit ID for metric-tonne has been deprecated in favor of tonne.
Several locales, including en , have changed the standard date/time formats replacing <date> at <time> with <date>, <time>. There is a new format type (the at type) available in the underlying data however these formats are not yet exposed in ex_cldr_dates_times. An update will be published before year end 2022 to apply this functionality.
The Yukon metazone has been un-deprecated
Day-periods are added for hi_Latn, and adjusted for mr to only have evening1
Support time zone data 2022e. For 2022e: After 2022-10-27, âAsia/Ammanâ and âAsia/Damascusâ removed from metazone âEurope_Easternâ with no replacement metazone.
With some great motivation and collaboration from @rubas, ex_cldr_numbers version 2.29.0 is published. The main new feature is the ability to provide a wrapping function to Cldr.Number.to_string/2 that makes it easy to wrap format elements. This is particularly useful if formatting for HTML and you want different parts of the format to have different classes.
It is also possible and recommended to use the Phoenix.HTML.Tag.content_tag/3 function if wrapping HTML tags since these will ensure HTML entities are correctly encoded. For example:
CLDR version 43 was released this month, triggering a release cycle for the ex_cldr family of libraries. All-in-all, 15 ex_cldr libraries have been updated, polished and published as part of this release cycle. Please consult the relevant changelogs for more information.
Hightlights
ex_cldr data is now versioned. This means that when installing locales, ex_cldr does a better job of detecting if there is any existing locale data and if that data is stale, the updated data will be downloaded and installed. This addresses a long standing issue.
ex_money_sql now has a set of helper macros for querying money amounts. This was contributed by @am-kantox - really appreciated.
All the additional calendar implementations (Ethiopian, Coptic, Persian, Lunisolar, Composite) are now all at version 1.0.0.
Lots of work on the lunisolar calendars to make the API more approachable for common requirements like determining the data of the lunar new year for a given Gregorian year. Many thanks to @nineclue for pushing me to get this library into good shape.
Adds a new calendar implementation, the Japanese calendar. This calendar is the Gregorian calendar but using the Japanese eras which count the number of years of an Emporerâs reign.
Upcoming
Despite this round of updates there are still some open issues which I know some very patient people are waiting for. With this round of updates complete I can now focus on the following over the next week:
Updating ex_cldr_routes to support localised verified routes. Highest priority.
Publishing ex_cldr_person_names which provides localised name formatting. The final two actions required are test conformance and documentation.
Improving ex_cldr_units to generalise the algebra for unit multiplication and division and to add a :wrapper option to Cldr.Unit.to_string/2 similar to the implementations for Cldr.Number.to_string/2 and Cldr.DateTime.to_string./2.
Localized Verified Routes are now available in ex_cldr_routes version 1.0 released today. Some might say, âfinallyâ!.
Localized Verified Routes uses ~q (Sigil_q). This choice is made in order to make clear when a verified route is localised or not. And the choice of q is because of its symmetry on US keyboards -q is top left, p is top right.
Examples
defmodule MyApp.Module do
# Same as Phoenix Router.VerifiedRoutes with the Cldr backend module name
# prepended. This module is generated at compile time.
# For this Cldr backend we assume the locales `:en`, `:fr` and `:de` are configured.
use MyApp.Cldr.Router.VerifiedRoutes, router: MyApp.Router, endpoint: MyApp.Endpoint
def some_function do
MyApp.Cldr.put_locale(:de)
# Returns "/users_de/de"
# The sigil is identical to the `~p` sigil except that
# the path segments `:locale`, :language` and `:territory` are
# interpolated into the generated `~p` path.
~q[/users/:locale]
end
end
How does it work?
Sigil_q generates a case expression with a clause for each locale configured on the Cldr backend. For the example above, the sigil expands to the following code:
# ~q"/users/:locale"
case MyApp.Cldr.get_locale().cldr_locale_name do
:de -> ~p"/users_de/de"
:en -> ~p"/users/en"
:fr -> ~p"/users_fr"
end
Thanks @kip for doing whatâs probably a pretty thankless job (maintaining an important library)! I have nothing but admiration for folks who give so unselfishly to the benefit of the rest of the community!
