Extract gettext translations inside macro

Questions / Help
1

Hi, I’m working on DSL and the last thing that I don’t like is how I’m extracting translations.

I have 2 version of code:

  1. My current implementation is based on Gettext.Extractor private API - it’s working without any issues

  2. An alternative is to use Gettext.Macros.dpgettext_noop_with_backend/4. It’s also a solution which works very well, but unfortunately it does not allow function generators (other macros, for loops etc.) as Gettext macros requires binary literals.

defmodule MyApp.Example do
  use MyLib.DSL, gettext_backend: MyApp.GettextBackend

  for data <- ~w[first second third] do
    some_dsl data
  end
end

I understand that private API is unstable and can be changed at any time, but I have simply no idea about other possible solutions. From now on I can see only 2 ways to fix it:

  1. Propose to make Gettext.Extractor a public API - that’s the simplest thing that could be done as it’s only about documenting code i.e. no change in code.

  2. Propose to support non-literals, so internal Gettext.Extractor calls are done within quote do … end block instead of directly inside macros.

Both solutions are rather something you consider at the very end. Support for gettext is only optional, so I don’t want to resign from function generators. The whole logic is done only in compile-time, so the generated code is as fast as possible in runtime.

With all above in mind I’m not satisfied in both implementations and I have no idea what to do with that. Did you had a similar problems with gettext when working on DSL? What do you think about making Gettext.Extractor public? Is there any other way to extract translations without above problems?

2

I’d love if Gettext.Extractor could become public api. I had asked for that a few months back on a more esoteric usecase and given the usecase the response was negative. This sounds more in line with the libraries goals. But I wonder why you cannot provide binary literals if you’re in a macro context.

1 Like
3

Glad to hear that not only me is looking for this.

This is about said data argument in some_dsl macro, see:

4

Ah, yeah. You’d cannot work with variables, you need to know the actual strings in your macro.

5

I believe you should be able to use the exposed public APIs in Macros since you can just unquote the variable containing the string.

require Gettext.Macros

for message <- ["message 1", "message 2"] do
  def translate(unquote(message)) do
    Gettext.Macros.dgettext_with_backend(
      Acme.Gettext,
      "default",
      unquote(message)
    )
  end
end

See Gettext.Macros — gettext v0.26.2

6

I have mentioned it as alternative way in 2nd point and why it’s not an option for me. For now I’m investigating expo as generating pot file sounds like an interesting alternative to current solutions.

7

Generally when using macros, you should have access to the literal and not just the variable. Can you share a simplified version of your DSL that shows the issue?

It’s the intention of gettext that this works. If it does not, we’ll need to have a closer look.

8

I’m not sure I fully understand your use case but in a project I work on we extract translations from the .POT file. Seems to work fine, we hand rolled a parser of those files but I think Expo.PO.parse_file!("priv/gettext/default.pot") probably works fine

9

Here you go:

defmodule DummyGettext do
  defmacro dummy_macro(input, _lang \\ "en") do
    # gettext macros require literals at this point (not within quote do … end block)
    IO.inspect(input)
    is_binary(input) == false && raise "gettext fail here"
  end

  def get_locales, do: ["en"]
end

defmodule MyLib.DSL do
  defmacro __using__(_opts \\ []) do
    quote do
      import MyLib.DSL

      require DummyGettext

      def translated_input(input, locale \\ "en")
    end
  end

  defmacro some_dsl(input) do
    for locale <- DummyGettext.get_locales() do
      translated_input = quote do: DummyGettext.dummy_macro(unquote(input))

      quote bind_quoted: [input: input, locale: locale, translated_input: translated_input] do
        def translated_input(unquote(input), unquote(locale)), do: unquote(translated_input)
      end
    end
  end
end

defmodule MyApp.Example do
  use MyLib.DSL

  # this one obviously works
  some_dsl("text to translate")

  # my macros are fine with that, but gettext requires literal at compile time
  # so below code fails
  for data <- ~w[first second third] do
    some_dsl(data)
  end
end
10

Wrong way. I think now about generating .pot file at compile time and then call mix gettext.merge priv/gettext --locale locale. It’s nothing related to parsing .pot file. Sorry if my post was not clear enough.

11

Generating a .pot file using expo is definitely the best solution. I have a full control over code, I don’t need to worry about private APIs and it’s also a very simple thing to do.

12

This works with a slight change (needs a quote / unquote cycle, not sure why):

defmodule DummyGettext do
  defmacro dummy_macro(input, _lang \\ "en") do
    # gettext macros require literals at this point (not within quote do … end block)
    IO.inspect(input)
    is_binary(input) == false && raise "gettext fail here"
  end

  def get_locales, do: ["en"]
end

defmodule MyLib.DSL do
  defmacro __using__(_opts \\ []) do
    quote do
      import MyLib.DSL

      require DummyGettext

      def translated_input(input, locale \\ "en")
    end
  end

  defmacro some_dsl(input) do
    quote do
      def translated_input(unquote(input), locale) do
        DummyGettext.dummy_macro(unquote(input), locale)
      end
    end
  end
end

defmodule MyApp.Example do
  use MyLib.DSL

  # this one obviously works
  some_dsl "text to translate"

  # my macros are fine with that, but gettext requires literal at compile time
  # so below code fails
  for data <- ["first", "second", "third"] do
    some_dsl unquote(data)
  end
end

There’s no need to create a function implementation per message / locale since gettext will already do that for you.

1 Like
13

Thanks, but this is not best for 2 reasons:

  1. data is better than unquote(data) - my DSL designed to be simply, so everyone can use it without …
  2. not sure why ): :smiley:

My main issue was about extracting message and private APIs. Generating a .pot file is best in my case.

That’s why it’s a small example. It’s part of a bigger DSL designed for fast runtime execution i.e. the gettext calls would happen only in compile time and yes in my case it makes sense, a very bad … I mean good … very good one! :smiling_imp:

14

True, it’s a bit ugly. I wonder if that variable could be expanded somehow since unquote clearly also seems to be able to.

Sure, that works as well. Just make sure to either use a different domain from normal gettext or to not add the elixir-autogen flag. Otherwise you will get conflicts.

Gettext is already defining a function body for each message / locale:

Doing it again will not make it faster :slight_smile:

1 Like
15

No need to worry about that. I’m using a custom domain every time. :slight_smile:

Also I’m not using any flag, because I generate a .pot file every time the module is re-compiled, so any new change will regenerate the template (pot file). Then I let gettext handle .po files.

Good point, I did not know that … It looks very complicated, so let’s summarise it if anyone is interested:

  1. Function in Gettext module calls a generated backend function
  2. When compiling backend (use Gettext.Backend, …) @before_compile Gettext.Compiler is generated
  3. So before backend module compiles a compiler module is then generating a function from .po files

I understand that, but I still use it as I’m not returning just translated text, so I can simply translate a text at compile time or work with quoted gettext call when dealing with AST from DSL. :thinking:

1 Like