Seeking your thoughts on Routex - an Extension Framework for Phoenix Router

UPFRONT

  • This post is seeking your thoughts, gather ideas and spark discussion. The lib is currently in Proof of Concept and not yet published.
  • This opening posts is updated based on responses below so it might look like participants did not read it.

Routex

TLDR; let’s split up Phoenix Localized Routes - Localized/multilingual routes in Phoenix so developers can pick what features they need instead of importing one ‘do-it-all’, or write features themselves in a few lines of code.

Routex is a framework to extend the functionality of Phoenix Frameworks’ router. Using a pluggable extension system it can transform, extend and generate code based on routes in your Phoenix Framework app.

It ships with a small set of extensions which cover common use cases. Routex provides helper functions to ease writing your own extensions when the default set is not enough.

Top Features and Benefits

  • adding router features is plug-and-play
  • combine extensions for a multiplier effect
  • write powerful extensions with minimal code, no plumbing required
  • supports multiple configurations in one router
  • extension callbacks have separate responsibilities and a unified interface; they can be individually tested.

Routex.Router

__using__
Macro callback that plugs Routex.Processing between route definition and compilation of the router module.

preprocess_using(routex_backend, opts \ [], list)
Wraps each enclosed route in a scope, marking it for preprocessing by Routex using given routex_backend (a configuration module). opts can be used to partially override values provided by the Routex Backend.

Routex.Processing

This module provides everything needed to process Phoenix routes. It executes the configure callback from extensions to build a configuration to use, transform callbacks to transform Phoenix.Router.Route structs and create_helpers callbacks to create one unified Helper module (see chapter Extensions for details about the callbacks)

Powerful but thin
Although Routext is able to influence the routes in Phoenix applications in profound ways, the framework and it’s extensions are a suprisingly lightweight piece of compile-time middleware. This is made possible by the way router modules are pre-processed by Phoenix.Router itself.

Prior to compilation of a router module, Phoenix Router registers all routes defined in the router module using the attribute @phoenix_routes. Each route is at that moment a Phoenix.Router.Route struct.

Any route enclosed in a preprocesss_using block has received a :private field in which Routex has put which Routex backend to use for preprocessing that particular route. By enumerating the routes, we can process each route using the properties of this backend and set struct values accordingly. This processing is nothing more than (re)mapping the Route structs’ values.

After the processing by Routex is finished, the @phoenix_routes attribute in the router is erased and re-populated with the list of mapped Phoenix.Router.Route structs.

Once the router module enters the compilation stage, Routex is already out of the picture and
route code generation is performed by Phoenix Router as usual.

Routex Extensions

Routex Extensions extend the functionality provided by Routex to transform routes or generate new route based helper functions. Each extension is a module which adopts the Routex.Extension specification. It has to implement one or multiple public functions:

  • configure/2
  • transform/3
  • create_helpers/3

Routex will call those public functions at different stages before Routex hands off the result to Phoenix.Router for compilation.

Stage 1: Configure

This stage enables extensions to preprocess backend options upfront and enables Routex to collect all resulting options in one list. As a result all known options are passed to all extension in subsequent stages.

The configure/2 callback is called with the options provided to Routex and the name of the Routex Backend. It is expected to return a new list of options.

Stage 2: Transform

This stage is meant to adapt the property values in Phoenix.Router.Route structs. The routes are grouped by Routex Backend and processed per group, allowing an extension to use accumulating values within one iteration.

The transform/3 callback is called with a list of routes, the name of the Routex Backend and the current environment. It is expected to return a list of Phoenix.Router.Route structs.

Flattening option values

Extensions can make use of option values provided by Routex itself, Routex Backends and other extensions.

To make the availability of options as predictable as possible, Routex uses a flat structure which is stored in a routes’ private.routex field. However, using a flat structure might conflict with developer experience; sometimes a nested structure to provide configuration options might be more suitable.

One responsibility of the transform/3 callback is to flatten the structure of options the extension uses for each route they receive, so other extensions can use options set by the current extension without knowledge of the initial configuration structure.

Example
The Alternate extension uses nested options and allows inheritance of properties from parent scopes. Notice how the “gb” scope has no :locale set.

scopes: %{
  "/" =>
    helper: nil,
    locale: "en",
    scopes: %{
      "nl" => %{
          helper: "nl",
          locale: "nl"
        },
      "gb" => %{
        helper: "gb", 
        }
    }
}

The Alternate extension is therefor responsible for flattening those options for (itself and) other extensions to use. To take the route responsible for the “gb” scope as an example, the extension should add flattened options in the Route struct. It can do so using a helper function.

put_routex_opts([locale: "en", helper: "gb"])

Now the Translation extension can search for the option :locale in the route’s opts, unaware of how that locale was initially configured and by which extension. (swapping the Alternate Extension with CLDR Route Extension should not matter as long as both provide the ‘:locale’ option)

Stage 3: Create helpers

In this stage helper functions can be generated which will be added to MyAppWeb.Router.RoutexHelpers.

The create_helpers/3 callback is called with a list of routes belonging to a configuration module, the name of the configuration module and the current environment. It is expected to return Elixir AST.

As a result the developer only has to import MyAppWeb.Router.RoutexHelpers for all helpers generated by extensions to be included in the app.

Current Status

  • Routex has almost feature parity with Phoenix Localized Routes, only one or two extensions yet to be written. Minor modifications might be needed to the backend module / configuration but I consider it a drop-in replacement already (Routex` Assigns extension supports custom namespaces, so I chose @loc for the Example app)
  • Main modules and their documentation are good enough for alpha release.
  • Helper functions to aid writing extensions are good enough for alpha release
  • Initial set of extensions is POC quality and documentation has to be written
  • Messages printed by extensions have to be deferred to after processing
  • Example app uses Routex backend named LocalizedRoutes.
  • The Example app compiles and works as expected
==> example
Compiling 13 files (.ex)
Completed: ExampleWeb.LocalizedRoutes ⇒ Routex.Extension.Expansion.configure/2
Completed: ExampleWeb.LocalizedRoutes ⇒ Routex.Extension.Translations.configure/2
Completed: ExampleWeb.LocalizedRoutes ⇒ Routex.Extension.VerifiedRoutes.configure/2

20:56:34.477 [info]
The default sigil used by Phoenix Verified Routes is overridden by Routex due to the configuration in `ExampleWeb.LocalizedRoutes`.

      ~p: localizes and verifies routes. (override)
      ~o: only verifies routes. (original)

Documentation: https://hexdocs.pm/routex/extensions/verified_routes.html

Completed: ExampleWeb.LocalizedRoutes ⇒ Routex.Extension.Expansion.transform/3
Completed: ExampleWeb.LocalizedRoutes ⇒ Routex.Extension.Translations.transform/3
Completed: ExampleWeb.LocalizedRoutes ⇒ Routex.Extension.Assigns.transform/3
Completed: ExampleWeb.LocalizedRoutes ⇒ Routex.Extension.Translations.create_helpers/3
Completed: ExampleWeb.LocalizedRoutes ⇒ Routex.Extension.ScopeGetters.create_helpers/3
Completed: ExampleWeb.LocalizedRoutes ⇒ Routex.Extension.Alternatives.create_helpers/3
Completed: ExampleWeb.LocalizedRoutes ⇒ Routex.Extension.VerifiedRoutes.create_helpers/3
Completed: Create or update helper module ExampleWeb.Router.RoutexHelpers

[info] Running ExampleWeb.Endpoint with cowboy 2.9.0 at 127.0.0.1:4000 (http)
[info] Access ExampleWeb.Endpoint at http://localhost:4000

#NOTE: This function returns all values known by Routex; only the values you have chosen are added to 'assigns' the Socket. Hence the duplication.

iex(1)> ExampleWeb.Router.RoutexHelpers.path_values("/europe/nl/producten/3")
%{
  assigns: %{
    loc: %{
      contact: "verkoop@example.nl",
      locale: "nl",
      name: "The Netherlands",
      scope_helper: "europe_nl"
    }
  },
  backend: ExampleWeb.LocalizedRoutes,
  contact: "verkoop@example.nl",
  locale: "nl",
  name: "The Netherlands",
  scope_alias: :europe_nl,
  scope_helper: "europe_nl",
  scope_path: ["europe", "nl"],
  scope_prefix: "/europe/nl"
}
3 Likes

I don’t quite understand what this does and the example doesn’t help very much. Is this some kind of middleware between the router and the controllers or I am missing something?

Thanks for your reply.

For a more ‘real world use case’ have a look at Phoenix Localized Routes - Localized/multilingual routes in Phoenix. That libs is ‘one blob’ which does most things the extensions listed above do. While refactoring it, it’s features became almost extensions so Routex was born as Proof of Concept.

The new approach would

  • allow developers to only include the extensions (and thus code) for features they need. Less code == less risk.
  • or write their own router feature (such as Cloak) with a few lines for code. No need for a heavy fork, dealing with macro’s and other plumbing.

Routex, or better: the extensions, are mainly about transforming routes (/foo → /foo, /en/foo, /nl/foo) and adding helper functions for in components and controllers (get_other_language_page_urls(@url).

Routes itself provides a very small lifecycle hook for LiveView which can (and is) used by extensions that need to combine compile time information with runtime information. But that’s a ‘behind the scene’ thing.

From what I understand this is done at compile-time and changes the macros/functions generated by router?

All ‘original’ code generated by Phoenix is preserved by Routex, but you do get the gist of it :slight_smile:. That it all is done at compile-time is 100% accurate!

I like the idea, my only concern is that it fights with router ideology. You define routes and expect them to be immutable, however routex can modify them, making the routes not the source of truth.

Although I see your point, Phoenix Router itself is no different in that it is not the most complete ‘source of truth’

  • it generates Plug.Router routes behind the scene (although with the same path)
  • ‘scopes’ segment routes in two or more parts. So ‘get /products’ might exists in code but not as URL.
  • a ‘live’ route creates a ‘get’ route (piled with hidden parameters)
  • a ‘resource’ route expands to a handful routes with different methods, paths and params
  • a /websocket is magically added
  • nitpicking: interpolation like :id and :token break ‘truth’ as those are replaced with arbitrary values

‘mix phx.routes’ comes closer and is the same for Phoenix Router and Routex, as Routex will generate Phoenix.Router.Route structs which Phoenix will compile to actual code[1]. So from Phoenix perspective the routes are just the same quality as the routes in router.ex

Routex generates more static routes such as /europe/nl/producten/:id instead of /:continent/:country/products/:id. So as far as immutability goes it’s actually clearer. You can exactly see which routes exist in the routes list.

——

  1. this is an improvement over how Localized Routes works nowadays.
  2. Thans for diving deeper and asking. Such questions should be answered by README/documentation; so they are very valuable.
2 Likes

I’ve had to localize a project myself and it is true, there is a lot of manual handling you have to do both at routes and in liveviews, however since I didn’t have that many liveviews, doing it without a tool was fast enough.

In general it seems like a great idea, I would even think that addition of this feature in phoenix itself will be a big plus, so I am excited to see the result!

Rewritten opening post as documentation for Routex was written.

The renewed Opening Post should make clear how Routex fits in. Please let me know when it’s still not clear to you; writing clear documentation is not (yet) my stronghold.

You feedback has already inspired me to extend the documentation of the POC, leading to valuable insights about terminology (configuration module → Routex Backend, extend_using → preprocess_using etc)

Update

  • Main modules and their documentation are good enough for first release.
  • Helper functions to aid writing extensions and their documentation are good enough for first release.
  • Initial set of extensions is POC quality and documentation has to be written.
  • (minor) Messages printed by extensions have to be deferred to after processing.
  • Needs more tests / moving extension tests to the extension folders

If someone is willing to help I am happy to provide access to the private repository :wink:

@BartOtten well since Phoenix 1.7 is now out I have to get off my backside and finish up ex_cldr_routes to include localised VerifiedRoutes.

So I’m in for some collaboration as you proposed - and see if we can take your new library and integrate it.

4 Likes