Seeking your thoughts on Routex - an Extension Framework for Phoenix Router

UPFRONT

  • This thread is an incubator thread; once Routex reaches v1.0 (soon!) it will have an new thread; this one serving for historical purposes.

Routex

TLDR; let’s split up Phoenix Localized Routes - Localized/multilingual routes in Phoenix so developers can pick what features they need instead of importing one ‘do-it-all’, or write features themselves in a few lines of code.

Routex is a framework to extend the functionality of Phoenix Frameworks’ router. Using a pluggable extension system it can transform, extend and generate code based on routes in your Phoenix Framework app.

It ships with a small set of extensions which cover common use cases. Routex provides helper functions to ease writing your own extensions when the default set is not enough.

Top Features and Benefits

  • adding router features is plug-and-play
  • combine extensions for a multiplier effect
  • write powerful extensions with minimal code, no plumbing required
  • supports multiple configurations in one router
  • extension callbacks have separate responsibilities and a unified interface; they can be individually tested.

Routex.Router

__using__
Macro callback that plugs Routex.Processing between route definition and compilation of the router module.

preprocess_using(routex_backend, opts \ [], list)
Wraps each enclosed route in a scope, marking it for preprocessing by Routex using given routex_backend (a configuration module). opts can be used to partially override values provided by the Routex Backend.

Routex.Processing

This module provides everything needed to process Phoenix routes. It executes the configure callback from extensions to build a configuration to use, transform callbacks to transform Phoenix.Router.Route structs and create_helpers callbacks to create one unified Helper module (see chapter Extensions for details about the callbacks)

Powerful but thin
Although Routext is able to influence the routes in Phoenix applications in profound ways, the framework and it’s extensions are a suprisingly lightweight piece of compile-time middleware. This is made possible by the way router modules are pre-processed by Phoenix.Router itself.

Prior to compilation of a router module, Phoenix Router registers all routes defined in the router module using the attribute @phoenix_routes. Each route is at that moment a Phoenix.Router.Route struct.

Any route enclosed in a preprocesss_using block has received a :private field in which Routex has put which Routex backend to use for preprocessing that particular route. By enumerating the routes, we can process each route using the properties of this backend and set struct values accordingly. This processing is nothing more than (re)mapping the Route structs’ values.

After the processing by Routex is finished, the @phoenix_routes attribute in the router is erased and re-populated with the list of mapped Phoenix.Router.Route structs.

Once the router module enters the compilation stage, Routex is already out of the picture and
route code generation is performed by Phoenix Router as usual.

Routex Extensions

Routex Extensions extend the functionality provided by Routex to transform routes or generate new route based helper functions. Each extension is a module which adopts the Routex.Extension specification. It has to implement one or multiple public functions:

  • configure/2
  • transform/3
  • create_helpers/3

Routex will call those public functions at different stages before Routex hands off the result to Phoenix.Router for compilation.

Stage 1: Configure

This stage enables extensions to preprocess backend options upfront and enables Routex to collect all resulting options in one list. As a result all known options are passed to all extension in subsequent stages.

The configure/2 callback is called with the options provided to Routex and the name of the Routex Backend. It is expected to return a new list of options.

Stage 2: Transform

This stage is meant to adapt the property values in Phoenix.Router.Route structs. The routes are grouped by Routex Backend and processed per group, allowing an extension to use accumulating values within one iteration.

The transform/3 callback is called with a list of routes, the name of the Routex Backend and the current environment. It is expected to return a list of Phoenix.Router.Route structs.

Flattening option values

Extensions can make use of option values provided by Routex itself, Routex Backends and other extensions.

To make the availability of options as predictable as possible, Routex uses a flat structure which is stored in a routes’ private.routex field. However, using a flat structure might conflict with developer experience; sometimes a nested structure to provide configuration options might be more suitable.

One responsibility of the transform/3 callback is to flatten the structure of options the extension uses for each route they receive, so other extensions can use options set by the current extension without knowledge of the initial configuration structure.

Example
The Alternate extension uses nested options and allows inheritance of properties from parent scopes. Notice how the “gb” scope has no :locale set.

scopes: %{
  "/" =>
    helper: nil,
    locale: "en",
    scopes: %{
      "nl" => %{
          helper: "nl",
          locale: "nl"
        },
      "gb" => %{
        helper: "gb", 
        }
    }
}

The Alternate extension is therefor responsible for flattening those options for (itself and) other extensions to use. To take the route responsible for the “gb” scope as an example, the extension should add flattened options in the Route struct. It can do so using a helper function.

put_routex_opts([locale: "en", helper: "gb"])

Now the Translation extension can search for the option :locale in the route’s opts, unaware of how that locale was initially configured and by which extension. (swapping the Alternate Extension with CLDR Route Extension should not matter as long as both provide the ‘:locale’ option)

Stage 3: Create helpers

In this stage helper functions can be generated which will be added to MyAppWeb.Router.RoutexHelpers.

The create_helpers/3 callback is called with a list of routes belonging to a configuration module, the name of the configuration module and the current environment. It is expected to return Elixir AST.

As a result the developer only has to import MyAppWeb.Router.RoutexHelpers for all helpers generated by extensions to be included in the app.

Current Status

  • Routex has almost feature parity with Phoenix Localized Routes, only one or two extensions yet to be written. Minor modifications might be needed to the backend module / configuration but I consider it a drop-in replacement already (Routex` Assigns extension supports custom namespaces, so I chose @loc for the Example app)
  • Main modules and their documentation are good enough for alpha release.
  • Helper functions to aid writing extensions are good enough for alpha release
  • Initial set of extensions is POC quality and documentation has to be written
  • Messages printed by extensions have to be deferred to after processing
  • Example app uses Routex backend named LocalizedRoutes.
  • The Example app compiles and works as expected
==> example
Compiling 13 files (.ex)
Completed: ExampleWeb.LocalizedRoutes ⇒ Routex.Extension.Expansion.configure/2
Completed: ExampleWeb.LocalizedRoutes ⇒ Routex.Extension.Translations.configure/2
Completed: ExampleWeb.LocalizedRoutes ⇒ Routex.Extension.VerifiedRoutes.configure/2

20:56:34.477 [info]
The default sigil used by Phoenix Verified Routes is overridden by Routex due to the configuration in `ExampleWeb.LocalizedRoutes`.

      ~p: localizes and verifies routes. (override)
      ~o: only verifies routes. (original)

Documentation: https://hexdocs.pm/routex/extensions/verified_routes.html

Completed: ExampleWeb.LocalizedRoutes ⇒ Routex.Extension.Expansion.transform/3
Completed: ExampleWeb.LocalizedRoutes ⇒ Routex.Extension.Translations.transform/3
Completed: ExampleWeb.LocalizedRoutes ⇒ Routex.Extension.Assigns.transform/3
Completed: ExampleWeb.LocalizedRoutes ⇒ Routex.Extension.Translations.create_helpers/3
Completed: ExampleWeb.LocalizedRoutes ⇒ Routex.Extension.ScopeGetters.create_helpers/3
Completed: ExampleWeb.LocalizedRoutes ⇒ Routex.Extension.Alternatives.create_helpers/3
Completed: ExampleWeb.LocalizedRoutes ⇒ Routex.Extension.VerifiedRoutes.create_helpers/3
Completed: Create or update helper module ExampleWeb.Router.RoutexHelpers

[info] Running ExampleWeb.Endpoint with cowboy 2.9.0 at 127.0.0.1:4000 (http)
[info] Access ExampleWeb.Endpoint at http://localhost:4000

#NOTE: This function returns all values known by Routex; only the values you have chosen are added to 'assigns' the Socket. Hence the duplication.

iex(1)> ExampleWeb.Router.RoutexHelpers.path_values("/europe/nl/producten/3")
%{
  assigns: %{
    loc: %{
      contact: "verkoop@example.nl",
      locale: "nl",
      name: "The Netherlands",
      scope_helper: "europe_nl"
    }
  },
  backend: ExampleWeb.LocalizedRoutes,
  contact: "verkoop@example.nl",
  locale: "nl",
  name: "The Netherlands",
  scope_alias: :europe_nl,
  scope_helper: "europe_nl",
  scope_path: ["europe", "nl"],
  scope_prefix: "/europe/nl"
}
3 Likes

I don’t quite understand what this does and the example doesn’t help very much. Is this some kind of middleware between the router and the controllers or I am missing something?

1 Like

Thanks for your reply.

For a more ‘real world use case’ have a look at Phoenix Localized Routes - Localized/multilingual routes in Phoenix. That libs is ‘one blob’ which does most things the extensions listed above do. While refactoring it, it’s features became almost extensions so Routex was born as Proof of Concept.

The new approach would

  • allow developers to only include the extensions (and thus code) for features they need. Less code == less risk.
  • or write their own router feature (such as Cloak) with a few lines for code. No need for a heavy fork, dealing with macro’s and other plumbing.

Routex, or better: the extensions, are mainly about transforming routes (/foo → /foo, /en/foo, /nl/foo) and adding helper functions for in components and controllers (get_other_language_page_urls(@url).

Routes itself provides a very small lifecycle hook for LiveView which can (and is) used by extensions that need to combine compile time information with runtime information. But that’s a ‘behind the scene’ thing.

From what I understand this is done at compile-time and changes the macros/functions generated by router?

All ‘original’ code generated by Phoenix is preserved by Routex, but you do get the gist of it :slight_smile:. That it all is done at compile-time is 100% accurate!

I like the idea, my only concern is that it fights with router ideology. You define routes and expect them to be immutable, however routex can modify them, making the routes not the source of truth.

Although I see your point, Phoenix Router itself is no different in that it is not the most complete ‘source of truth’

  • it generates Plug.Router routes behind the scene (although with the same path)
  • ‘scopes’ segment routes in two or more parts. So ‘get /products’ might exists in code but not as URL.
  • a ‘live’ route creates a ‘get’ route (piled with hidden parameters)
  • a ‘resource’ route expands to a handful routes with different methods, paths and params
  • a /websocket is magically added
  • nitpicking: interpolation like :id and :token break ‘truth’ as those are replaced with arbitrary values

‘mix phx.routes’ comes closer and is the same for Phoenix Router and Routex, as Routex will generate Phoenix.Router.Route structs which Phoenix will compile to actual code[1]. So from Phoenix perspective the routes are just the same quality as the routes in router.ex

Routex generates more static routes such as /europe/nl/producten/:id instead of /:continent/:country/products/:id. So as far as immutability goes it’s actually clearer. You can exactly see which routes exist in the routes list.

——

  1. this is an improvement over how Localized Routes works nowadays.
  2. Thans for diving deeper and asking. Such questions should be answered by README/documentation; so they are very valuable.
2 Likes

I’ve had to localize a project myself and it is true, there is a lot of manual handling you have to do both at routes and in liveviews, however since I didn’t have that many liveviews, doing it without a tool was fast enough.

In general it seems like a great idea, I would even think that addition of this feature in phoenix itself will be a big plus, so I am excited to see the result!

Rewritten opening post as documentation for Routex was written.

The renewed Opening Post should make clear how Routex fits in. Please let me know when it’s still not clear to you; writing clear documentation is not (yet) my stronghold.

You feedback has already inspired me to extend the documentation of the POC, leading to valuable insights about terminology (configuration module → Routex Backend, extend_using → preprocess_using etc)

Update

  • Main modules and their documentation are good enough for first release.
  • Helper functions to aid writing extensions and their documentation are good enough for first release.
  • Initial set of extensions is POC quality and documentation has to be written.
  • (minor) Messages printed by extensions have to be deferred to after processing.
  • Needs more tests / moving extension tests to the extension folders

If someone is willing to help I am happy to provide access to the private repository :wink:

@BartOtten well since Phoenix 1.7 is now out I have to get off my backside and finish up ex_cldr_routes to include localised VerifiedRoutes.

So I’m in for some collaboration as you proposed - and see if we can take your new library and integrate it.

6 Likes

A long time has passed but it was worth it! To celebrate a nice set of improvements, I released Routex 0.3.0-alpha.2.

As Routex was very much alpha, I took the liberty to break a few things for the better. If you used Routex already, check the Changelog where the breaking changes are listed.

Call for help!

I would love to collaborate on this lib! See some comments, PR’s or extensions from community members. The quality of this lib will benefit from multiple eyes. Feel free to contact me by personal message or other means.

Highlights

new! the process dictionary can be used to determine route branch
Routex solely used pattern matching on URL’s to determine which route branch (former scope) to use. Now the process dict key :rtx_branch -when set- overrides the URL matching.

new! extension Route interpolation
A route can be defined with route attribute interpolation. Any route attribute can be used!

live "/#{region}/products/#{language}/:id", ProductLive.Index, :index

new! module Routex.Matchable.
Uniform, better, faster, easier to use and it Just Works™. The functions in this module allow developers unexperienced with Macro’s and/or AST to create simple helper functions, while more experienced AST developers will be happy with the abstractions it provides as.

Matchables are an essential part of Routex. They are used to match run time routes with compile time routes.

This module provides functions to create Matchables, convert them to match pattern AST as well as function heads AST and to check if the routing values of two Matchable records match.

new! Module Routex.Branching
Provides a set of functions to build branched variants of macro’s. It’s interface is still a bit rough but it’s usable enough to build a new version of extension Verified Routes with it.

The renewed Verified Routes extension.
Using Routex.Branching it now also creates branching variants of url/{1,2,3} and path/{2,3} with, of course, configurable names. When the names of Phoenix are used an improved warning is displayed during route compilation.


improved: added the attribute ‘match?’ to the results returned by ‘alternatives/1’
The value is determined by a pattern match, so performance wise we’re good. It does make code clean.

<.link
        :for={alternative <- Routes.alternatives(@url)}
        rel="alternate"
        hreflang={alternative.attrs.language}
        patch={alternative.slug}
      >
        <.button class={ (alternative.match? && "bg-[#FD4F00]") || "" }>
          <%= alternative.attrs.name %>
        </.button>
      </.link>

Example app
Updated to use the latest version.
Example App
Github Example App

5 Likes

For those using Cldr.Routes and wanting to try Routex, a Cldr extension is on it’s way! :fire:

The convenience of Cldr.Routes combined with the power of Routex :muscle:

1 Like

It’s a wrap! Routex now has an adaptor for ex_cldr . In combination with other extensions it supports all features of Cldr.Routes such as translated path segment, interpolating locale data and verified routes…and then some[^1]!

@kip as you known infinity more than I do about Cldr, could you review the usage of Cldr? :slight_smile:

[^1] “And then some” is an informal idiom that means and even more. It is used to emphasize that there is a large number or quantity of something.

1 Like

A lib without a logo is as a vulnerability without a cool name. So I took the opportunity to dust of Illustrator and come up with a logo.

Every day a step closer to the first stable release :fire:

2 Likes

Not sure if I am

  1. procrastinating
  2. unhappy with the “looks like an Olympic Games logo” comments or
  3. simply like to be back at designing after a few years.

Either way no code was written tonight.

Let me know if you have a strong opinion or favor in this matter :slight_smile:

I like B more.

For some reason I can’t stop thiking that this would be the perfect logo for Igniter :smiley: .

—nvm—

I think it already has a logo? igniter/README.md at v0.4.6 · ash-project/igniter · GitHub