Raxx - Interface for HTTP webservers, frameworks and clients (1.0 now released!)

[Crowdhailer]

@Qqwy

Interesting idea. Is it abstractable?

Not sure quite what you mean. So far it’s not been to bad because in many cases middleware is a layer of indirection. replacing the two or three things that would have been sensible middleware admittedly costs a few lines but it is ludicrously explicit, easy to understand and test.

[OvermindDL1]

@Crowdhailer

I have something that just extracts the session when asked

I quite prefer this explicit form actually, plus I can control when I actually need to do it so I don’t necessarily always incur the cost like with plug.

[Qqwy]

I should have explained my previous question a bit more thorougly:

Because you have to provide statements that call the internal functionality of the middleware explicitly, what happens when I want to write a middleware ‘A’ that itself internally depends on another middleware ‘B’ to function? Does the user that wants to use ‘A’ also need to write the explicit statements to make sure ‘B’ works, is calling ‘A’ enough and can it dispatch to ‘B’ internally on its own?

Maybe it’s still a bit vague, and maybe I am also describing a scenario that is more complex than the scenarios you currently are exploring.

[josevalim]

@OvermindDL1

I quite prefer this explicit form actually, plus I can control when I actually need to do it so I don’t necessarily always incur the cost like with plug.

With Plug you have full control too. fetch_session and plugs are regular functions, just call it when you want to, you don’t need to have it as part of a pipeline.

[OvermindDL1]

Yep I know, I do that at times too, properly baking the init at compiletime and all.

However I tend to thread my conn all over the place and different parts need different bits, sometime at the same time, but I cannot be sure that it’s already acquired or not so I test in some (otherwise expensive) cases. This is why it would be nice if all the getter functions returned a Tuple of both the value and the conn, that way it caches it on the conn if not there and I can thread it with impunity without worrying about writing my own tests for if it is already cached or not. Returning just the value means I have to remember to do things like fetch_session manually at the right times. (Basically I think conn should be a state monad threaded through all of its calls, every single call regardless of if used or not.)

[OvermindDL1]

Hmm, an idea, conn could have a value key on it a which is the return of the most recent getter otherwise ignored, then every function would just take and return conns and it could be piped with impunity (especially if about to name returns via another call, everything could become such a nicer pipe).

[josevalim]

@OvermindDL1

This is why it would be nice if all the getter functions returned a Tuple of both the value and the conn, that way it caches it on the conn if not there and I can thread it with impunity without worrying about writing my own tests for if it is already cached or not.

Cowboy did that on v1 and basically reverted back because it pushed the complexity to most applications while they absolutely do not care about this.

In any case, you can implement such function easily using the fetch_session (which is a no-op if the session is already fetched) and get_session:

def fetch_session_key(conn, key) do
  conn = fetch_session(conn)
  {get_session(conn, key), conn}
end

[OvermindDL1]

That one is a noop yes, but not all were, I ran into some expensive ones that weren’t, like grabbing the body I think was one? In addition to other libraries.

And I can see the complexity with the Tuple, a couple extra things to type, but putting a value return on the struct itself makes it far easier, hence why suggested.

[Crowdhailer]

@josevalim

With Plug you have full control too

This is certainly true, once there is an assumption of middleware there will be libraries that are written that assume some functionality is only useful as middleware and bake in that assumption.
It’s certainly interesting to explore the alternatives. @OvermindDL1 I would love to see anything you have that would fit into the topic of “You might not need that middleware” I hope to write something about that eventually but don’t see it happening soon

[josevalim]

@OvermindDL1

And I can see the complexity with the Tuple, a couple extra things to type, but putting a value return on the struct itself makes it far easier, hence why suggested.

Afaik, the only thing we don’t store is the body, because storing the body has large implications.

@Crowdhailer

This is certainly true, once there is an assumption of middleware there will be libraries that are written that assume some functionality is only useful as middleware and bake in that assumption.

Exactly. Which is one of the reasons why we decided to avoid the notion of middleware in plug altogether.

[OvermindDL1]

@Crowdhailer

@OvermindDL1 I would love to see anything you have that would fit into the topic of “You might not need that middleware” I hope to write something about that eventually but don’t see it happening soon

Honestly I’ve never really used middleware, I’ve always preferred explicit calls, even when I’ve made web servers in C++, though I may be a bit of an odd-ball out. ^.^

[tmbb]

@OvermindDL1

Basically I think conn should be a state monad threaded through all of its calls, every single call regardless of if used or not.

Needs more type classes…

[Crowdhailer]

I like this. I have looked wrapping a session up in a state monad.

Session.for request do
  value <- get(:user_id)
  _ <- set(csrf: new_token)
after
  response
end

This is dreamcoding no implementation in the works yet

[Crowdhailer]

Better erlang support in 0.15.4 release.

The main module now has a parallel Erlang friendly implementation. anyone wanting to try to use Ace + Raxx in erlang can now use handlers like this.

-module(greetings_www).

-behaviour('Elixir.Raxx.Server').

-export ([handle_head/2]).

handle_head(_Request, _Config) ->
  Response = raxx:response(ok),
  raxx:set_body(Response, <<"Hello, World">>).

Implementation.

The whole :raxx module simply delegates every function in the Raxx module

defmodule :raxx do
  @moduledoc false
  # A module that is clean to call in erlang code with the same functionality as `Raxx`.

  for {name, arity} <- Raxx.__info__(:functions) do
    args = for i <- 0..arity, i > 0, do: Macro.var(:"#{i}", nil)
    defdelegate unquote(name)(unquote_splicing(args)), to: Raxx
  end
end

[Crowdhailer]

Extra helpers for fetching port and host information from a request added: 0.15.5

iex> Raxx.request(:GET, "http://www.example.com/hello")
...> |> Raxx.request_host()
"www.example.com"

iex> Raxx.request(:GET, "http://www.example.com:1234/hello")
...> |> Raxx.request_host()
"www.example.com"

iex> Raxx.request(:GET, "http://www.example.com:1234/hello")
...> |> Raxx.request_port()
1234

iex> Raxx.request(:GET, "http://www.example.com/hello")
...> |> Raxx.request_port()
80

iex> Raxx.request(:GET, "https://www.example.com/hello")
...> |> Raxx.request_port()
443

[Crowdhailer]

Serialization and Parsing tools added 0.15.7

These tools are part of the Raxx library as they can be used to implement both servers and clients.
The Ace server and a ServerSentEvent Client library are being updated to make use of these tools.
Hopefully a Raxx based client library will follow.

iex> request = Raxx.request(:GET, "http://example.com/path?qs")
...> |> Raxx.set_header("accept", "text/plain")
...> {head, body} =  Raxx.HTTP1.serialize_request(request)
...> :erlang.iolist_to_binary(head)
"GET /path?qs HTTP/1.1\r\nhost: example.com\r\naccept: text/plain\r\n\r\n"
iex> body
{:complete, ""}

iex> response = Raxx.response(200)
...> |> Raxx.set_header("content-type", "text/plain")
...> |> Raxx.set_body("Hello, World!")
...> {head, body} =  Raxx.HTTP1.serialize_response(response)
...> :erlang.iolist_to_binary(head)
"HTTP/1.1 200 OK\r\ncontent-length: 13\r\ncontent-type: text/plain\r\n\r\n"
iex> body
{:complete, "Hello, World!"}

[Crowdhailer]

Raxx now has a simple client for HTTP/1.1

Examples

synchronous

request = Raxx.request(:GET, "http://example.com")
|> Raxx.set_header("accept", "text/html")

{:ok, response} = Raxx.SimpleClient.send_sync(request, 2000)

asynchronous

request = Raxx.request(:GET, "http://example.com")
|> Raxx.set_header("accept", "text/html")

channel = Raxx.SimpleClient.send_async(request)
{:ok, response} = Raxx.SimpleClient.yield(channel, 2000)

Simple Client

Client is simple because it makes very few assumptions about how to deal with requests and responses.

For example.

• Cookies are not managed, each request is handled in isolation
• Connections are not limited or reused, each request gets a new connection

These omissions of functionality make the client much simpler to work with and reason about.
They are also not limitations in many cases. An API client probably doesn’t use cookies.

Composable requests

Because a Raxx.Request is just a data structure,
using this client separates the logic of building the request from the side effect of sending it.

This makes it very easy add custom functionality for your own clients.

 import Raxx

def set_request_id(request, request_id) do
  request
  |> set_header("x-request-id", request_id)
end

def set_json_payload(request, json) do
  request
  |> set_header("content-type", "application/json")
  |> set_body(Poison.encode!(json))
end

# later
request = Raxx.request(:POST, "http://api.com/create_user")
|> set_request_id("12345")
|> set_json_payload(%{"username" => "alice"})

The value of this separation for me in my own projects has been with testing.
It’s now very easy to create invalid requests. just forget to set a request_id and then send it to the API.

Raxx 0.15.8 adds Raxx.SimpleClient

[Crowdhailer]

Raxx.View and Raxx.Layout added in 0.15.9.

These changes include an EEx.HTMLEngine, this module will be moved to a separate project at somepoint.

Raxx.Layout

Creating a new layout

# www/layout.html.eex
<h1>My Site</h1>
<%= __content__ %>

# www/layout.ex
defmodule WWW.Layout do
  use Raxx.Layout,
    layout: "layout.html.eex"

  def format_datetime(datetime) do
    DateTime.to_iso8601(datetime)
  end
end

Creating a view

# www/show_user.html.eex
<h2><%= user.username %></h2>
<p>signed up at <%= format_datetime(user.interted_at) %></p>

# www/show_user.ex
defmodule WWW.ShowUser do
  use Raxx.Server
  use WWW.Layout,
    template: "show_user.html.eex",
    arguments: [:user]

  @impl Raxx.Server
  def handle_request(_request, _state) do
    user = # fetch user somehow

    response(:ok)
    |> render(user)
  end
end

Raxx.View

Example

# greet.html.eex
<p>Hello, <%= name %></p>

# layout.html.eex
<h1>Greetings</h1>
<%= __content__ %>

# greet.ex
defmodule Greet do
  use Raxx.View,
    arguments: [:name],
    layout: "layout.html.eex"
end

# iex -S mix
Greet.html("Alice")
# => "<h1>Greetings</h1>\n<p>Hello, Alice</p>"

Raxx.response(:ok)
|> Greet.render("Bob")
# => %Raxx.Response{
#      status: 200,
#      headers: [{"content-type", "text/html"}],
#      body: "<h1>Greetings</h1>\n<p>Hello, Alice</p>"
#    }

[Crowdhailer]

0.16.0 A Bucket of changes, focused around adding utilities and improving error cases.

Includes BREAKING CHANGE:

• set_body automatically adds the content length.
• set_body raises exception for responses that cannot have a body.
• html_escape moved to EExHTML project.

CHANGELOG

Added

• :maximum_body_length options when using Raxx.Server so protect against bad clients.
• Raxx.set_content_length/3 to set the content length of a request or response.
• Raxx.get_content_length/2 to get the integer value for the content length of a message.
• Raxx.set_attachment/2 helper to tell the browser the response should be stored on disk rather than displayed in the browser.
• Raxx.safe?/1 to check if request method marks it as safe.
• Raxx.idempotent?/1 to check if request method marks it as idempotent.
• Raxx.get_query/1 replacement for Raxx.fetch_query/1 because it never returns error case.

Changed

• Raxx.set_body/2 will raise an exception for responses that cannot have a body.
• Raxx.set_body/2 automatically adds the “content-length” if it is able.
• Requests and Responses now work with iodata.
  • Raxx.body spec changed to include iodata.
  • Improved error message when using invalid iolist in a view.
  • Raxx.NotFound works with iodata for temporary body during buffering.
  • render function generated by Raxx.View sets body to iodata from view,
    without turning into a binary.
• Raxx.set_header/2 now raises when setting connection specific headers.

Removed

• EEx.HTML replaced by EExHTML from the eex_html hex package.
• Raxx.html_escape/1 replaced by EExHTML.escape_to_binary/1.

Fixed

• Raxx.HTTP1.parse_request/1 and Raxx.HTTP1.parse_response/1 handle more error cases.
  • response/request sent when request/response expected.
  • multiple “host” headers in message.
  • invalid “content-length” header.
  • multiple “content-length” headers in message.
  • invalid “connection” header.
  • multiple “connection” headers in message.

[Crowdhailer]

Breaking changes to separate the Streaming interface from the Simple interface are coming in the next version.

Introduce separate behaviour for streaming and simple cases

This pull request introduces the changes as well as linking to one project for an example of upgrading.
Feedback invited on the PR, or in slack

The reason for these changes is to lay the groundwork for adding better middleware.

@michalmuskala

I wonder about the decision to make middlewares compile-time. I generally consider the fact that plugs are compile-time the biggest design flaw for plug - it’s often very limiting and annoying

For example the new middleware will be able to be configured at Runtime.

@Qqwy

However, I am not really happy with how the middlewares currently work, since they ‘decorate’ the original handle_xxxx call with their own wrapper at compile time which may result in performant code, but it makes the middleware code itself quite hard to read.

New middleware will also not require the use of Macros.