Generate API from Docs, e.g API blueprint/swagger/raml

General Prog (Archived)
1

I looking to build a library where some documentation, in one of the standard formats mentioned acts as the router.

For example if I was using API Blueprint and using this documentation.

FORMAT: 1A

# My Message [/message]
OK, `My Message` probably isn't the best name for our resource but it will do
for now. Note the URI `/message` is enclosed in square brackets.

## Retrieve a Message [GET]
Now this is informative! No extra explanation needed here. This action clearly
retrieves the message.

+ Response 200 (text/plain)

        Hello World!

## Update a Message [PUT]
`Update a message` - nice and simple naming is the best way to go.

+ Request (text/plain)

        All your base are belong to us.

+ Response 204

A router could simply point to a blueprint file

defmodule MyApp do
  blueprint "./myapp.api"
end

This would then look for modules MyApp.RetrieveAMessage and MyApp.UpdateAMessage which would be controllers and get routed the correct requests.

I thought this might really put the focus on documentation and be more readable than annotation in existing routing apis, e.g. https://github.com/xerions/phoenix_swagger

My Questions are:

  • I think API blueprint looks like the leading contender, what experience do people have and can they share any feedback
  • Do any of the above support documenting a streaming endpoint, via server sent events (“text/event-stream”) or perhaps websockets
2

At the risk of being a broken record, this is one of the things I like about GraphQL. The same code that ensures data integrity also can generate documentation, and yes, it provides a unified API and documentation structure over both request response and real time data.

However this is plainly about REST stuff so I’ll bow out :wink:

1 Like
3

No experience with it, but it would not be too hard to parse at all (I even have a parsing library that can help!).

Nope, those go down different paths from the Endpoint.

+1

4

Do tell? I discovered Prex uses drafter(ApiBlueprint parser) but be neater if there was an elixir solution.

No particular attachment to REST but I would like it to be as generic as possible HTTP

5

ExSpirit, I use it quite a bit myself (as have a few others recently too, yay!). ^.^

6

The issue is, this basically eliminates websocket support. Any API tool built around HTTP verbs simply won’t match up with websockets.

1 Like
7

I think a bit low level for my need. Certainly for a proof of concept I would use the full API Blueprint parser

8

One of the downsides with websockets? Seeing as HTTP/2.0 in many cases makes websockets redundant I’m not worried about supporting them. A handle to streamed HTTP request would still be interesting.

How does GraphQL handle realtime data?

9

Good points! My original reason for singling out websockets was because

streaming endpoint, via server sent events (“text/event-stream”) or perhaps websockets

However I see why HTTP/2 seems to obviate the websocket specific concerns. Let me address what GraphQL does and I think that will illustrate what I’m trying to say.

GraphQL’s main real time data approach is Subscriptions, which operate in a pubsub type manner. What’s interesting is how much this integrates with already established request / response content that already exists in your API. Consider an ordinary request response type GraphQL query to list comments on a post

{
  post(id: 123) {
    title
    body
    comments {
      body
      author { name id }
    }
  }
}

The code required to make this go on the server would be:

query do
  field :post, :post do
    arg :id, non_null(:id)
    resolve fn _, %{id: id}, _ -> {:ok, Repo.get(Post, id)} end
  end
end

object :post do
  field :comments, list_of(:comment), resolve: assoc(:comments)
  field :author, :user, resolve: assoc(:author)
  field :title, :string
  field :body, :string
end

object :comment do
  field :body, :string
  field :author, :user, resolve: assoc(:author)
end

object :user do
  field :name, :string
  field :id, :id
end

Note that what comes back is totally dynamic. The :post object has an :author field but because our query document isn’t asking for the posts’s author it won’t be loaded from the DB, nor will it come back in the response to the client.

Now, how does this relate to subscriptions? Well suppose when we go to a page we want to be notified in real time of new comments as they appear on the page. You’d submit a subscription document that looks like:

subscription {
  newComment(postId: 123) {
    body
    author { name id }
  }
}

To support this on the server we need incredibly little code, because we can hook directly to our already established :comment object, and get the ability to lazy load the author for free.

subscription do
  field :new_comment, :comment do
    arg :post_id, non_null(:id)

    topic fn %{post_id: post_id}, _ ->
      {:ok, post_id}
    end
  end
end

The return type of our :new_comment field is just exactly the same :comment type that we already have in our schema. No additional work has to be done in order to ensure that the :author can come back, because our :comment type already defines a way to load the author, and that way works whether you’re doing request response or real time.

And of course finally, this whole schema is both code that validates incoming data, provides flexibility for outgoing data, and also generates documentation.

I can get into more details of how the actual pubsub component works internally, but this is a rough sketch of the primary real time data approach for GraphQL (there are others, but less established at the moment).

2 Likes
10

I thought this might really put the focus on documentation

:thumbsup: For documentation focus. I love that the elixir community cares about docs :slight_smile:
As long as there is a single source of truth by either generating docs from code, or code from docs, then it’s a nice workflow for keeping accurate docs.

I think API blueprint looks like the leading contender, what experience do people have and can they share any feedback

I’m under the impression that swagger has the most tooling support, eg it can be used to create an aws api gateway. I’ve used API Blueprint with Apiary for hand written docs, but switched to swagger since there were existing tools to generate a spec from code.

I’ve also gotten a lot of use out of Bureaucrat and the java equivalent spring rest-docs for augmenting a swagger spec with example request-response documentation captured from controller tests.

If you are considering generating a router from a spec, then you might also look into:

  • Generating embedded ecto schemas to represent the request params
  • Changeset functions to validate requests
  • Test helpers to confirm that responses conform to the spec response models
  • Tests that confirm examples in the spec are accurate

Some users of phoenix_swagger hand-write the swagger spec, then just use the library for the validations and test helpers, so I think a router-generator could be a welcome addition :wink:

11

I’ve since discovered this api elements which as a common layer would be great because then I could support all the formats that have api-elements support

Your other suggestions are interesting and I hope if my implementation is extensible then they could be created as needed by others.
I however have no interest in the api-docs making assumptions about how the model of my application looks. I think the average Elixir application would benefit from forgetting more rather than less of the heritage MVC patterns that have come before.

12

Thanks to a great talk given at Elixir London (cc @samphilipd) I now know much more about GraphQL and have followed up further on subscriptions. I think that it subscriptions + mutations (as commands) could be a very interesting usecase when combined with an evensourcing backend. I found this scala example which is almost exactly what I would like to try an implement but in Elixir (of course). It also uses server sent events which I find very interesting because that could work with HTTP/2 and fallback to those remaining HTTP/1 client.

Me too, I never shared it here but I got the blueprint driven router to work. you can see it in action in the 10 minute video I posted in another thread.

13

Agreed, we’ve played around with this on some side projects and it fit really well. GraphQL mutations map exceptionally well to commands, and having first class events makes pushing subscriptions a breeze.

The main question we ran into is, when thinking about subscriptions, do you want to be pushing events or do you want to be pushing updated view models? Our particular example involved the tracking of a pallet of goods, and we found that the front end development was a lot easier of you subscribed to basically the current state of the pallet (essentially a view model) which would be pushed out whenever an event happened, vs subscribing to the event data itself. If you do the event data, the front end has to figure out how to combine it all together, which is generally more pain that its worth.

I’m sure it’ll depend on each situation though.

1 Like
14

Nice! I’ve also started something like a successor to phoenix_swagger in open_api_spex. It’s currently swagger 3.0 only so not very useful until all the rest of the tooling ecosystem catches up.
One of the big differences is there is no longer a mix task that assumes Phoenix conventions in your project - thanks for the nudge in that direction :smile:

1 Like
15

We also chose to send a materialised view of the state to the client. our hope was to make the client dumber in the process. It has worked but I agree there is quite likely merits to both approaches.

1 Like
16

I’ve never quite understood all the extra changes phoenix makes to a standard mix project. I see the project still says plug applications, which is not something I can say about many of my applications :slight_smile:

17

I don’t know exactly what the grammar is for the Blueprint file, but I want to stress that unlike a PEG parser or equivalent, ExSpirit can parse indentation dependent file formats with no lexinh step or introduction of custom tokens into the token stream.

1 Like
18

Here is the original talk in case anybody is interested:

What is a GraphQL? (and you can too!)

It covers:

  • Basics of GraphQL
  • Why you would choose GraphQL or REST
  • Live coding a simple GraphQL server in Elixir using Absinthe
1 Like
19

Nice work. Do you have any estimates when it will land in hex?

20

Do you have any estimates when it will land in hex?

Done! open_api_spex package and docs now in hex.

2 Likes