How to generate schemas at run-time from configuration in GraphQL/Absinthe?

Hi,

I have the following situation: We have “objects” that can be of different types, and the different types has some common fields and also type-specific fields that can be different.

The objects and their fields are configured dynamically and new object and fields can be added at run-time. I would like to query the objects using GraphQL, so I had a look at Absinthe. It seems that the schemas are specified using macros at compile-time so I’m curious if anybody has a good solution to this.

Configurations change rarely so it’s perfectly fine to recompile modules when they change, if that makes the solution easier.

As an simplified example, we have item objects and document objects configured similar to this

[
{"type": "item", "fields": {"item_number": "string", "name": "string", "documents": "list of document"}},
{"type": "document", "fields": {"category": "string", "header": "string", "filename": "string"}
]

and I would like to query them something like this

item(id: "100") {
  item_number
  name
  documents {
    category
    header
    filename
  }
}

so the schema would have to be generated from a configuration in the database. The resolver functions are easy enough since they can use the configuration at run-time, but I have not thought of a good way to generate the schemas at run-time.

Any suggestions would be much appreciated!

Thanks,
Martin

Hi @norpan.

As of 1.5, Absinthe schemas can be built entirely at runtime, using ordinary Elixir structs. You can see a small example here in the test suite: https://github.com/absinthe-graphql/absinthe/blob/master/test/absinthe/schema/manipulation_test.exs#L107

Unfortunately because it’s a relatively new feature, there isn’t any kind of comprehensive guide on how to do this. The easiest way to get going though is to create a notation file with an example of the kind of entity that you want to represent, and then look at the blueprint it creates.

defmodule Foo do
  use Absinthe.Schema.Notation

  object :user do
    field :id, non_null(:id)
    field :name, :string
  end
end

Foo.__absinthe_blueprint__ |> IO.inspect(pretty: false)
#=> %Absinthe.Blueprint{
  adapter: nil,
  directives: [],
  errors: [],
  execution: %Absinthe.Blueprint.Execution{
    acc: %{},
    adapter: nil,
    context: %{},
    fields_cache: %{},
    fragments: %{},
    result: nil,
    root_value: %{},
    schema: nil,
    validation_errors: []
  },
  flags: %{},
  fragments: [],
  initial_phases: [],
  input: nil,
  name: nil,
  operations: [],
  prototype_schema: nil,
  result: %{},
  schema: Foo,
  schema_definitions: [
    %Absinthe.Blueprint.Schema.SchemaDefinition{
      __private__: [],
      __reference__: %{location: %{file: "iex", line: 0}},
      description: nil,
      directive_artifacts: [],
      directive_definitions: [],
      directives: [],
      errors: [],
      flags: %{},
      imports: [],
      module: Foo,
      source_location: nil,
      type_artifacts: [],
      type_definitions: [
        %Absinthe.Blueprint.Schema.ObjectTypeDefinition{
          __private__: [],
          __reference__: %{location: %{file: "iex", line: 5}, module: Foo},
          description: nil,
          directives: [],
          errors: [],
          fields: [
            %Absinthe.Blueprint.Schema.FieldDefinition{
              __private__: [],
              __reference__: %{location: %{file: "iex", line: 6}, module: Foo},
              arguments: [],
              complexity: {:ref, Foo,
               {Absinthe.Blueprint.Schema.FieldDefinition, {:user, :id}}},
              config: {:ref, Foo,
               {Absinthe.Blueprint.Schema.FieldDefinition, {:user, :id}}},
              default_value: nil,
              deprecation: nil,
              description: nil,
              directives: [],
              errors: [],
              flags: %{},
              function_ref: {:user, :id},
              identifier: :id,
              middleware: [...],
              ...
            },
            %Absinthe.Blueprint.Schema.FieldDefinition{
              __private__: [],
              __reference__: %{location: %{file: "iex", line: 7}, module: Foo},
              arguments: [],
              complexity: {:ref, Foo,
               {Absinthe.Blueprint.Schema.FieldDefinition, {:user, :name}}},
              config: {:ref, Foo, 
               {Absinthe.Blueprint.Schema.FieldDefinition, {:user, :name}}},
              default_value: nil,
              deprecation: nil,
              description: nil,
              directives: [],
              errors: [],
              flags: %{},
              function_ref: {:user, ...},
              identifier: :name,
              ...
            }
          ],
          flags: %{},
          identifier: :user,
          imports: [],
          interface_blueprints: [],
          interfaces: [],
          is_type_of: {:ref, Foo,
           {Absinthe.Blueprint.Schema.ObjectTypeDefinition, :user}},
          module: Foo,
          name: "User",
          source_location: nil
        }
      ],
      type_extensions: []
    }
  ],
  source: nil,
  telemetry: %{}
}

This is a bit verbose but it will show you the type definitions inside the schema, and the internal structures like the object definition:

%Absinthe.Blueprint.Schema.ObjectTypeDefinition{
  __private__: [],
  __reference__: %{location: %{file: "iex", line: 5}, module: Foo},
  description: nil,
  directives: [],
  errors: [],
  fields: [
    %Absinthe.Blueprint.Schema.FieldDefinition{
      __private__: [],
      __reference__: %{location: %{file: "iex", line: 6}, module: Foo},
      arguments: [],
      complexity: {:ref, Foo,
       {Absinthe.Blueprint.Schema.FieldDefinition, {:user, :id}}},
      config: {:ref, Foo,
       {Absinthe.Blueprint.Schema.FieldDefinition, {:user, :id}}},
      default_value: nil,
      deprecation: nil,
      description: nil,
      directives: [],
      errors: [],
      flags: %{},
      function_ref: {:user, :id},
      identifier: :id,
      middleware: [
        {:ref, Foo, {Absinthe.Blueprint.Schema.FieldDefinition, {:user, :id}}}
      ],
      module: Foo,
      name: "id",
      source_location: nil,
      triggers: {:ref, Foo,
       {Absinthe.Blueprint.Schema.FieldDefinition, {:user, :id}}},
      type: %Absinthe.Blueprint.TypeReference.NonNull{errors: [], of_type: :id}
    },
    %Absinthe.Blueprint.Schema.FieldDefinition{
      __private__: [],
      __reference__: %{location: %{file: "iex", line: 7}, module: Foo},
      arguments: [],
      complexity: {:ref, Foo,
       {Absinthe.Blueprint.Schema.FieldDefinition, {:user, :name}}},
      config: {:ref, Foo,
       {Absinthe.Blueprint.Schema.FieldDefinition, {:user, :name}}},
      default_value: nil,
      deprecation: nil,
      description: nil,
      directives: [],
      errors: [],
      flags: %{},
      function_ref: {:user, :name},
      identifier: :name,
      middleware: [
        {:ref, Foo, {Absinthe.Blueprint.Schema.FieldDefinition, {:user, :name}}}
      ],
      module: Foo,
      name: "name",
      source_location: nil,
      triggers: {:ref, Foo,
       {Absinthe.Blueprint.Schema.FieldDefinition, {:user, :name}}},
      type: :string
    }
  ],
  flags: %{},
  identifier: :user,
  imports: [],
  interface_blueprints: [],
  interfaces: [],
  is_type_of: {:ref, Foo,
   {Absinthe.Blueprint.Schema.ObjectTypeDefinition, :user}},
  module: Foo,
  name: "User",
  source_location: nil
}

You can read more about schema modifiers here: https://hexdocs.pm/absinthe/Absinthe.Schema.html#module-custom-schema-manipulation-in-progress

If you combine that with the persistent term backend https://hexdocs.pm/absinthe/Absinthe.Schema.PersistentTerm.html then you can construct schemas entirely at runtime.

7 Likes

Thanks! That’s extremely helpful and now I know what to spend my day doing! :slight_smile:

Thanks again, I’ve been testing to manipulate the blueprint but I’m still a bit confused how to run it at run-time. As I understand the example it still runs the functions at compile time.

I’m still wondering how to actually modify the blueprint at run-time to then update the schema at runtime. I’m not that good at reading macro code yet.