Optimize front CI build when using absinthe

Has anybody split apart their Phoenix project to compile their absinthe schemas without referencing the backend, so that frontend CI builds can only compile graphql doc and then front end?

Hi @kanishka I’m not completely following what you’re asking here. Can you lay out the compilation order you’re looking to do in a step by step way?

So something like the following:

  • an elixir project that only consists of absinthe graphql types with placeholder resolver implementations - compiled first
  • front end build takes the graphql schema from this step and launches into front end compilation
  • a phoenix application uses the skeleton absinthe graphql types and connects them with real resolver implementations - compiled concurrently in another ci worker

@kanishka This is rather tricky these days because schema compilation pipelines allow your application code to customize which types are created. I’m not sure it’s feasible to get the graphql types out of an app without compiling basically the full app.

I think the larger commercial users will eventually want this ability. Waiting an extra 4 minutes in CI for a front end only change feels like the first CI bottleneck people will address. I guess you can work around it intelligent use of artifacts and caching schema with a key based on source directory contents.

Your project alone (without needing to recompile deps) takes 4 minutes to build? I’d be looking to fix that with better build artifact caching to begin with then.

For what it’s worth, Absinthe has a mix task to export the schema out to both SDL and JSON. If you have distinct front end and backend steps, then you can have the front end step depend on that artifact. If you change the backend code however then you need to compile the whole backend before you can reproduce that artifact.

At the risk of being a bit blunt, larger commercial users are more than welcome to contribute whatever is necessary to Absinthe to make that work then.

2 Likes

I’ll ask around amongst our devs. I am unsure if anybody will be motivated enough to try it.

I think you have more of a code organisation problem. As @benwilson512 explains you can use the mix task to generate your schema in SDL or JSON format. I am assuming your front-end code uses this to generate its code. If you (in development) commit this schema to your front-end code, there is no need to compile the backend in your CI just to get the schema.
I have a large project that uses elm in the front and I do just that. The only discipline needed is when a change results in a new schema, this needs to be generated and committed to the front.

Mono repo with many developers, people would have to keep resolving git conflicts in the generated file. I think caching to artifacts is a good stop gap, but we’ll see if people want to take on some code reorg or extending absinthe internally.

I don’t think there is a generalized solution in Absinthe itself for this, because Absinthe allows you to run arbitrary application code when constructing your schema. Absinthe can’t know a-priori which code this is.

If you wanted to build a custom export mix task that went ahead and compiled specific files you use for building schemas, that might work for you. Alternatively, improved artifact caching of the _build directory should hopefully minimize how many files have to be built per commit.

1 Like

I do something like this for a project that uses import_sdl to define its schema, compiling the schema but not the backend (resolvers and hydrations) when not needed.