Hey everyone!

What I’ll cover in this post:

1. Major refactors
2. The future of Ash.Flow
3. Current state of atomics and bulk actions
4. Whats next?
5. Thanks

Major Refactors

We’ve had an interesting few weeks in Ash land, been working with a lot of our power users to test and vet some major refactors of the internals in Ash. The primary goal of these refactors was simplification. There were some design choices made long ago when Ash was much simpler, that funneled all actions and other logic through the Ash.Engine. I won’t go into those decisions here and why they didn’t pan out, but this engine is still around as it powers Ash.Flow (for now, read on for more).

The internals of every action type was completely overhauled. They are drastically simpler, easier to reason about, and easier to maintain. Additionally, relationship loading and calculation dependency loading/dependency resolving were completely rewritten. These have also become significantly simpler.

This new design removes a major bit of indirection that would often make it difficult to determine where something has gone wrong when reporting bugs. You’d get stack traces that looked like the error came from somewhere in the engine, something like Ash.Engine.resolve_step. Except all logic went through that same function. Not very useful! Now, it’s all about as vanilla elixir code as we can get. Its much easier to figure out what went wrong and why when things do go wrong, which is great for everyone involved

What we weren’t targeting necessarily, but were very hoping to see, were performance improvements. I’m happy to say that we’ve seen massive performance improvements as a result of these refactors. Create/update/destroy actions are 2-3x faster (custom action excluded, we can’t make your functions faster ) and read actions are 2-5x faster in simple cases and 50x faster in cases with lots of calculations and relationship loading. This was a very good result!

Additionally, update & destroy actions have been modified to support something called “automatic atomic upgrades”, which I will discuss more in the section on atomics and bulk actions.

The future of Ash.Flow

We have officially halted work (but not support) on Ash.Flow in favor of @jimsynz’s project Reactor. We will be shipping Ash.Reactor natively with Ash, which is not an “ash extension”, but rather a “reactor extension” that adds Ash-specific step types to Reactor.

Reactor is already a superior product to Ash.Flow in most aspects, and we are working to get it to parity with Ash.Flow. The main things left to get to that point are some higher level step types as well as better error handling. Once we reach feature parity and are happy with its behavior, we will provide a guide on migrating your Ash.Flows to Reactor + Ash.Reactor. While this may be inconvenient for those using Ash.Flow heavily (which I don’t believe is the bulk of Ash users) this will be a very positive change in the long run.

How does this affect current Ash.Flow users? In 3.0, Ash.Flow will be moved into its own package, allowing us to fully remove the Ash.Engine from core, now that the only thing using it is Ash.Flow. All of its functionality will remain the same, and we will continue to fix any bugs that arise with it. We will not, however, add any new features, step types or options unless absolutely necessary.

Current state of atomics and bulk actions

Their state is: done! Well, the first iteration of them is done and is now officially released in 2.19.0. However, you will not find a guide explaining how to properly use them. There is reference documentation, i.e you can see the functions available Api.bulk_create and Api.bulk_update, as well as the new callbacks available for defining atomic composable behavior, like Ash.Resource.Change.atomic/3 and Ash.Resource.Change.after_atomic/3.

Additionally, you will automatically reap the benefits of atomic updates/destroys in cases where you use builtin behavior only. Regular destroy actions and update actions that don’t have custom changes will automatically be upgraded to be done atomically. For instance:

update :increment_score do
  change increment(:score)
end

In the past, this(which has obvious issues w/ concurrent processing of actions, and is one example of why it was critical that we finish atomic updates), would add 1 to the score in memory and save it. But now, it will result in UPDATE ... WHERE ... SET score = score + 1. This same thing applies to builtin validations and policies. They will be done in the data layer query, providing concurrency safety. AshGraphql and AshJsonApi will be upgraded to use this new bulk update and bulk destroy functionality to avoid doing a lookup then an update. This will make many actions practically twice as fast, without any change required on your part. More on this in the upcoming guides.

You are free to experiment with and use these things, they are officially supported. However, you may find them difficult to understand in their current state. The most important thing that we will be doing next is a complete documentation overhaul, focused on providing a clean path for users to learn Ash, best practices, and most importantly, how to write good resource actions. In this new documentation we will dive deeply into bulk actions and atomics.

A guide may be forthcoming ahead of 3.0, but in reality these features are actually just 3.0 features that we released early because we needed to verify that they contained no breaking changes when using current default configuration. As such, they will be fully documented in 3.0.

This brings us to

Whats next?

2.19.0 will be the last minor release before 3.0

CleanShot 2024-02-16 at 16.53.18@2x1038×138 8.92 KB

The 3.0 release has one major theme: Developer Experience.

3.0 consists of a series of breaking changes that change current default behavior to things that are safer and/or more expected. These changes, along with new documentation, will be the beginning of the shift to focus on DX. With these core refactors and features complete, I’ll have more time to dedicate to documentation, quality of life improvements, and making tangible improvements to high level packages like ash_admin and ash_double_entry.

Upgrading

We will provide comprehensive guides to upgrading your application to account for each breaking change. Our current worst-case target for how long it should take to upgrade to 3.0 is one developer day (of a moderate-to-skilled Ash user).

Closing

I’m really excited for the new features and refactors that have gone in. More than that, I’m excited to illustrate what makes atomics and bulk actions so unique for Ash, and how they can give you superpowers! Declarative design really shines here, allowing us to translate your actions to support streaming/batched updates/destroys, as well as updates/destroys in a single call to the data layer, with full support for all of your validations and policy logic. This will all become clearer in the new documentation, but I’ve not seen anything capable of this kind of behavior before.

Thanks

Thank you for everyone who helped us get to this point. I’d like to thank Alembic and all of the fine Alembians for their support of this project, without which we would never have been able to come so far so fast. I’d also like to thank the folks that have helped me by trying out all of these refactors in their own application and helping me find and fix errors. We affectionately refer to these folks as “main liners”.

Main Liners

In no particular order:

@chef
@rgraff
@ahey
@barnabasJ
Kernel (don’t know their elixir forum handle)
@wintermeyer
@joshprice
@frankdugan3
@dblack

Contributors

We now have 122 total contributors. This isn’t counting users that have submitted issues, or assisted with debugging, or helped support other users. I can’t list everyone, but thank you all for believing in this project and helping us make it the best that it can be! We have an automatically updating list of contributors on our homepage at https://ash-hq.org!

CleanShot 2024-02-16 at 17.19.56@2x1200×1510 163 KB