Ash Framework 3.0: Release Candidates

[zachdaniel]

Ash Framework 3.0: Release Candidates!

The day is finally here! This is the culmination of years of work from the Ash team and community

We are ready for the adventurous folks to try out the latest release candidates of Ash and associated packages! Every Ash package (that the Ash team maintains) now has a published release candidate. See the GitHub issue for the status of known community packages. That same issue tracks our progress in updating example applications. While we can’t update those packages, any authors of community packages that need assistance with the upgrade should feel free to reach out with any questions/issues that they face.

We’ll be spending the next 3-4 weeks working on revamping our documentation, rewriting guides and adding a new cookbook (as well as giving a training at ElixirConf EU and Gig City Elixir!). During that time, we’ll be responding to feedback on 3.0, working out any kinks, and getting things ship-shape for the official launch.

What is changing?

While there are far too many things to put them all here, I want to give those who haven’t been following along with our 3.0 teasers an idea of where the focus is for Ash 3.0. There are three themes that cover most of the changes:

• Security: We want to make sure that Ash is safe and secure by default, and that it is free of foot guns that could lead to security issues, data consistency or other critical failures. Some examples of this are:

  • Actions no longer accept all public attributes by default
  • All fields are now private by default
  • Authorization is always done by default

• Simplicity: Ash is a huge ecosystem, and there is a lot to learn about how it works. Every confusing term and inconsistency increases the mental burden on developers, and increases the likelihood of bugs and wasted time. Some examples of this are:

  • Ash.Api is now Ash.Domain
  • Not selected attributes show %Ash.NotLoaded{} instead of nil

• Developer Experience: With 2.x versions adding support for atomics and bulk actions, the core functionality of Ash is in a very good place. This gives us the time and mental cycles to devote to developer experience. While many of the 3.0 changes are focused on DX, we also expect to see a much greater focus on this moving forward! Some examples of this are:

  • Ash.ToTenant protocol
  • Custom Expressions
  • Policies & Code Interfaces on the domain

How do I upgrade?

It is very important that you read and follow the upgrade guide! You will need to go through it section by section and consider if there are changes from that section that you must make. Don’t forget that some packages have their own upgrade guides for breaking changes contained in that package. Links to those guides are at the top.

We were initially targeting 1 developer day for an upgrade. I don’t think we managed to keep things quite so simple, but our initial feedback is good that the upgrade is a manageable piece of work.

What if I need help?

If you encounter an issue or change that is not covered in the upgrade guide please contact us or open a PR to add to the guide. Even if it’s just a note reminding folks to look out for something that might be a gotcha. Every little bit helps

Feel free to ask questions here on the forums or on our discord server if you need assistance.

What about 2.x?

Once 3.0 is out of release candidacy phase, Ash 2.x will receive a minimum of 6 months of critical bug fixes and security upgrades. Critical bug fixes include issues that have no reasonable workaround. Security upgrades includes things like updating potentially vulnerable packages, or fixing any issues related to policies, authentication, etc. We won’t be leaving folks in the dust, but it will absolutely be in your best interest to find a time in the next few months to upgrade, as there are many features in 3.0 to take advantage of and there are already more on the way!

Closing

It truly is a great time to be an Elixirist, and it is my sincerest hope that Ash Framework can contribute to Elixir and Elixir developers in tangible and significant ways. Thank you all for your time, contributions but above all, thank you for being an amazing community to be a part of!

I can’t wait to see what folks think of 3.0

Oh, and one more thing…

I’d like to tease a project that we’ve been formulating for some time now, and is finally underway in earnest.

This is just a mockup, not the final cover, we’re not graphic designers
You’ll be hearing more from @sevenseacat in the coming weeks

[zachdaniel]

Special Thanks

The Core Team

You’ve all done amazing work, and your help and support is instrumental to Ash’s success.

• @jimsynz
• @barnabasJ
• @joshprice
• @rgraff

The community

While there are too many folks to name, I’d like to highlight a few who have made an especially large impact lately.

• @vonagam | Consistently provides excellent pull requests (often fixing bugs that I personally added )
• @rbino | Upgraded AshGraphql to support Ash 3.0, one of the largest/most complex upgrades required!
• @dblack | Upgraded AshPhoenix to support Ash 3.0.

If you’ve been contributing but didn’t get a shout out, please know your contributions are not any less valuable. This only works because we’re all working towards a shared goal, and it’s an amazing privilege to be rowing in the same direction as so many fine folks. I’m currently investigating more ways that we can recognize community members for their great work

Alembic

Alembic is funding the time of myself, @jimsynz and often other Alembians to work on Ash. Their willingness to put their money where their mouth is shows not only their conviction that Ash is a better way to build software, but also the calibre of their team. They are putting Ash through its paces on tons of ambitious projects, which drives many of the benefits and improvements that Ash users receive! I’m beyond grateful for their support and expertise.

ElixirForum

I’d like to make a special callout to the fine folks running this forum, and the community here. A few months ago we made the choice to focus our support efforts here on ElixirForum. This was unequivocally a good choice! Not only has it helped increase the discoverability of Ash in general(via its wildly good SEO), it also makes sure that our support efforts don’t disappear behind a walled garden. We still have our discord server, and it is a great place to chat and interact with the Ash community, but we encourage users to ask questions here on the ElixirForum primarily. This community is well moderated, kind and helpful! A community is always representative of its leaders, and so I’d like to thank @AstonJ and others for their excellent stewardship.

In fact, now that we’ve got 300 threads we’ve been upgraded to a full 3-section Ash Forum, with dedicated sections for News, Chat, and Questions!

[zachdaniel]

Please check out our upcoming events as well! We’re giving a training that will be updated for Ash 3.0 at ElixirConf EU and Gig City Elixir

ElixirConf EU:

Ash Framework: The Fast Track to Full Stack

Gig City Elixir:

Gig City Elixir 2024

[Eiji]

I have just reminded that there’s ash_events repository, but it looks like it did not had much attention for over a half year and it’s also not ready for production yet. Are there any plans to finish this project and make it compatible with 3.x version?

[arcanemachine]

Cool stuff. I’ve been wanting to try out Ash again, but have been waiting for the new release to give it a shot. Looking forward to it for sure!

[zachdaniel]

I will update it for 3.0 Its a very small library, it can be copied into folks apps. Until I or someone else wants to use it in production, it will probably stay in its current state.

[mcsan]

Congratulations !

Ash seems interesting, I don’t know much about the framework but the book is something I would definitely be interested in

[AstonJ]

Congrats on the RC of Ash 3 It’s been fantastic seeing its progress here on the forum and you and the Ash team have done a great job onboarding people by answering their questions quickly and comprehensively

Also great to see you managed to get Dave to publish an Ash book for us all! I am sure it will be very popular!!

It’s also fantastic seeing the Elixir eco-system expanding with projects like Ash - it marks a new level of maturity for the language and I am sure will help with the next phase of adoption. So thank you and the Ash team (and all other similar projects) for your hard work

[lardcanoe]

Just upgraded my codebase from 2 to 3.0 RC18 and figured I would leave feedback. Didn’t see any other thread for this. It took me 2 hours WITH EXTENSIVE TESTING for a codebase with ~20 resources spread out over 3 domains.

First off, WOW y’all changed a lot. Do you sleep?

I figured I would just upgrade without RTFM because I am a typical developer. That did not last long. Like 2 minutes. So just RTFM. Though, even after I read, I am pretty sure I am still not doing things the “ash” way, or as Zach intended.

For mix.ex, took the simple path and just pointed at github:
{:ash, github: “ash-project/ash”, override: true},
{:ash_postgres, github: “ash-project/ash_postgres”, override: true},
{:ash_authentication, github: “team-alembic/ash_authentication”, override: true},
{:ash_authentication_phoenix, github: “team-alembic/ash_authentication_phoenix”, override: true},

I love the change to domains.

Adding public? to pretty much everything was tedious, but fine. I didn’t realize I needed to add it to all my “belongs_to” as well, and that led to some broken pages. I am not using either json or graphql apis, so I am not worried (yet).

I have an “archived_at” field that should never be changed outside of a specific destroy action, and it had “private?” which no longer exists. Instead of figuring out how to deal with it, I simply removed that restriction, and moved on with life.

I added “default_accept :*” to all resources because it just seemed easier than figuring out how to match my existing code.

Again, added “require_atomic?” to pretty much all :update actions, because again, a few resources needed it, and i just didn’t want to figure out all that needed it, so I went the easy route and just did them all. Not what Zach intended, but I didn’t want to learn the intricacies of the atomic stuff.

COMPILING IS SIGNIFICANTLY SLOWER. Like painfully slow. If I change a resource, it used to be maybe 5s, and now I am usually over 20s waiting. My guess is some cyclic issue that I need to use the mix task to locate. Luckily, resources don’t change too often.

Something in VScode now bombs on my most complex resource, “deadlocked waiting on module MyApp.MyModule” (renamed here) and this never happened before. I close VScode and reopen.

If I forget to add a field to public or part of the accepted list, then I still get:

[error] GenServer #PID<0.5936.0> terminating
** (AshPhoenix.Form.NoFormConfigured) fields at path must be configured in the form to be used with inputs_for. For example:

which is the most useless error. It takes me a few min to realize the real issue. Would love if that finally is addressed.

I just want an easy way to load calculations of an embedded resource. External stuff shouldn’t know certain fields are “special” and need to be loaded. Am I missing some option on a calculation to load by default?

No joke, you need to retest literally everything in your app. Absolutely ever page, action, email, whatever. Literally everything was broken for me.

Great work though! Good luck everyone!

[zachdaniel]

Hey there! Thanks for the feedback! In the latest release candidate, I added a compatibility option to help with the upgrade. Couple things:

1. require_atomic?

config :ash, :require_atomic_by_default?, false

Its listed in the upgrade guide (in the rc after you started upgrading) as something to allow you to upgrade without worrying about that step.

2. public?: true

Agreed, it is tedious but worth it IMO.

3. default_accept :*

Yep, much easier to do that, and then go back and address if you feel like it

4. compile times

Let’s chat about this. Definitely want to address this. If this is something that is caused by the upgrade, we need to work it out. Things should be faster not slower, so please if you get some time, let’s run through what could be causing that.

5. no form configured

This is really interesting, I think that we could absolutely give a better error here. Specifically, we can add some hint text at the bottom, that checks

• is there a private attribute w/ this name
• is there a relationship w/ this name
• is there an argument w/ this name

and it can hint to you what the problem might be. I’ll look at that shortly.

6. loading calculations of embedded resources

Add this to your embedded resource:

changes do
  change load([:your, :things, :that, :always, :load])
end

preparations do
  prepare build(load: [:your, :things, :that, :always, :load])
end

EDIT: error hints for NoFormConfigured errors have been added, will be available in the next release of ash_phoenix improvement: add error hints for `NoFormConfigured` errors · ash-project/ash_phoenix@1f511de · GitHub

EDIT: oh, and adding require_atomic? false is totally cool Learn about atomics when you have the inclination/time. We’re also working on better docs that explain how they work. Check the rc docs for actions and changes to see more.

[frankdugan3]

@lardcanoe

I have an “archived_at” field that should never be changed outside of a specific destroy action, and it had “private?” which no longer exists. Instead of figuring out how to deal with it, I simply removed that restriction, and moved on with life.

I think Zach missed replying to this one, but public? is the inverse of private?. public? defaults to false, so you are automatically getting the equivalent of private?: true. You can explicitly set public?: false if you want, but it’s not needed.

[lardcanoe]

as always, thank you for responding!

i went to get some timing for my code on Ash 2.0, and when I switched back to the V3 stuff and recompiled I got a deadlock. After a few hours of narrowing down the culprit, it seems to be specifying the domain on an embedded resource. If I wipe out my _build folder and compile without the domain specified on the 2 embedded resources, things compile, but then obviously the forms that use those fail since a domain isn’t specified (even though the parent resource has a domain set!).

Once compile fails, I can go back to main resource, change the embedded attributes to be {:array, :map}, and compile works. i can then revert back to the embedded type {:array, MyEmbeddedResource} with a domain specified in it, and compile now works.

I don’t know the erlang/elixir compile process well, but I point this out because people should probably wipe out their _build folder before thinking things are ok!

happy to move triaging this to another location, discord? but i am kind dead in the water with ash 3 right now.

This makes no mention of the embedded resources, but they are it afaik:

== Compilation error in file lib/my_app/scheduling/main_resource.ex ==
** (CompileError) deadlocked waiting on module MyApp.Scheduling
    (spark 2.1.16) lib/spark/dsl/extension.ex:128: Spark.Dsl.Extension.persisted!/3
    (spark 2.1.16) lib/spark/dsl/extension.ex:282: Spark.Dsl.Extension.get_opt_config/3
    (spark 2.1.16) lib/spark/dsl/extension.ex:240: Spark.Dsl.Extension.fetch_opt/4
    (spark 2.1.16) lib/spark/dsl/extension.ex:208: Spark.Dsl.Extension.get_opt/5
    (ash 3.0.0-rc.18) lib/ash/actions/helpers.ex:181: Ash.Actions.Helpers.add_actor/3
    (ash 3.0.0-rc.18) lib/ash/actions/helpers.ex:111: Ash.Actions.Helpers.set_opts/3

== Compilation error in file lib/my_app/scheduling.ex ==
** (CompileError) deadlocked waiting on struct MyApp.Scheduling.MainResource
    (elixir 1.16.2) src/elixir_fn.erl:17: anonymous fn/4 in :elixir_fn.expand/4

Compilation failed because of a deadlock between files.
The following files depended on the following modules:

  lib/my_app/scheduling/main_resource.ex => MyApp.Scheduling
            lib/my_app/scheduling.ex => MyApp.Scheduling.MainResource

[zachdaniel]

There is no need to specify a domain for embedded resources. Does omitting that resolve the issue?

[zachdaniel]

Sorry, just read further. Okay, this is a bug in ash_phoenix then. Will fix!

[zachdaniel]

Okay, so I’ve just pushed what might be a fix up to ash_phoenix main. I can see places where we weren’t threading the domain down to nested forms that were created. I’ve cleaned all of those up, but don’t have time to test it myself until tomorrow. If you could please try out main

{:ash_phoenix, github: "ash-proejct/ash_phoenix", override: true} and see if that resolves your issue, that would be great.

If it does, let me know and I will cut a release first thing. If it doesn’t, please log an issue on the ash_phoenix GitHub repo.

It makes sense that the embedded resource declaring a domain would cause a circular compilation issue, but I hand’t thought of it before. I will add a compile time error tomorrow preventing the domain option from being present on embedded resources. The basic problem is domain -> resource -> embedded resource -> domain.

In Ash, resources have compile time dependencies on types, and not relationships. Since :embedded resources are treated as types, that causes this issue.

[lardcanoe]

Fixed! Thank you so much!

(I had a temp fix of just defining a domain for embedded resources )

Recompile times seem ok now. I editted a bunch rapidly to see what happens and they are back to at least feeling as long as Ash 2.

mix xref graph --format cycles is basically the same as from Ash 2.0

Just about every (or maybe every) resource has a cycle of length 3:

lib/my_app/domain/resource_a.ex
lib/my_app/domain.ex
lib/my_app/domain/resource_a.ex

Nothing can be done about that, right? Seems expected, and not sure it even matters.

[zachdaniel]

Actually, that is interesting… I think I can make that better Will let you know.

[zachdaniel]

Actually, I’m not seeing those same cycles. Do you have things like anonymous functions in your resources that call functions defined in your domain? Would be worth looking into Perhaps lets start a new topic for it though.

EDIT: @lardcanoe maybe try mix xref graph --format cycles --label compile? runtime cycles are not really a problem.

[zachdaniel]

Setting domain on an embedded resource is now a compile error, to prevent folks from accidentally having a confusing circular dependency.

[arcanemachine]

Reading about your experience makes me feel like the RC is pretty stable.

I’ve been waiting to take Ash for a proper spin for quite a while! I kicked the tires a bit when Ash v2 was newer but other projects ended up taking up all of my time.