Integrating Ash into an existing project

Ash Forum Ash Questions / Help
,
1

I have a pretty large existing project and I want introduce Ash. I have existing schemas, migrations, etc. It’s a Phoenix project. The good news here is that the scope of the project touches really lightly on existing schemas. It’s not quiet greenfield, but close enough.

The documentation for Ash is… a lot. This is a good thing. I’ve watched some videos, gone through the LiveBook, read stuff all over the place (actual docs, blogs, etc.). I’m now at the place where the rubber meets the road and for Ash Postgres… it’s got its own repo, migrations (well sorta), etc. Is there some kind of guide one pitfalls, gotchas, best practices… maybe an unfinished blog article on the matter? Haha. (So it’s not lost in this post, that is the actual question.)

Longer Version for context:

I have been eyeballing Ash for at least a year. The system in question uses a library I put together ages ago (my first Elixir project actually) and some conventions to mimic the resource model. (We’re even running JSON:API.) When I dug into Ash… “OMG. This is basically an awesome version of what I was doing put together by someone who was able to articulate the the problem I was trying to solve much better than I was!”

My long term goal is to see how we like it in reality on this project and, if it works as well as I expect, put together a plan to replace our custom stuff with Ash. Not only does it appear to be far superior, but I think the work involved in our backporting–though in no way trivial–will pay far greater dividends than trying to fix/improve/better document our own stuff. Plus we get a community behind us rather than working in a proprietary silo.

Anyway, this is both my goal and frankly, who doesn’t love getting to mess with something new in their professional life? Any info before I actually start coding would be appreciated.

6 Likes
2

I don’t believe that there is any particular guide for this :thinking:

I’m happy to use this thread to answer questions on this regard. The most important thing to keep in mind is:

You can turn your Ecto repo into an AshPostgres.Repo and still use it with Ecto :slight_smile: .
Ash’s generated migrations are also standard ecto migrations meaning that they should be compatible as-is. Because you already have some database structure, you may end up generating migrations and then deleting or hand-modifying the generated migrations sometimes etc.

You may also be interested in the task to bootstrap a suite of Ash resources from a postgres database:

mix ash_postgres.gen.resource MyApp.MyDomain

https://hexdocs.pm/ash_postgres/Mix.Tasks.AshPostgres.Gen.Resources.html

3 Likes
3

Having done this migration on a large ERP with tons of schemas, I can advise these steps:

  1. Prefix the Ash stuff you add, like MyApp.Ash.[...]. This makes it easy to namespace Ecto schemas vs Ash resources. They can coexist while you migrate. If you are depending on PubSub, you’ll need to do a little bit of cross-notification integration between the two in places where the app is using the same schema/resource in both paradigms. Personally, I found it easier in general to avoid that by just migrating an entire context at once. YMMV.

  2. Implement all new features in Ash.

  3. As you migrate from your old contexts, you can gradually remove your deprecated modules when they are no longer used.

  4. Once all your old modules are gone, you can do a little search & replace to get rid of the .Ash. namespace if you want.

Note: Pay close attention to Zach’s advice about the migration generator. It is very worth using, just make sure you delete anything that was already done in your previous migrations.

5 Likes
4

That is pretty much exactly what I’m doing in my codebase. There are places, as an example the graphql resolvers, where I have started to call functions that are defined in Ash resources and things like that without introducing the full blown AshGraphql stuff.

1 Like
5

That instantly made this simpler. Okay. Just changed out the use at the top and things just went right on truckin’. Test suite passed so… what more can I ask for?

Got a warning about min_pg_version not being there but that was easy enough to solve.

We have lift off!

I’m presuming there is a bug in your response and in the docs. Should it be:

mix ash_postgres.gen.resources MyApp.MyDomain

The above command does something. I have no idea what though. It just hangs on my machine. I tried with --snapshots-only as well. No dice.

As an aside, as I go through this, I would like to instruct my team to do at the outset, especially if we need to come here or to the Discord for clarification, is to update the docs where we see gaps, or at least just submit PRs there. I’m going to guess no one would be sad about that?

Anyway, after getting stuck last night on this I fired up a vanilla Phoenix app and went through the Getting Started Guide to get a sense of what Ash would inject at the outset.

mix ash.setup

That appears to just create the database. It doesn’t make any code changes.

mix ash.codegen initial_migration

This, on the other hand, does a bunch. I have a migration for extensions and functions. I presume I would want to port that into my existing project. (Or get it there some other way.) There’s an initial Blog migration which is cool, that matches the attributes.

Under repo I see a JSON file for extensions and a file defining Posts which I’m guessing is the secret sauce for creating differential migrations as we go?

Anyway, I went and ran the same stuff on my existing project and…

Voila! (I think.) The initial migration to create all the base functions is there and I have a migration for the one resource I created. That’s definitely a start. I have a couple more questions about interacting with existing data which I’ll post separately.

This might be the path I take when we start back porting. Fortunately, at the start of this I’m working with mostly new stuff. My User and an access list resource are some of the only things that need to exist in both worlds. (Although… I guess if I even one right now, I should adopt this, lol.)

6

What it’s meant to do is create a resource-per-table. Please open a bug about your experience with it. I understand that you will not be able to provide much info given that it just hangs for you, but that helps me track what needs to be fixed.

7

Is there anything I can do? A log? Debugging mode? Something to give you more info?

8

So, I ran the following on my current app:

mix ash.setup
mix ash.codegen initial_migration

This created that initial migration with the functions and it also created a migration for the one resource I created by hand.

I presume this is fine to start. (The generated migration is sweet. The defaults are inferred. The timestamp generation is in the database. You give me a UUID7 function for free in the database. This is exactly the kind of stuff that I was hoping to see and it’s really, really slick.)

Is there a document on the files generated in /priv/repo/<domain>/*? I presume this is how Ash figures out how to handle diffs on the related migrations after changes. I figure grokking this will probably give me the tools I need to really understand how I work in stuff outside of Ash and how I ultimately bring my non-Ash stuff into the fold.

Are the timestamps on the migration for the resource and the JSON file linked in some way or is that simply a byproduct of them being created at the same time?

1 Like
9

The snapshot files are considered a “private” (insofar as it is not specified and may change at anytime) format. You’re correct as to what it’s for.

In the future, however, we’re actually going to redesign that system to instead be operations and it will be emitted by core. This will allow any extension to react to changes in resources for any reason, instead of the way it currently works. that will be a specified set of data structures. i.e %EntityAdded{path: [:attributes], entity: %Attribute{}}. Then each extension will just keep a list of which operations it has handled. It will probably be quite some time before we get to that however.

The timestamps are only related by happenstance. Once the migrations are generated, and the snapshots are saved, they are unrelated. The intent there is that you could modify them however you liked after the fact. Sometimes you even must edit them, if they aren’t doing what you need. The snapshots should not be edited though. The only thing you might do with snapshots is delete them, to regenerate the migrations for whatever they represent.

10

Actually. It’s just really slow.

Igniter:

    No proposed content changes!

The following tasks will be run

* ash_postgres.generate_migrations import_resources

...

Extension Migrations: 
No extensions to install

Generating Tenant Migrations: 

Generating Migrations:
No changes detected, so no migrations or snapshots have been created.

That’s what I ended up with.

11

This is a good question. Do you see anything at all? Any log output? You could set your log level to :debug to potentially get more information? Is your schema that you’re using private? You could email me a schema only pg dump which I could test out myself. What version of postgres are you running locally?

12

Okay great. That makes sense enough to understand the relationship between the migrations and the resources. I can even generate brand spanking new migrations for non-Ash stuff during the conversion, compare the actual structure and hand tool as needed.

Fortunately, I’m not in deep enough to need the redesign!

Anyway, thanks for the quick responses. I’ll get outta your hair for now. I need to work on a fairly simple resource for a bit in this context and see where I get stumped. Hopefully I get some basic authorization working with my existing system without too much pain.

1 Like
13

Hey!

Just to answer your first post bold question:

Is there some kind of guide one pitfalls, gotchas, best practices… maybe an unfinished blog article on the matter?

I’ve written a few months ago a series of articles about my own experience of migrating a Phoenix project to Ash.

Here is the link: Migration to Ash series | Blog of the (p)repair project

It covers with some details and complete code example the migration process from an Ecto Schema to an Ash Resource.

I wrote it for me, as a way to validate my comprehension of the process and differences between frameworks approaches. I also wrote it with the goal to share it with the community. Because I spent a lot of time during last spring (April and May 2024) on the Ash Discord, and I received many help from Ash developers and users :heart:.

Finally I had to switch my life project and to find a new job to get revenue (my first job as a programmer, and unfortunately it’s not in Elixir but PHP and JavaScript, but now that I start to get used of these “new” languages, I’m switching back to my (p)repair project on my free time). Well I guess, that’s out of the current topic :grinning:.

Hope these articles can help new Ash adopters which would came from Phoenix !

7 Likes