AshStorage - Attachment and file management for Ash Framework

zachdaniel

Creator of Ash

Introducing AshStorage! Attachment and file management that slots directly into your resources

I had hoped to get this to a releasable state before sharing. It’s not quite there yet but I’m announcing it anyway as I know the Ash community is waiting.

It isn’t on hex yet, but I’ve gotten quite a bit of the roadmap done and its in a decent place to accept contributions from the community, and for beta testers to give it a shot.

Examples:

defmodule MyApp.Post do
  use Ash.Resource,
    extensions: [AshStorage]

  storage do
    service {AshStorage.Service.S3, bucket: "my-app-uploads"}
    blob_resource MyApp.StorageBlob
    attachment_resource MyApp.StorageAttachment

    has_one_attached :cover_image do
      analyzer MyApp.ImageDimensions,
        write_attributes: [width: :image_width, height: :image_height]

      variant :thumbnail, {MyApp.Resize, width: 200, height: 200}, generate: :eager
      variant :hero, {MyApp.Resize, width: 1200, height: 630}, generate: :oban
      variant :webp, {MyApp.Convert, format: :webp}
    end

    has_many_attached :documents do
      analyzer MyApp.FileInfo
      analyzer MyApp.PdfPageCount, analyze: :oban

      variant :preview, MyApp.PdfThumbnail, generate: :oban
    end
  end

  attributes do
    uuid_primary_key :id
    attribute :title, :string, allow_nil?: false, public?: true
    attribute :image_width, :integer, public?: true
    attribute :image_height, :integer, public?: true
  end

  actions do
    create :create do
      accept [:title]
      argument :cover_image, Ash.Type.File, allow_nil?: true

      change {AshStorage.Changes.HandleFileArgument,
              argument: :cover_image, attachment: :cover_image}
    end
  end
end

defmodule MyApp.StorageBlob do
    use Ash.Resource,
      extensions: [AshStorage.BlobResource, AshOban]

    postgres do
      table "storage_blobs"
      repo MyApp.Repo
    end

    oban do
      triggers do
        trigger :run_pending_analyzers do
          action :run_pending_analyzers
          read_action :read
          where expr(pending_analyzers == true)
          scheduler_cron("* * * * *")
        end

        trigger :run_pending_variants do
          action :run_pending_variants
          read_action :read
          where expr(pending_variants == true)
          scheduler_cron("* * * * *")
        end

        trigger :purge_blob do
          action :purge_blob
          read_action :read
          where expr(pending_purge == true)
          scheduler_cron("* * * * *")
        end
      end
    end

    attributes do
      uuid_primary_key :id
    end
  end

  # Create with file — dimensions written to parent, thumbnail generated instantly
  post = Ash.create!(Post, %{title: "Hello", cover_image: upload})
  post.image_width  #=> 1920

  # Variant URLs — thumbnail ready, hero generating in background
  post = Ash.load!(post, [:cover_image_thumbnail_url, :cover_image_hero_url])
  post.cover_image_thumbnail_url  #=> "https://my-app-uploads.s3.amazonaws.com/..."

  # On-demand — webp variant generated on first request
  post = Ash.load!(post, :cover_image_webp_url)

https://github.com/ash-project/ash_storage

8 comments

/ash #library #file-uploading

36 2017 8

2026-07-15 10:20:03 UTC

Showing Posts 1 to 8

Show Best Posts
Show All (oldest first)
Show All (newest first)

pierrelegall

I was working on a non-Ash Elixir package to manage S3 use for dev env (stored in the filesystem) and test env (stored in an ETS). See: GitHub - pierrelegall/ps3: S3-compatible storage plug for Elixir/Phoenix applications designed for dev & test environments. · GitHub

Thank you to solve this the Ash way!

I’ll be happy to beta test it as soon as possible

Post #3

munksgaard

Really cool! Any plans/ideas for something like ash_storage_phoenix for making handling files in forms easier, or what’s the plan there?

Post #4

zachdaniel

Creator of Ash

I don’t think anything special would be needed here. The :file type supports phoenix uploads already, and AshStorage can work with those arguments. What kind of thing do you have in mind?

Post #5

munksgaard

Admittedly, I haven’t tried ash_storage yet, so maybe it does just work as you suggest. I just found that for our existing upload functionality (ash + liveview), there is a lot of “ceremony” around handling and managing file uploads e.g. associated with a blog post.

Post #6

hwuethrich

Thanks, @zachdaniel! Having used Rails and ActiveStorage back in the days, I was really hoping for a similar solution that integrates with Ash and it is already looking very promising!

I always wanted to make an Ash storage library that uses PG Large Objects, because it allows accessing data from multiple nodes without adding additional services like S3.

Thanks to you, this got a lot easier and I’m happy to share our AshStorage service for PG Large Objects:
https://github.com/zebbra/ash_storage_pglo

One advantage of using PG is that creating the blob/attachment and storing the data is sharing the same transation and a rollback will clean up everything. This means it also works very well in async tests. But people should also be aware of some limitations.

I’ll await the first official release of AshStorage before publishing it to hex. In the meantime, feedback is very welcome!

Some things I already noticed:

All service options are stored on the blob in the resource. Some options (like base_url I copied from the disk service) have to be updated when changed. I’m not sure this really has to be persisted of could be defined elsewhere (eg. application env).
When using AshStorage.Plug.Proxy, each request will have to read the data from the database. While this is one of the drawbacks of using PG LO, it could be mitigated by adding caching to the proxy (etag, local disk, etc). Is there already something on the roadmap in this regards? I haven’t tried it, but this could probably be solved by using GitHub - tanguilp/plug_http_cache: An Elixir plug that caches HTTP responses · GitHub.

Anyway, thanks a lot for adding another piece to the Ash puzzle!

Post #7

zachdaniel

Creator of Ash

This is very cool! I definitely think caching is something we have yet to really tackle but ideally could be done at the resource layer itself to solve it generically. That is a huge project though in all honesty, so we’ll have to see

The base_url is typically the base url of the target storage, if that was updated the files themselves wouldn’t necessarily change their actual locations right? i.e if you configured a new one, you’d need to decide whether or not to update them for example. i.e did you move the images, or are just using a new location from here on.

But definitely worth exploring to find whatever is the most ergonomic solution there

Post #8

hwuethrich

FYI, I just managed to get caching (client- and server-side) working with plug_http_cache:

{:plug_http_cache, "~> 0.4.0"},
{:http_cache, "~> 0.4.0"},
{:plug_cache_control, "~> 1.1"},
{:http_cache_store_disk, "~> 0.3.0"} # or http_cache_store_memory

config.ex:

config :http_cache_store_disk,
  cache_dir: "priv/cache",
  memory_limit: 256 * 1024 * 1024

Note: Had to use fixed memory_limit because my Mac is running low on memory, and the store rejects if memory >70% by default.

In my router.ex:

pipeline :cache do
  plug PlugHTTPCache, store: :http_cache_store_disk
  plug PlugCacheControl, directives: [:public, max_age: {24, :hour}]
end

scope "/" do
  pipe_through [:browser, :cache]

  forward "/storage", AshStorage.Plug.Proxy,
    service:
      {AshStoragePGLO.Service,
       lo_resource: AshStorageTest.Gallery.StorageLO, base_url: "/storage"}
end

Ideally, the proxy should set etag/date headers based on the blob meta data (maybe using plug_http_validator).

Could also be useful when using proxy with S3 and other external services.

Post #9

kevinschweikert

Hey @zachdaniel, i am currently evaluating ash_storage. Our use case would the to transcode videos to HLS streams. But if i understand correctly, variants map 1:1 file-wise. HLS produce a tree of files referencing each other by relative path (master playlist → rendition playlists → segments).

Also it would be great if we could stream the files to the analyzers and variants or reuse the same file for all pipelines. The videos can we very big and it would be better to keep them in storage instead of RAM.

Is there a solution today for this use case? Is this something ash_storage will support in the future?

Post #12

— All posts loaded —