How do you handle user uploads?

Hi all,

I would be very interested in how are you all handling file uploads right now in your Phoenix projects? Do you use live uploads with LiveView or something else? Do you do some checking/processing? What was your pain point?

We have a couple of Phoenix apps - both use AWS S3 presigned URLs and the client uploads directly to S3. A Phoenix controller is used to generate the presigned URL and the Ecto model stores the S3 key (once the user submits the form).

4 Likes

In one project I do not use live upload with liveView. I just chop up the data in chunk, base64 encode them then pushEvent them over. The reason is I want to do in-line processing on the data so I don’t like the data to be written out to a file just to be read back in.

This method is obviously wasting bandwidth; I wish liveView supports native binary data in event message though. Or I could use a separate socket but so far my need is modest so I have not bother.

2 Likes

I will also write for myself & the app we are working on. We don’t have user uploads for now, so we only have two modules - one to save files we get or generate to Azure storage and one to generate pre-signed URLs.

So for now I see people approach this very minimalistic. I think the new LiveView feature looks exiting, but then the whole page has to be LiveView, right? You cannot combine regular pages with LiveView upload or can you?

No You cannot, You need the page to be live…

You might need to define what to do with the uploaded files. I like to use waffle and waffle_ecto for this part.

Yep, I thought that much. Thanks for the confirmation.

I also looked briefly into Waffle, I don’t think there is anything more robust right now for Elixir. If someone knows about something, let me know.

This is something that I’m really concerned about, as an Elixir/Phoenix newbie.

What’s the best file upload library? Arc looks unmaintained, Waffle looks good but I’m not sure it’s production ready.

Would be cool to have a singular library to handle this in the ecosystem, like Active Storage. Would fill a gap!

Would you might showing the request/responses in the work flow a bit? Does the user initiate a request which generates the presigned url, then they upload the file, then when the file upload is done they report back to another Phoenix endpoint?

Pretty much that, yes.

The user is posting an Event that has the following model:

schema "events" do
    field :device_id, :string
    ...
    field :media, {:array, :string}, default: []

    timestamps()
end

Before posting the event, the user asks for a presigned PUT URL, uploads the file and adds the key to the media array.

Whenever, the event is rendered, a new presigned GET URL is generated. See code samples below:

def put_file(conn, %{"key" => key}) do
    user_id = conn.assigns.current_user.id
    uuid = Ecto.UUID.generate()
    key = "#{uuid}.#{ext(key)}"
    render(conn, "media.json", key: key, url: generate_presigned_url(user_id, :put, key))
end

def get_file(conn, %{"key" => key}) do
    user_id = conn.assigns.current_user.id
    render(conn, "media.json", key: key, url: generate_presigned_url(user_id, :get, key))
end

def generate_presigned_url(user_id, method, key) do
    {:ok, url} = S3.presigned_url(config(), method, "#{@folder}/#{user_id}", key)
    url
end

We are using ex_aws_s3.

1 Like

Waffle is a maintained fork of Arc :slight_smile:

4 Likes