Problem serving multiple dynamic images with Phoenix

Phoenix Forum Questions / Help
, ,
1

I am trying to serve images that the user uploaded. then in an “overview page”, i want to put thumbnail-ish images in the html, which load sometimes and sometimes not.

Quick demo with a gif → gif doesn’t work apparently…:
ftr-image-bug
gif just emphasizes that when you refreshes, sometimes image 1 doesn’t work and the other one works, sometimes they work both and sometimes none.

When I go to the image link (/cases/image/1 for example), it loads every time. It is only on the page in the gif that it fails. When I get the failed image icon, the following error occurs:

[info] GET /case/image/2
[error] #PID<0.6379.0> running FirsttimerightDashboardWeb.Endpoint (connection #PID<0.6378.0>, stream id 1) terminated
Server: localhost:4000 (http)
Request: GET /case/image/1
** (exit) an exception was raised:
    ** (File.Error) could not read file stats "uploads/images/1-user_upload_3.png": no such file or directory

Yet the file is there. The information in the database is also correct (as far as i know of course…).
DB output:

image
image919×251 25.9 KB

The images are also in the folder with the correct name.

Also tried with ctrl + r/f5 force purge cache reload, though it doesn’t seem to help.

Thank you in advance.

2

Hi! The server says ‘uploads/images/1-user_upload_3.png’ isn’t found, but it’s called ‘user_upload_3.png’ in the db it looks like? (1- prefix missing). Not sure if that’s it? What’s the filename in the uploads dir?

3

to avoid duplicate file names i prefix the index of the database, thus 2 files that are called the same will become 1-file.png and 2-file.png.

Apologies for the lack of explanation

Code that is being used:

  def local_path(%CaseImage{} = upload) do
    [@upload_directory, "#{upload.id}-#{upload.filename}"] |> Path.join()
  end
4

Maybe with just the information above it is quite hard for people to help, here is a sample repository reproducing the same issue: GitHub - WannesFransen1994/phoenix-dynamic-images: serve dynamic images with phoenix

once again some sample screenshots (same page reloaded twice):

samplescreenshot1
samplescreenshot1400×603 48 KB
samplescreenshot2

relevant code:
controller -

def index(conn, _params) do
    images = Repo.all(Image)
    render(conn, "index.html", images: images)
  end

  def create(conn, %{"upload" => %Plug.Upload{} = up}) do
    {:ok, _u} = up |> ImageContext.create_image()
    redirect(conn, to: Routes.page_path(conn, :index))
  end

  def display(conn, %{"id" => id}) do
    i = Repo.get(Image, id)
    conn |> put_resp_content_type(i.content_type) |> send_file(200, Image.local_path(i))
  end

imagecontext -

def create_image(%{filename: _, path: tmp_path, content_type: _} = upload) do
    hash = File.stream!(tmp_path, [], 2048) |> Image.sha256()

    with {:ok, %File.Stat{size: size}} <- File.stat(tmp_path),
         data_merged <- Map.from_struct(upload) |> Map.merge(%{size: size, hash: hash}),
         {:ok, upload_cs} <- %Image{} |> Image.changeset(data_merged) |> Repo.insert(),
         :ok <- tmp_path |> File.cp(Image.local_path(upload_cs)) do
      {:ok, upload_cs}
    else
      {:error, reason} -> Repo.rollback(reason)
    end
  end

Image -

schema "images" do
    field :filename, :string
    field :content_type, :string
    field :hash, :string
    field :size, :integer
  end
def local_path(%Image{} = upload) do
    [@upload_directory, "#{upload.id}-#{upload.filename}"] |> Path.join()
  end
5

Solution was to work with absolute paths.

Apparently Phoenix sometimes changes the current working directory, for example when code reloading, and thus the relative path fails.

Credit goes to Nobbz (Slack) and Jose Valim (Git issue).

2 Likes
6

Nice Functionality you got there but,
With this code can i be able to insert multiple images in one database field.?

7

Hi Jetmush,

I really didn’t expect someone else looking at this code after a year ^^’ After looking at it again, the version you saw only supported one file at a time. I quickly adjusted this (it really isn’t much work! Just add the ‘multiple: true’ option in your template and you’ll receive an array of plug.Upload structs.)

After uploading 2 files, this is what you should see in a IEx.pry’d controller action:

pry(1)> up
[       
  %Plug.Upload{
    content_type: "application/octet-stream",
    filename: "bcm2708-rpi-b.dtb",
    path: "/tmp/plug-1595/multipart-1595601713-270993472630691-4"
  },
  %Plug.Upload{
    content_type: "application/octet-stream",
    filename: "bcm2708-rpi-b-plus.dtb", 
    path: "/tmp/plug-1595/multipart-1595601713-367099312190116-4"
  }
]

I left the pry in the repo so that you can clone, compile, run and see what happens. Now specifically for your question regarding:

multiple images in one database field.

I prefer storing the images on a AWS S3, or on the file system and store a path in the database. Since an upload is mostly one file, I’d implement uploading multiple files as multiple single uploads (and thus multiple database entries). Is it important that you keep track of the fact which files were uploaded in a single batch? If so, I’d personally suggest either one of:

  • Add a new table, e.g. “batch upload”. Then you can add a FK in your uploads as to which “batch upload” they belong.
  • I’d personally rather not suggest this, though I cannot put my finger on to the “why” part, but you could separate your file names with a unique separator. After then reading the entry, split on that separator and you have a list of filenames. You might resolve this with saving it as a JSON entry I think, but then what when you e.g. periodically check the integrity of files and one file is corrupt / no longer present?

With the second approach it seems to me a lot can go wrong and should be solved with code (e.g. the removal of a file means a bunch of code, while in essence it could be just one small table entry deletion). Hence my preference for the first solution, though I’d recommend this with URL’s to AWS S3 (definitely more scalable :wink: )

Note: whole rant above is if you’d want to keep track of batch uploads. If this is not necessary, I’d suggest not to add unnecessary complexity and see each file as a separate upload in its respective table.

In case other people read this and find this absolute blasphemy, please do tell me! Always eager to learn!!!

Hope this helped!

1 Like