Hey folks, having an odd issue with one of my Phoenix controllers. I’m serving files out of S3-compatible storage at paths like /storage//.. Because we may eventually want to authenticate access to documents, I’m not using pre-signed URLs with my storage subsystem (Minio) because those can be shared. So the controller retrieves the file from storage and serves it using ExAws.
The issue I’m hitting is that, for large files (77 MB in local testing) my Phoenix process locks up entirely, but only if I request the file from a browser session, and only after the first request concludes. This doesn’t happen for smaller files (12 MB or so). It also doesn’t happen via wget/curl. I can wget these files until I’m sick of hitting up-arrow/enter, and they download very fast, but the instant I load them in a browser, I lock up. Further, once the process locks, wget/curl don’t work. So it isn’t a browser issue. When I visit URLs in a browser, I’m hitting URLs like:
http://localhost:8080/storage/2cc4d1d4-8754-46c6-8c2f-5809fb6f7e6d/ig-ag302.mp3
I.e. I’m hitting the file directly, so this isn’t an issue with layouts or the broader rendering pipeline.
My assumption is that sessions are involved for some reason, but I can’t think of why this might be. I’m using database-backed sessions, so I can see the data they contain. Nothing looks out of the ordinary or particularly large. I’d also understand if 77 MB was being held in memory for some reason, but then hammering my server with wgets should trigger that far quicker than my browser does.
Any thoughts as to what may be happening here? I have a few options if I can’t make this work. I can expose Minio externally and redirect to pre-signed URLs with a short TTL, or maybe stream the file internally. But this is mostly working code, and I don’t much like solving problems without understanding them, so I’d rather not put in a fix without understanding why this is broken. Anyhow, my controller code:
defmodule ScribeWeb.StorageController do
use ScribeWeb, :controller
alias Scribe.Documents
def show(conn, %{"id" => id, "filename" => filename} = params) do
document = try do
Documents.get_document(id)
rescue
# May not need this anymore but it doesn't trigger in this scenario anyway
_ ->
conn
|> send_resp(:not_found, "")
end
case document do
nil ->
conn
|> send_resp(:not_found, "")
_document ->
bucket = Application.get_env(:waffle, :bucket)
case ExAws.request(ExAws.S3.get_object(bucket, "#{id}/#{filename}")) do
{:ok, %{body: body}} ->
extension = Path.extname(filename)
|> String.trim_leading(".")
conn = conn
|> put_resp_content_type(MIME.type(extension))
conn = if Map.has_key?(params, "download") do
conn
|> put_resp_header("content-disposition", "attachment; filename=\"#{filename}\"")
else
conn
end
conn
|> send_resp(200, body)
{:error, _} ->
conn
|> send_resp(:not_found, "Not found")
end
end
end
end
Thanks.