I’m thinking of building a site similar to imgur or TinyPic for personal usage and learning. I don’t have any questions on how to do this with Elixir but I was curious about the infrastructure: how do these sites work? Obviously I can’t upload the actual file to a database, so I’m assuming I hash them and store that in the database… but where do the images go?
I’m also curious about cost and stuff. Like I said, this is for personal usage and I’ll probably upload less than 10mb a month, so hopefully it’ll be cheap.
For those not familiar, I want a site where I upload an image and get a link. Pretty simple.
I am currently working on a small side project which also involves uploading files. I put a path to my configuration file (config/config.exs) where I store all the data. The database entry for each uploaded document then has a “filename” field where the local filename is referenced. If you just want an image upload, you can leave all this aside - you don’t even need a database and can just serve the files directly out of the directory.
To avoid duplicate names, I use a very simple function that uses the upload filename (i.e. “image.png”), puts a random number in front of it and base64 encode it (you could just leave out the base64 encoding obviously). I know that I won’t store many files so this is enough for me.
I found this guide on uploading files very helpful: http://www.phoenixframework.org/docs/file-uploads
As for the pricing, you can get really cheap VPS from OVH, use the free AWS tier or spin up a $5 DigitalOcean droplet. At least that’s what I am using right now.
Why not use something like Arc to handle uploads?
A few more you might be interested in:
I’m using Arc, and it’s awesome. You should try it too!
Here is good tutorial for starting point : https://smashingboxes.com/blog/image-upload-in-phoenix/
@AstonJ @alzearafat I will use Arc! I mostly meant where do I store the images? This post is fairly language-agnostic and more about the architecture of the app.
Thanks @Rukenshia! I will look into this and your answer
I mostly meant where do I store the images?
Personally I’d used AWS S3; I use it for a site with a lot of large
images and my monthly bill is around $0.98 US
Ya especially since Arc already works with S3, you won’t need to build anything custom to work with another storage service.
I implemented uploading files with www.ibgib.com (GitHub), and I simply made a
/files folder where I store the images. I actually get a sha-256 hash of the image data to both avoid duplicating content, as well as providing unique filenames. In your database, you can simply store this hash.
My (non-ARC) upload code starts here, with the saving-specific code here. It also creates a thumbnail in there if you’re interested, but if you use ARC (which was a little complicated/overkill for me) then the implementation details would be different for you.
In the off chance that you use Docker when deploying it, you can set it up as a persistent volume as in my
docker-compose.yml. This makes it so that when you re-up the web server docker image, the volume persists. (Be sure to read up on the volume documentation for this )
As for cheap, there is an AWS free tier if you aren’t aware. It ends up being free for a year anyway, during which time you could maybe find somewhere else to host it.
Lot of answers! Thank you very much to everyone—love the Elixir community.
If you are doing this for learning, don’t use Arc. Store your files in a /data folder at the root of your application. Generate a UUID using https://github.com/zyro/elixir-uuid . Store the file as
data/uuid/filename.ext and just return a url similar to
http://yourdomain.com/image/uuid/filename.ext . Create a simple ImageController which serves images from that path using
I would store them as files in a /shared/uploads directory on the server (then configure your deployment tool so that it doesn’t overwrite the shared folder).
I wouldn’t store them in the DB - this will make db backups huge and slower than if simple files.
Arc should be able to configure the storage directory and naming conventions
Why this recommendation?
Generate a UUID using https://github.com/zyro/elixir-uuid1 .
Why not just use
Isn’t that better than taking on a dependency just for UUID’s?
how fast are you uploading new files that
:os.system_time is not unique enough?
In this particular instance (the fact that this is a personal project), system time would work.
However for more serious sites, I’d definitely reach for a uuid because it is standardized and can be stored nicely in a database and wouldn’t ever create a collision. I could definitely see a case of two users uploading at the same instance.
and imagine the day you have two different machines …