I’m thinking of building a site similar to imgur or TinyPic for personal usage and learning. I don’t have any questions on how to do this with Elixir but I was curious about the infrastructure: how do these sites work? Obviously I can’t upload the actual file to a database, so I’m assuming I hash them and store that in the database… but where do the images go?
I’m also curious about cost and stuff. Like I said, this is for personal usage and I’ll probably upload less than 10mb a month, so hopefully it’ll be cheap.
For those not familiar, I want a site where I upload an image and get a link. Pretty simple.
I am currently working on a small side project which also involves uploading files. I put a path to my configuration file (config/config.exs) where I store all the data. The database entry for each uploaded document then has a “filename” field where the local filename is referenced. If you just want an image upload, you can leave all this aside - you don’t even need a database and can just serve the files directly out of the directory.
To avoid duplicate names, I use a very simple function that uses the upload filename (i.e. “image.png”), puts a random number in front of it and base64 encode it (you could just leave out the base64 encoding obviously). I know that I won’t store many files so this is enough for me.
As for the pricing, you can get really cheap VPS from OVH, use the free AWS tier or spin up a $5 DigitalOcean droplet. At least that’s what I am using right now.
@AstonJ@alzearafat I will use Arc! I mostly meant where do I store the images? This post is fairly language-agnostic and more about the architecture of the app.
Thanks @Rukenshia! I will look into this and your answer
I implemented uploading files with www.ibgib.com (GitHub), and I simply made a /files folder where I store the images. I actually get a sha-256 hash of the image data to both avoid duplicating content, as well as providing unique filenames. In your database, you can simply store this hash.
My (non-ARC) upload code starts here, with the saving-specific code here. It also creates a thumbnail in there if you’re interested, but if you use ARC (which was a little complicated/overkill for me) then the implementation details would be different for you.
In the off chance that you use Docker when deploying it, you can set it up as a persistent volume as in my docker-compose.yml. This makes it so that when you re-up the web server docker image, the volume persists. (Be sure to read up on the volume documentation for this )
As for cheap, there is an AWS free tier if you aren’t aware. It ends up being free for a year anyway, during which time you could maybe find somewhere else to host it.
If you are doing this for learning, don’t use Arc. Store your files in a /data folder at the root of your application. Generate a UUID using https://github.com/zyro/elixir-uuid . Store the file as data/uuid/filename.ext and just return a url similar to http://yourdomain.com/image/uuid/filename.ext . Create a simple ImageController which serves images from that path using send_file.
I would store them as files in a /shared/uploads directory on the server (then configure your deployment tool so that it doesn’t overwrite the shared folder).
I wouldn’t store them in the DB - this will make db backups huge and slower than if simple files.
Arc should be able to configure the storage directory and naming conventions
In this particular instance (the fact that this is a personal project), system time would work.
However for more serious sites, I’d definitely reach for a uuid because it is standardized and can be stored nicely in a database and wouldn’t ever create a collision. I could definitely see a case of two users uploading at the same instance.
)
