How serving static files to 10K client

Questions / Help
,
1

My static files is png/jpeg and many
video ( under 20mb )

Each client download one times

I not interested in amazon service (s3)

I read ngnix benchmark from itself website .

It can handle this load in single machine
with 8core and 16Ram.

Is it good solution or exist another ?

1 Like
2

BackBlaze + Cloudflare seems to be quite popular these days and should be very cost effective.

2 Likes
3

Its a CDN ??

4

Cloudflare is a CDN. BackBlaze is a cloud storage.

1 Like
5

In my country have a issue to using international service . We just can use cloud service of our country ,

Here exist a great CDN but
Not exist a good cloud storage.

For that reason I must use a open source
Software like Ngnix or GridFS or …

6

Since you mentioned nginx, why don’t you just start with that, assuming you are familiar with it.
Performance-wise it should be as good as any. The only weakness of nginx is that it is hard to do run time reconfiguration.

7

Another option is MinIO. MinIO supports Amazon’s S3 API and works with waffle & ex_aws_s3.

I’ve successfully used it as a simple self hosted CDN with the idea that if traffic and/or storage space ever exceeded comfortable levels, I could move to a cloud storage provider that supports the S3 API with basically what amounts to a configuration change.

3 Likes
8

Only downside of minio is that it doesn’t support ACLs, but only bucket/prefix level permissions.

9

because i want first sure then start .
then i think it is good for start, thank you

10

HayStack is facebook photo storage/serving

Seaweedfs use that architecture and
Have 13k github star

Do anyone used it ??

11

I have used nginx and it is better for serving static assets. Nginx served static assets to a lot more clients concurrently than you are looking for in production. You have to enable cache control and expiry headers in nginx. Also do some linux configurations about open files, max connections, etc and you can start serving static assets very efficiently.

Start with lowest possible hardware suitable for production and do a load testing for your assets. If it doesn’t perform well check whether it is a configuration issue or system resources like cpu, network are maxed out. If the system resources are maxed out - upgrade hardware to next level.

Once in production you can keep monitoring the server behaviour and tweak hardware.

I would go with nginx anyday for serving static assets if I am not using CDNs.

1 Like
12

Thanks but when my client increase.

Nginx have a solution for distributing
Or just can I used in single node for file serving ??

13

When you have to scale horizontally(add more servers) - you will have to use load balancer. Load balancer can distribute traffic based on different algorithms like round robin, connections, resource usage, etc.

Before you scale horizontally - its better to try out scaling vertically (increase the hardware of the current server).

Nginx will surprise with its performance on how much it can deliver given enough hardware. Only problem with nginx is understanding the configuration part of nginx(sometimes it becomes tricky). There are some tools out there which can help - NGINXConfig | DigitalOcean .

As with any production configuration - you have to be sure of what your are doing. “When in doubt - research more”. Try out your changes on a dev server or staging server.

3 Likes