Video hosting, transcoding and delivery

Questions / Help
,
1

Hi all,

First of all, please remove this post if it’s no appropriate for the forum. I am using Elixir / Phoenix to power my app and some libraries might be available in Elixir to help me with what I’m doing, but the similarities stop there.

I run an e-sports coaching app. It allows coaches to create their own website and have users upload videos to it for feedback. Coaches can comment at various points in the video with what you’re doing wrong / right. If this is a bit hard to conceptualize (I don’t blame you!) a good example of once of the coaches using the system is https://kilk.vodon.gg/

I’m at the tail end of a fairly reasonable rewrite that was required to enable a few features that weren’t possible in V1 and that rewrite required me to shift the video hosting from YouTube to Cloudflare Stream. Cloudflare Stream is performing well but I’m a little nervous with the pricing around it. I’m also not quite sure if it’s a 100% fit with what I want to do.

I already use Digital Ocean’s Spaces feature for hosting avatars and other images uploaded from the site and I’m thinking about using it for video delivery. However, I’m not quite up to speed on what’s required for modern video hosting and I want to make sure that I’m fully understanding things before I start to write any code.

  1. Hosting the video on Spaces would mean I give up things like Stream’s automatic transcoding for different formats and HLS/DASH delivery. I think that’s actually ok, I’m happy to deliver just one resolution for all users (ideally 1080 or 4k, 30 or 60fps). This would mean giving up adaptive bitrates too. However, I’m happy with the video not downgrading for slow connections even if it means additional buffering. Visual fidelity is key in my content and delivering a blurry video is worse than no video.

  2. As I understand the modern HTML video tag, I need to generate a couple of different formats to support the majority of browsers, WebM/VP8 and MP4/H.264 (although it appears that Safari now has support for WebM so I might be able to get away with just one encoding?)

  3. I want to understand how I would model the flow of uploads from a user. Currently with Cloudflare Stream I can perform a client upload where the users browser sends the data directly to Cloudflare and I simply wait on a webhook to tell me that the video is ready to be served. With a Spaces approach am I right in thinking it’s going to go something like:

a. Client fetches a presigned URL from the app to perform the upload
b. Client performs the upload directly to spaces
c. Client tells me that the upload is complete
d. I perform some kind of transcoding job on the uploaded video.
e. Once transcoding is complete, I update the DB record to indicate the video is ready to be served.

  1. Step D is a bit unknown for me as well. I assume I have to have a tool that would download the video from spaces, transcode to the required format(s) and upload again to spaces. Once this is complete I’d then inform that main service that the video is ready for streaming.

This step sounds hard. If I’m thinking about it correctly, can any recommend a library, service, application that is designed for doing this.

  1. Aside from losing the adaptive bitstream of the video by hosting on Spaces, what else might I be giving up that I don’t expect? Does “seeking” in the video still work for example?
1 Like
2

Take a look at Mux.com - they handle all the transcoding for different consumers of video. Last time I played with it, getting custom security tokens etc was pretty straightforward and Elixir is well supported (I believe they use a fair bit of Elixir in their stack).

Pricing is very transparent, too. You will quickly be able to figure out if it fits your budget.

3

Don’t do it, their object storage isn’t good enough, the access time was terrible (at least it was in october last year) and that’s a death sentence for videos, I’ve put the files on backblaze b2 (S3 compatible) and a (paid) cdn at the front, bunnycdn was what I used, it worked flawlessly and was cheap enough, I used a service to do the transcoding/encryption and upload the files straight to the object storage (qencode), but I could use open source tools to do it too shakaencoder and shaka streamer can do the heavy work for you, but qencode was cheap enough for us to not worry about setting up our own infrastructure to do it.

Do not trust the user browser, even though iphone users usually are in more recent versions, you’ll definitely hit some edge cases of people with old safari versions, you can decide to ignore safari and go with vp8, but if you want to do only one format, go with h.264 it is supported everywhere (be aware of licensing issues, if you’re small they probably won’t care, but it is a thing to worry about).

Your step by step is exactly what we did, people uploaded the videos through our page direct to the object storage, it created an entry in the db with a status, we triggered the transcoding manually because there were only a couple dozen videos, but you can easily write a task that triggers as soon as the upload is complete.

As for step D, it depends on the scale and business model to see what is economically viable, it is not hard to write a python script that read tasks from a queue (redis or rabbitMQ for example), download the file from the bucket and uses shaka streamer to do the transcoding to the desired formats and reupload to another bucket, a prototype should not take more than a few hours or a day to someone with reasonable experience.

The problem is the scale, transcoding video is usually heavy, so to avoid unnecessary costs with idle powerful (and expensive) servers you have to have a way to provision these servers on the fly, this means k8s or some other tool/cloud provider to do it, which is another layer of complexity to the problem.

In my last project with video I used qencode because the transparent pricing and no minimum commitment are great and because the economics of the project didn’t allow for services like cloudflare stream or mux (that are great, but I wouldn’t use them for video that has to be served for free or really cheap).

As I said, don’t use DO spaces, the performance isn’t good enough for video, you can seek the video because byte-range requests are allowed, but the performance will make it a terrible experience, use other object storage, there are a ton to choose from, and depending on the use case, put a CDN in front of the object storage.

Any more questions about video you can ask in this post or feel free to send me a DM.

6 Likes
4

I’ve been in touch with Mux already, and while I love the product and the team, I’m a bit concerned about the pricing :grimacing:

5

Happy to pay for a service to handle the transcoding and Qencode looks great, I’ll check them out in a bit more detail now.

What was the flow you used with Qencode / Backblaze / Bunny? I’m guessing something like:

  1. User uploads to Backblaze using presigned URL
  2. Trigger Qencode to fetch the file from Backblaze, transcode and save the converted video back to Backblaze again
  3. Serve the file via Bunny CDN (point the CDN to Backblaze then serve it via the bunny URL?). Any special steps here required to tell Bunny about the upload or it’s smart enough to just map to the contents of the Backblaze bucket?

This is great feedback btw, thanks for much for posting it.

EDIT: I see that they can perform direct uploads to Qencode and I can set a storage destination of almost whatever I want. Alright, I think I’ve got it!

6

I’ve used AWS Elemental MediaConvert and it’s pretty cheap. it’s not too hard to make a lambda that will watch a bucket dir and just convert an mp4 that lands in there.

This github repo is a terraform module GitHub - trackit/aws-workflow-video-on-demand: AWS Workflow for video-on-demand with automated API that deploys a pre-made mediaconvert job.

7

The flow was more or less what you posted with a bunch of steps in the middle because the users of the project weren’t tech-savvy so we wanted ways to intervene and fix any mistakes before they propagated.

As for the cdn, they have docs that explain how to integrate b2 and and their product.

8

You could try cloudinary too.
It manages images and vidéos.