Video streaming: the design, step by step

Part of the free System Design course on LogicWiz, in the chapter: Design video streaming. The interactive whiteboard exercises and simulations run in the app.

Week 15: From one server to a video pipeline

"Our first video demo was one server and one folder of MP4 files. It worked for the six people in the meeting room."

Kabir, on SwiftKart's first recipe-video experiment

Version one, and the number that breaks it

The simplest design: a creator saves a file on the server's disk, and the viewer's app asks the same server for that file. Check it against the numbers from lesson 1.

A server with a 10 Gbps network card sends 10,000 Mbps. Each stream takes 3 Mbps, so one server feeds 10,000 / 3 = about 3,300 viewers. For 5 million viewers at the peak we would need about 1,500 servers, and each would need a copy of the popular files or a way to find them. A second problem: one file means one quality. A 6 Mbps file on a 1 Mbps train link stalls again and again.

Meera assumed a video is one file. After processing, it is a set of many small files in several qualities. Three ideas fix version one:

  • Do the heavy work once, at upload, in a pipeline.
  • Cut each video into small pieces in several qualities, and let the player choose.
  • Keep copies of popular pieces near viewers.

Upload: store the original, then queue the work

The creator's app asks the API for an upload. The API replies with signed URLs, temporary links that allow one upload to object storage. Object storage is a store for large files addressed by a key, with copies kept in several places. The file goes up in parts of about 8 MB. If the connection drops, the app resends only the missing parts. Parts go straight to storage, so API servers never carry them. Lesson 1 showed why: 7 uploads a second x 600 MB is about 4.2 GB a second.

When the last part arrives, the app calls "complete". The upload service marks the video as processing, puts a message on a queue (a line of jobs that workers pick up) and replies 202.

Transcoding: one source, many versions, in parallel

Transcoding means converting a video to another size and bitrate. A 10-minute video must become 5 renditions. If one machine did it, each rendition would take about 10 minutes, so about 50 minutes of work in a row. The creator would wait an hour.

So we split the work:

  1. Cut the original into chunks of 6 seconds. Cuts fall where a new keyframe starts, a full picture that later frames refer to, so each chunk plays by itself.
  2. That gives 600 / 6 = 100 chunks. With 5 renditions that is 500 jobs. Each job is "encode chunk 37 as 720p".
  3. The 500 jobs go on the queue. Workers take jobs in parallel. Each job takes seconds, so with enough free workers the whole video is ready in a minute or two. With 40 workers that is 500 x 6 / 40 = about 75 seconds.
  4. Each job writes a segment, one small file, for example v/{video}/720p/37.m4s. Cut points are the same in every rendition, so the player can switch quality at any boundary.
  5. When the lowest rendition is complete, the video becomes playable. The others follow, and the status becomes ready.

A job can fail or run twice. It writes to the same key every time, so a repeat only overwrites the same file. The queue retries failures.

Segments, the manifest and adaptive bitrate

The manifest is a small text file listing the renditions and the URL of every segment. HLS and DASH are two common manifest formats. The player downloads it first, then fetches one segment at a time.

sequenceDiagram
  participant P as Player
  participant C as CDN
  P->>C: manifest
  C-->>P: qualities and segment list
  P->>C: segment 1, lowest quality
  C-->>P: bytes, speed measured
  P->>C: segment 2, quality picked from speed

This is adaptive bitrate streaming: the player picks the quality of the next segment from its own measured speed. A simple rule: speed = segment size x 8 / download time. Pick the highest rendition below 80% of that speed, and drop one step if less than 10 seconds of video is buffered.

Example: a 2.25 MB segment, which is a 6-second piece at 3 Mbps, takes 3 seconds to download. Speed = 2.25 x 8 / 3 = 6 Mbps. 80% of that is 4.8 Mbps, so the next segment is 720p at 3 Mbps. If the next one takes 6 seconds, speed falls to 3 Mbps, 80% is 2.4, and the player drops to 480p at 1.5 Mbps.

Someone might say the server should detect the network and push the right quality. It would have to track every viewer, and it cannot see their last-mile network as well as the player can. The player sees its own speed. The server only serves plain files, which also makes them cacheable.

Delivery: object storage behind a CDN

Segments are plain files that never change, and every viewer of a given rendition gets the same bytes. That is ideal for a CDN. The player gets CDN URLs. A signed URL carries a signature, so the CDN must check it but cache the file by its plain path. Otherwise every viewer's URL looks different and nothing is cached. On a miss (the CDN does not hold the file) the CDN fetches the segment from our segment store, called the origin, and keeps a copy. The next viewer nearby gets it from the edge. Most of the 15 Tbps never touches our origin. Lesson 3 covers cost, tiers and the long tail.

Catalogue, resume position and one full play

The catalogue is titles, descriptions and images. A metadata service reads it from a database with a cache in front. Peak reads are 35,000 a second and one database comfortably handles about 2,000. So the cache must answer at least 1 - 2,000 / 35,000 = about 94%. Aim for 95% or more. Catalogue data changes rarely, so a 5-minute expiry works. The trade-off: an edited title can look stale for up to 5 minutes.

Progress is different. Each player reports its position every 30 seconds: 5 million / 30 = about 170,000 writes a second. We keep only the latest position per user and video, in a key-value store split by user id into about 20 shards, each handling about 10,000 writes a second. If a phone crashes we lose up to 30 seconds. That is acceptable.

One play, start to finish:

  1. The app calls the playback endpoint. The service checks the user, reads the title from cache and the resume position from the progress store, and returns a signed manifest URL.
  2. The player fetches the manifest from the CDN, then segments, starting at a low rendition.
  3. After each segment it measures speed and picks the next quality.
  4. Every 30 seconds it sends its position, and the service overwrites the stored value.

📌 Summary: Store the original, split it into 6-second chunks, encode chunks in parallel through a queue, and serve plain segments through a CDN. The player picks quality from its measured speed.

What Meera learns next

The pipeline works on paper. Next: three places it hurts at scale, and what to say when the interviewer asks "what breaks?".

Check your understanding

  1. Version one serves every viewer from one server with a 10 Gbps card. Each stream uses 3 Mbps. About how many viewers can it feed?
  2. A 10-minute video becomes 5 renditions. One core takes about 10 minutes per rendition. The team says one big machine should do it all. What is the better plan?
  3. A 2.25 MB segment took 3 seconds to download. The player picks the highest rendition below 80% of the measured speed. The ladder is 0.4, 0.8, 1.5, 3 and 6 Mbps. What does it pick next?
  4. Catalogue reads peak at 35,000 a second. One database comfortably handles 2,000. What cache hit ratio do you need at least?
  5. Five million viewers each send their position every 30 seconds. Why keep only the latest position per user and video?