Object storage: where photos, videos and files live

Part of the free System Design course on LogicWiz, in the chapter: Storage engines, search and geo. The interactive whiteboard exercises and simulations run in the app.

The database that ate the photos

"The orders database grew by 3 TB last month. Less than 1% of that was orders."

Kabir, at the monthly capacity review

Photos in the orders database

SwiftKart added proof of delivery: the rider photographs the bag at the customer's door. The team stored each photo in the orders database, in a BLOB column. BLOB means binary large object: a column type that holds the raw bytes of any file. Product photos already lived there the same way.

Now the numbers. About 500,000 orders a day, one photo each, about 200 KB after compression. That is 100 GB a day, roughly 3 TB a month. The order rows themselves add about 0.5 GB a day. The database grew 200 times faster than its real data, and things broke one by one:

  • Backups and replicas. The nightly backup copied terabytes. A restore took hours, not minutes. A new replica needed a day of copying before it could serve.
  • Memory. The database keeps hot pages, the blocks of data used most, in memory. Photo bytes pushed out order rows, so lookups that used to hit memory went to disk.
  • Every view. Each photo view ran through an app server and a database query. A primary comfortable at about 2,000 simple queries a second spent its time streaming bytes.
  • Cost. Database disks are fast, replicated and expensive. Photos need none of that speed.

A teammate's review comment said: "Store the image as base64 in a text column. One less system to run." Base64 writes bytes as text using 64 safe characters. Every 3 bytes become 4 characters, so each photo grows by a third. It keeps every problem above and makes each one a third worse. You do run one less system, but the one you keep gets much heavier.

What object storage is

Object storage is a service that keeps whole files, called objects, by name, cheaply and with several copies. Most teams use object storage such as S3-compatible stores. Three words to learn:

  • A bucket is a named container, such as swiftkart-delivery-photos.
  • A key is the object's full name inside the bucket, such as pod/2026/11/04/order-81723004.jpg.
  • The object is the bytes plus metadata: small facts about them, such as content type, size and a checksum (a short fingerprint that reveals damaged bytes).

The namespace is flat. There are no real folders; the slashes are part of the key. Tools show folders by grouping keys that share a prefix, the same starting text. You can PUT a whole object, GET it (all of it or a byte range), DELETE it, and LIST keys by prefix. You cannot edit the middle of an object. To change it, you write a new one.

The store keeps several copies across separate machines and buildings, so losing an object is extremely rare. It is cheap: assume about 2 cents per GB per month. Each month's 3 TB of new photos adds about $60 a month to the bill, and the total keeps growing. It is slower per request than a cache. The first byte takes tens of milliseconds, against under a millisecond for an in-memory cache.

Think of the left-luggage counter at a railway station. You hand over a bag and get a token, and later the token gets the bag back. Staff do not open your bag to swap one shirt: you collect it and check in a new one. The analogy stops in two places. The counter keeps one copy in one room, while object storage keeps several in different buildings. And, as you will see, you can give someone a token that works for one bag and only for five minutes.

Metadata in the database, bytes in the bucket

The pattern every design uses: the database row holds small facts, and the bucket holds the bytes.

delivery_photos(order_id PK, photo_key, size_bytes, content_type, uploaded_at)

Each row is about 200 bytes, so 500,000 a day add about 100 MB, a thousand times less than the photos. The database stays small, fast and quick to back up. Questions like "which photos belong to this order?" hit the database, never the bucket.

Order the writes so that a failure is harmless. Upload the bytes first, then write the row. If the row write fails, you have an object nobody points to, and a cleanup job deletes such objects after a day. The other order is worse: a row pointing to a photo that never arrived.

Uploads that skip your servers

The first design sent each photo from the rider's phone to the order service, which forwarded it to storage. On weak 4G at about 100 KB a second, a 200 KB photo holds a server connection for 2 seconds. The server carries every byte twice, in and out, and does nothing useful with them.

A pre-signed URL fixes this. It is a link to one object, with an expiry time and a signature your server makes with a secret key only it knows. Whoever holds the link can do exactly one thing, such as PUT this key, until it expires. Signing is local arithmetic, so it costs the server almost nothing.

sequenceDiagram
  participant R as Rider app
  participant S as Order service
  participant O as Object storage
  participant D as Orders DB
  R->>S: "POST photo for order 81723004"
  S-->>R: "pre-signed PUT URL, 5 min"
  R->>O: "PUT photo bytes"
  O-->>R: "200 OK"
  R->>S: "upload done"
  S->>O: "check object exists"
  S->>D: "save photo key"

Before signing, the service checks that this rider is assigned to this order. The bytes then travel straight from phone to bucket, and your servers handle only small JSON calls. Private photos are read the same way: the service hands out a pre-signed GET link that expires in minutes.

Big files need one more step. A seller's 200 MB product video should not restart from zero when a shaky connection drops at 90%. A multipart upload splits the file into parts, say 8 MB each, so 25 parts. The app uploads them in parallel and retries only the parts that fail. One final "complete" call joins them into one object.

Serving through a CDN, and lifecycle rules

Product photos are public and read constantly. On a busy evening the product page is requested about 2,000 times a second. A CDN (content delivery network) keeps copies of files on servers near users and fetches from the bucket only on a miss. At a 95% hit ratio, the bucket sees about 100 requests a second instead of 2,000. Give each new photo a new key, such as amul-milk-v3.jpg. Then the CDN can cache it for a long time and never needs a purge, a forced removal from every CDN server.

Proof-of-delivery photos show customers' front doors. They stay private: pre-signed GET links, never a public CDN path.

Old files are rarely read. Delivery disputes arrive in the first week, and after that almost nobody looks. Object storage offers storage tiers:

  • A standard tier for files read often.
  • An infrequent-access tier that costs less to store but charges for each read.
  • An archive tier, cheapest of all, that takes minutes to hours to read back.

A lifecycle rule moves or deletes objects by age, automatically. For example: move pod/ objects to infrequent access after 30 days and to archive after 180 days, then delete them after 2 years. Add a rule that aborts multipart uploads left unfinished for a day, because their parts take space and cost money.

What object storage is bad at

Object storage is built for large files written once and read many times. It is the wrong tool for:

  • Small, frequent updates. Changing one field means rewriting the whole object. Order status, wallet balances and counters belong in a database.
  • Appending. You cannot add a line to the end of an object. Logs are written as many objects instead, say one per minute.
  • Listing huge prefixes. A LIST call returns about 1,000 keys a page, so 50 million keys take 50,000 calls. Keep your own index of keys in the database.
  • Tiny, fast reads. Tens of milliseconds per request is too slow for a hot path. Put a cache or a CDN in front.

📌 Summary: Keep file bytes in object storage and only keys and metadata in the database. Upload with pre-signed URLs, serve public files through a CDN, and let lifecycle rules move old files to cheaper tiers.

What Meera learns next

Photos, videos and policy documents now live in object storage. Next week those documents feed the newest kind of system: a support bot that answers from your own data using a large language model.

Check your understanding

  1. A teammate suggests storing each delivery photo as base64 in a text column, "so we run one less system". What is the best reply?
  2. Why does the rider app upload its photo with a pre-signed URL instead of sending it through the order service?
  3. A seller uploads a 200 MB product video on a shaky connection, and it fails at 90% every time. What fixes it?
  4. A teammate wants to keep each order's live status as a small object, rewritten every few seconds. Why is that a poor fit?