System design glossary

Every term the LogicWiz System Design course uses, in plain words, with the lesson that teaches it.

Access pattern (also query pattern)

The exact questions your app will ask the data, such as "orders for one customer, newest first". You design tables and keys around these questions, not the other way round. Lesson

API gateway

A single front door for many backend services. It routes each request to the right service and handles shared jobs like login checks, rate limits and logging in one place. Lesson

At-least-once delivery

A messaging promise that every message arrives, but some may arrive twice (for example after a retry whose first attempt actually succeeded). Consumers must handle duplicates safely. Lesson

Availability (also uptime, nines, three nines, four nines)

The share of time a system answers correctly, often written in "nines". 99.9% allows about 8.8 hours of downtime a year; 99.99% allows about 53 minutes. Lesson

B-tree (also B+ tree, Btree)

A sorted, wide and shallow tree stored in fixed-size disk pages, used by most SQL databases for indexes (and, in some, the tables themselves). Any row is found in 3 or 4 page reads, and rows are updated in place. Lesson

Back-of-the-envelope estimation (also estimation, capacity estimation, napkin math)

Rough arithmetic with round numbers (a day is about 100,000 seconds) to find requests per second, storage and bandwidth, so the numbers decide which components you need. Lesson

Bloom filter

A tiny bit array that answers "is this key possibly here?" It can say "maybe" wrongly but never says "no" wrongly, so a database can skip reading files that cannot contain the key. Lesson

BM25 (also relevance scoring, Okapi BM25)

A standard formula for ranking search results. A document scores higher when it contains the query words often, when those words are rare across all documents, and less so just for being long. Lesson

Bulkhead

Giving each dependency its own limited pool of threads or connections, like watertight compartments in a ship, so one slow dependency cannot use up every thread and sink the whole service. Lesson

Cache (also Redis, Memcached)

A faster copy of data you already have, usually in memory, so most reads skip the database. It is disposable: losing it should cost speed, never data. Lesson

Cache stampede (also thundering herd, dogpile, cache stampede problem)

When a popular cache entry expires and thousands of requests miss at the same moment, each one runs the same expensive database query and can overload the database. Lesson

Cache-aside (also lazy loading, look-aside cache)

The common caching pattern: the service checks the cache first; on a miss it reads the database, stores the answer in the cache, and returns it. Lesson

CAP theorem (also CAP, Brewer's theorem)

When a network partition cuts replicas off from each other, a system must choose: refuse some requests to stay consistent (CP), or keep answering with possibly stale data (AP). Lesson

CDN (also content delivery network, edge cache)

A content delivery network: servers in many cities that keep copies of static files like images, scripts and videos, so users download them from somewhere close instead of your origin server. Lesson

Chunking

Splitting long documents into smaller passages (say 300 to 800 tokens, often with some overlap) before embedding them, so retrieval can return the paragraph that answers a question, not a whole manual. Lesson

Circuit breaker

A switch around calls to a dependency. After too many failures it "opens" and fails fast for a while instead of waiting on timeouts, then lets a few test calls through to see if the dependency recovered. Lesson

Compaction

Background work in an LSM tree that merges sorted files, keeps only the newest version of each key and drops deleted ones, so reads check fewer files and old data stops wasting disk. Lesson

Consistent hashing (also hash ring)

Placing servers and keys on the same hash ring, where each key belongs to the next server clockwise. Adding or removing a server moves only the keys next to it, not almost all of them. Lesson

Cursor pagination (also keyset pagination, seek pagination)

Paging where the server returns a bookmark to the last item seen (such as its time and id) and the next request asks for items after it. New items cannot shift pages, so nothing is shown twice. Lesson

Dead-letter queue (also DLQ)

A side queue where messages go after failing every retry, such as one with a malformed phone number, so they stop blocking the main queue and a person can inspect them later. Lesson

Denormalisation (also denormalization)

Copying data on purpose, such as storing the product name inside each order row, so a common read needs no join. You pay with extra storage and more places to update. Lesson

Distributed lock

A lock shared by many machines, so only one of them does a job at a time, such as booking a seat. Because the holder can pause or crash, it is usually a lease paired with a fencing token. Lesson

Distributed tracing (also tracing, traces, span, OpenTelemetry)

Following one request across every service it touches by passing a shared trace id along, then showing each hop as a timed bar. It tells you which service made the request slow. Lesson

DNS (also Domain Name System)

The Domain Name System: the internet's phone book that turns a name like swiftkart.in into the IP address of a server. Answers are cached for a TTL, so changes take time to spread. Lesson

Embedding (also vector embedding, vector)

A list of a few hundred to a few thousand numbers that a model produces for a piece of text, placed so that texts with similar meaning get nearby vectors. Used to search by meaning, not exact words. Lesson

Error budget

The amount of failure an SLO allows. A 99.9% monthly SLO leaves about 43 minutes of errors a month; while budget remains you ship fast, and once it is spent you slow down and fix reliability. Lesson

Evals (also evaluations, LLM evals)

Automated test sets for an LLM feature: fixed questions with expected answers or scoring rules, run on every prompt or model change so you can tell whether quality went up or down. Lesson

Eventual consistency

A guarantee that replicas will agree once updates stop arriving, but a read right after a write may still return the old value for a short time. Lesson

Exponential backoff (also backoff)

Waiting longer before each retry, for example 1, 2, 4 then 8 seconds, so a struggling service gets room to recover instead of being hammered by instant retries. Lesson

Failover (also split brain)

Promoting a replica to become the new primary when the old primary dies, and pointing traffic at it. Done badly it can lose recent writes, or leave two machines both accepting writes (split brain). Lesson

Fallback model

A second LLM, often from another provider or a smaller model, that the gateway switches to when the main model errors, times out or hits its rate limit, so the feature degrades instead of failing. Lesson

Fan-out (also fan-out on write, fan-out on read, push model, pull model)

Delivering one event to many readers, such as a post to every follower's feed. Fan-out on write copies it to each feed when posted; fan-out on read gathers posts when a feed is opened. Lesson

Fencing token

A number that goes up every time a lock is granted. The storage rejects any write carrying an older number, so a client that paused and lost its lock cannot overwrite the new holder's work. Lesson

Functional requirement (also FR, feature requirement)

Something the system must do for a user, written as a feature you can test, such as "a customer can see where their order is". Lesson

Geohash

A short string encoding a latitude and longitude, built by repeatedly halving the map. Nearby places usually share a prefix, so "shops near me" becomes a prefix lookup on an ordinary index. Lesson

Golden signals (also four golden signals)

The four numbers to watch on any service: latency (how slow), traffic (how many requests), errors (how many fail) and saturation (how full its CPU, memory or queues are). Lesson

Graceful degradation

Keeping the core of a product working when a part fails, such as hiding recommendations but still letting people check out, instead of showing an error page. Lesson

Grounding (also citations, grounded generation)

Making an LLM answer only from the retrieved passages and cite which ones it used, so readers can check claims and the model says "I don't know" instead of inventing facts. Lesson

gRPC (also RPC, protobuf)

A fast way for services to call each other, using compact binary messages over HTTP/2 and a shared contract file. Popular for internal service-to-service calls; browsers rarely use it directly. Lesson

Guardrails (also content filter, safety filter)

Checks around an LLM call that block or fix unsafe input and output, such as removing personal data, refusing off-topic requests or validating that the answer is well-formed JSON. Lesson

Headroom

Spare capacity you plan on purpose, usually about 2x the estimated peak, so a traffic spike or a lost server does not push the system over the edge. Lesson

Health check

A small request a load balancer sends to each server every few seconds; a server that fails it stops receiving traffic until it passes again. Lesson

Horizontal scaling (also scale out)

Handling more load by adding more machines and spreading work across them, rather than buying a bigger machine (vertical scaling). It needs a load balancer and usually stateless servers. Lesson

Hot key (also hot partition, hotspot, celebrity problem)

One cache or database key that receives a huge share of traffic, such as the sale's "top deals" list, overloading the single machine that holds it. Fixed by splitting it or copying it locally. Lesson

Hybrid search

Running keyword search (like BM25) and vector search together and merging the results, so exact terms like order IDs and paraphrased questions both find the right passages. Lesson

Idempotency (also idempotent, idempotency key, Idempotency-Key header)

The property that doing an operation twice has the same effect as doing it once. Setting a price to 100 is idempotent; adding 100 is not, unless the client sends an idempotency key the server uses to spot the repeat. Lesson

Idempotent consumer (also deduplication)

A message handler that records which message ids it has already processed and skips repeats, so at-least-once delivery cannot send two SMSes or charge a card twice. Lesson

Index (also database index, secondary index)

An extra sorted structure on a column, like the index at the back of a book, that lets the database jump to matching rows instead of scanning every row. It makes writes slightly slower. Lesson

Inverted index (also search index, full-text index)

A map from each word to the list of documents that contain it, the way a book index maps a word to page numbers. It is how search engines find matches without reading every document. Lesson

Jitter

A small random amount added to retry or expiry times so thousands of clients or cache keys do not all act at the same instant and overload the system together. Lesson

L4 vs L7 load balancing (also layer 4, layer 7, L4, L7)

A layer 4 balancer routes by connection (IP address and port) without reading the request, so it is very fast. A layer 7 balancer reads HTTP, so it can route by path, header or cookie. Lesson

Latency (also p99, p95, p50, response time, tail latency)

How long one request takes from start to answer, usually in milliseconds. Measured as percentiles: p99 under 300 ms means 99 of every 100 requests finish within 300 ms. Lesson

Leader election

How a group of machines picks one of them to accept writes: candidates ask for votes, and the one that wins a majority becomes leader until it fails. Lesson

LLM gateway (also AI gateway, model gateway)

One service that every app calls instead of calling LLM providers directly. It handles keys, rate limits, caching, fallback models, guardrails, logging and cost tracking in one place. Lesson

Load balancer (also LB, round robin, least connections)

A server that sits in front of many copies of a service and spreads incoming requests across them, using a rule such as round robin or least connections, and skips copies that fail health checks. Lesson

Logs (also structured logs, logging)

Timestamped records of individual events, such as "order 812 failed: card declined". Best for answering what happened to one specific request, worst for spotting trends cheaply. Lesson

LSM tree (also log-structured merge tree, LSM)

A storage design built for heavy writes: writes go to an in-memory table (the memtable) and a log, are flushed as sorted immutable files, and those files are merged in the background. Used by Cassandra and RocksDB. Lesson

Message queue (also queue, MQ, SQS, RabbitMQ)

A waiting line between services: a producer drops a job in and moves on, and workers take jobs out at their own pace. Each job is handled by one worker and removed once done. Lesson

Metrics (also time series, telemetry)

Numbers measured over time, such as requests per second or p99 latency per minute. Cheap to store and ideal for graphs and alerts, but they tell you that something is wrong, not which request. Lesson

Network partition (also partition (network))

A network fault that splits machines into groups that cannot reach each other, even though each machine is still running. Lesson

Non-functional requirement (also NFR, quality attribute)

A quality the system must have, stated with a number you can test: how fast (p99 under 300 ms), how available (99.95%), how much scale, how fresh the data must be. Lesson

NoSQL (also key-value store, document store, DynamoDB, Cassandra, MongoDB)

Databases that drop the fixed-table, join-everything model of SQL, such as key-value, document or wide-column stores. They suit huge volumes of simple lookups by key. Lesson

Outbox pattern (also transactional outbox)

Saving the order and the "order placed" message in the same database transaction, then a separate process publishes messages from that outbox table. You never save one and lose the other. Lesson

Peak factor (also peak-to-average ratio, peak QPS)

The ratio of the busiest rate to the average rate, such as 5 for dinner-time traffic. Average requests per second times the peak factor is what you design for. Lesson

Permission-aware retrieval (also ACL filtering, access-controlled retrieval)

Filtering search results by what the asking user may see, before the passages reach the LLM, so a chatbot cannot leak a salary sheet to someone without access to it. Lesson

Post-mortem (also postmortem, incident review, RCA)

A written review after an incident: the timeline, impact, root causes and fixes with owners. It is blameless, meaning it asks how the system let the mistake happen, not who to punish. Lesson

Posting list (also postings)

The list stored for one word in an inverted index: the ids of every document containing that word, kept sorted, often with counts and positions, so lists can be merged quickly. Lesson

Prompt injection (also indirect prompt injection)

An attack where text the model reads, such as a web page or uploaded document, contains instructions like "ignore your rules", and the model follows them as if they came from you. Lesson

QPS (also RPS, requests per second, throughput, TPS)

Queries (or requests) per second: the basic load number. Daily requests divided by about 100,000 seconds gives the average; multiply by a peak factor for the design target. Lesson

Quadtree

A map index that splits any square holding too many points into four smaller squares. Dense cities get tiny squares and empty areas big ones, so "nearest drivers" searches stay fast. Lesson

Quorum (also W + R > N, majority)

The minimum number of replicas that must agree for a read or write to count. With N copies, writes to W and reads from R where W + R > N always overlap, so a read sees the latest write. Lesson

Raft (also consensus)

A consensus algorithm: machines elect a leader by majority vote, the leader copies each change to the others, and a change counts only once a majority has stored it. Used by etcd and Consul. Lesson

Rate limiting (also throttling, HTTP 429, too many requests)

Capping how many requests a client may make in a time window, such as 100 a minute per user, and rejecting the rest with HTTP 429 so one client cannot overload the service for everyone. Lesson

Read replica (also replica, follower, secondary)

A copy of the database that receives every change from the primary and serves reads. It spreads read load and survives the primary's death, but may be slightly behind. Lesson

Read-your-writes (also read-after-write consistency)

A guarantee that after you change something, your own next read shows the change, even if other users may briefly see the old value. Often done by reading from the primary just after a write. Lesson

Redundancy (also N+1, replication)

Running more than one copy of a part so another takes over when one fails. Two independent 99% servers in parallel give 99.99%, while two parts in series give only about 98%. Lesson

Replication lag

The delay between a write reaching the primary and the same write appearing on a replica; reads from a lagging replica can return old data. Lesson

Request coalescing (also single-flight, request collapsing)

On a cache miss, letting only the first request fetch from the database while identical requests wait for its answer, so 6,000 simultaneous misses become one query. Lesson

Reranker (also cross-encoder, re-ranking)

A slower, more accurate model that rescores the top results of a first search, such as 50 passages, by reading each one alongside the question, then keeps only the best few for the LLM. Lesson

REST (also RESTful API, HTTP API)

An API style where URLs name resources (nouns like /orders) and HTTP methods are the verbs: GET reads, POST creates, PUT or PATCH changes, DELETE removes. The default for public APIs. Lesson

Retrieval-augmented generation (also RAG)

Answering with an LLM by first searching your own documents for relevant passages and pasting them into the prompt, so answers use current, private facts the model was never trained on. Lesson

Retry storm (also retry amplification)

When a slow service makes every caller retry, and callers of callers retry too, multiplying load (3 attempts at each of 3 layers is up to 27 times the traffic) just when the service is weakest. Lesson

Saga (also compensating transaction)

A long operation split into local steps across services, such as reserve stock, charge card, book courier. If a step fails, earlier steps are undone by compensating actions like a refund. Lesson

Semantic cache

A cache for LLM answers that matches questions by meaning using embeddings, so "how do I reset my password?" can reuse the answer to "forgot password, what now?" and skip a costly model call. Lesson

Shard key (also partition key)

The field that decides which shard a row lives on, such as customer_id. A good one has many distinct values, spreads load evenly and matches your main query; city would create a Mumbai hotspot. Lesson

Sharding (also horizontal partitioning, partitioning, shard)

Splitting one large dataset across several databases so each machine stores and serves only a slice, such as customers A to M on one and N to Z on another. Lesson

Single point of failure (also SPOF)

Any one part whose failure takes the whole system down, such as a lone load balancer or database. You remove it by adding a redundant copy that can take over. Lesson

SLI (also service level indicator)

A service level indicator: a measured number that reflects what users feel, such as the share of checkout requests that succeed in under 500 ms. Lesson

SLO (also service level objective, SLA)

A service level objective: the target for an SLI over a window, such as 99.9% of checkouts succeed within 500 ms over 30 days. An SLA is the contract with penalties built on top. Lesson

SSTable (also sorted string table)

A sorted string table: an immutable file of key-value pairs sorted by key, written when a memtable is flushed. Being sorted, it can be searched and merged with other SSTables quickly. Lesson

Stale-while-revalidate (also SWR, serve stale)

Serving the expired cached copy for a short while and refreshing it in the background, so users never wait on the database and no stampede happens at expiry. Lesson

Stateless service (also stateless)

A server that keeps no user data between requests in its own memory; sessions and carts live in a shared store. Any copy can serve any request, so you can add or kill copies freely. Lesson

Stream (also event log, Kafka, Kinesis, event stream)

An append-only log of events kept for a while. Many consumers each read it from their own position, and a new consumer can replay from the past. Kafka is the common example. Lesson

Strong consistency (also linearizability, linearizable)

A guarantee that every read sees the latest successful write, as if there were only one copy of the data. Needed for balances and bookings, at the cost of latency or availability. Lesson

Symptom-based alerting (also alerting, paging, burn-rate alert)

Paging people on what users feel, such as error rate or latency breaching the SLO, rather than on causes like high CPU, which often fire when nothing is wrong for users. Lesson

Timeout (also deadline)

A maximum time to wait for a reply before giving up, such as 2 seconds for a payment call. Without one, a hung dependency holds your threads forever and your service hangs with it. Lesson

Token (also tokens, context window)

The unit LLMs read, write and bill by: a word or piece of a word, roughly 4 characters of English. Prompt plus answer tokens set both the cost and the latency of each call. Lesson

Token bucket

A rate-limiting rule: a bucket holds up to N tokens and refills at a steady rate; each request spends one token and is rejected when the bucket is empty. It allows short bursts but caps the average. Lesson

Top-k retrieval (also k-nearest neighbours, kNN, ANN, similarity search)

Returning the k passages whose embeddings are closest to the question's, such as the top 5. Usually done with an approximate nearest-neighbour index, trading a little accuracy for speed. Lesson

TTL (also time to live, expiry)

Time to live: how long a cached copy (or DNS answer) may be used before it expires and is fetched fresh, such as 60 seconds for fast-changing prices. Lesson

Vector database (also vector store, vector DB, pgvector)

A store built to find the embeddings nearest to a query embedding among millions, fast, usually with metadata filters. Examples include pgvector, Pinecone and Milvus. Lesson

WebSocket (also WS, server push)

A connection that stays open after it is set up, so the server can push messages to the client at any time, as live chat and delivery tracking need, instead of the client asking repeatedly. Lesson

Write amplification

When one logical write causes many physical bytes to be written, such as a whole 8 KB page rewritten for a 100-byte row, or the same data rewritten again and again by compaction. Lesson

Write-ahead log (also WAL, redo log, commit log)

An append-only file where a database records every change before applying it. After a crash it replays the log, so no acknowledged write is lost. Lesson

Write-back cache (also write-behind)

A cache that acknowledges writes at once and saves them to the database later in batches. Writes are very fast, but anything not yet saved is lost if the cache dies. Lesson

Write-through cache (also write-through)

A cache where every write goes to both the cache and the database before it is acknowledged, so cached data stays fresh at the cost of slower writes. Lesson