When One Nova Isn't Enough

Part of the free Generative AI course on LogicWiz, module: Nova Builds a Team.

Episode 22: When One Nova Isn't Enough

"Two a.m. Priya's security dashboard lights up with 400 alerts at once. Nova — one lone agent — reads the first, reaches for a tool, freezes on which one, and the other 399 pile up behind it. By sunrise the queue has won."

The Night the Alerts Won

It's 2 a.m. at a security company, and the dashboard just lit up with hundreds of alerts at once. Each one might be a real attack, so each one has to be checked.

Here's the catch: checking a single alert isn't one step. Someone reads the logs, someone decides if it's actually dangerous, someone opens a ticket, someone messages the customer, someone writes up what happened. Different people, different tools, one alert — and start to finish it takes about 30 minutes and costs the company roughly $15 in staff time (all those people's paid hours add up). For one alert.

Now picture hundreds of them, at 2 a.m., with a handful of tired people. The queue only grows. And the only way to go faster is to hire more people — which never quite keeps up.

The fix isn't a smarter AI. It's a team of them — many small agents, each handling one part of the job, all working at the same time. That's a multi-agent system. This whole episode is about when you actually need one — because, honestly, most of the time you don't.

Nova, So Far, Is One Agent

Remember where we left Nova. In Chapter VI she stopped being a plain retriever and became an agent: she reasons about a question and calls tools to answer it — search the ticket archive, look up an order, escalate to a human. One brain, a handful of tools, a reasoning loop. For a support bot, that is often the whole product.

So the honest question isn't "how do I build a multi-agent system?" It's "do I even need one yet?"

When Even Great Retrieval Hits a Wall

One more piece of Nova's foundation, because the real world runs at enterprise scale. Plain RAG is lovely on a tidy knowledge base, but point it at 1,000 documents of 400–700 pages each and it buckles: too much text, too many near-identical chunks, and accuracy nosedives. Two techniques rescue retrieval at that scale.

Page indexing treats a giant document like a textbook with a table of contents. Instead of scanning everything, you store a hierarchy and navigate it:

flowchart TD
    Index["Index page"] --> C1["Chapter 1"]
    Index --> C2["Chapter 7 · Optics"]
    C2 --> S1["Section 7.1"]
    C2 --> S2["Section 7.2"]
    S2 --> Sub["Subsection → the answer"]

Ask "what are optics?" and the system reads the index, jumps to Chapter 7, then drills to the exact subsection — no blind scan. It stores this as a structured dictionary (not a vector store), which is why it wins for enormous single documents where plain RAG's ranking would thrash.

Parent-child retrieval is the other fix — and the one you'll reach for most. The name sounds fancy, but the idea is plain: the parent is a whole document, and the children are the small chunks inside it.

Think of finding a fact in a library. You don't read every book — you do it in two steps:

Pick the right book. Skim the catalog and each book's blurb to find the one book that's likely to have your answer.
Find the right page. Open only that book and go to the exact paragraph.

Parent-child retrieval works the same way. When you first load your documents, you also save a short summary of each one (a blurb) next to its chunks. Then a search happens in two hops: first it matches the question against those summaries to pick the right document (the parent), and only then does it search the chunks inside that one document (the children) for the exact answer — instead of scanning all half-a-million chunks at once.

You grade either approach the way you'd grade any retriever — Top-K accuracy on a set of question→expected-document pairs (does the right doc land in the top 3 or 5?). Mix simple and hard questions in that set, or you'll flatter yourself with easy wins.

But here's the pivot into this chapter: even world-class retrieval only fetches. Deciding, acting, calling tools, juggling a multi-step job — that's the agent's work. Nova became one. And even one agent has limits.

The Three Walls a Single Agent Hits

A single agent is brilliant until it slams into one of three walls. When it does, that's your signal — not before.

Too many tools. Give one agent 25 tools and it can't choose. Every tool's description crowds its context, and it starts grabbing the wrong one. (A rough rule of thumb: past ~5–7 tools, a single agent gets shaky.)
Specialized knowledge. Picture an education bot that must teach math, biology, and law. Cram all three into one agent and the contexts collide — the model overloads, blurs the domains, and hallucinates. Math and law share almost no context; forcing them into one brain creates a conflict.
Sequential constraints. Some jobs must run in a strict order — step 2 only after step 1 succeeds, unlocking the next capability only once a milestone is met. One agent has no clean way to enforce and checkpoint that.

Hit any wall and the answer is the same: stop making one agent do everything. Split the work.

The Golden Rule: One Agent First

Here's the discipline that separates good systems from over-engineered ones:

📌 Always build and test a single agent first. Only reach for multiple agents when that single agent — given the right tools and prompt — genuinely can't do the job.

Why so cautious? Because multi-agent systems aren't free. More agents means more LLM calls, more tokens, more coordination overhead, more ways to break. A single agent with a sharp prompt and a tidy toolset often matches a multi-agent setup at a fraction of the cost. Multi-agent is a specialized tool for a subset of problems — not the default.

So What Is a Multi-Agent System?

A multi-agent system is just a network of specialized agents working together to answer one query — usually a coordinator agent that plans and delegates, plus specialist agents that each own a domain.

If that sounds abstract, you already know the shape from software. A giant app like Booking.com isn't one tangled block of code — it's modules: one for flights, one for hotels, one for experiences. Turn each module into an autonomous agent, put a coordinator on top, and you have a multi-agent system.

That modular instinct is decades old in software, and it buys the same things here:

Fault isolation — if the Hotels agent falls over, Flights and Experiences keep serving. No single point of failure.
Distributed development — different teams own different agents behind clear boundaries.
Parallelization — independent subtasks run at the same time, so the whole job finishes sooner.
Scalability — need more capacity? Add more agents; the system doesn't slow down.
Context management — each agent holds only its knowledge, so no one model's context window gets overwhelmed.
Resilience & speed — many agents on different parts of a problem finish it quicker and keep going if one stalls.

💡 "Faster" here doesn't mean a single call runs at lower latency — it means the whole job finishes sooner because ten agents tackle ten parts at once, like ten people clearing a warehouse instead of one.

The Real Skill: Context Engineering

If you take one idea from this episode, take this. Designing a multi-agent system is mostly context engineering: deciding what information each agent gets to see.

The whole system is only as good as this routing of context. The math agent must receive math context and not be handed the biology prompt; the Flights agent needs flight tools and nothing else. Get the right context to the right agent and the system sings. Get it wrong — leak everything to everyone — and you've just rebuilt the overloaded single agent, but slower and pricier.

Before Nova can coordinate a team, she has to do the first, humblest job of context engineering: look at a question and decide which specialist it even belongs to. Here's that idea at a glance:

Now try it in code — the cell below is exactly this: a router that reads a question and picks the one specialist it belongs to.

What Nova Learns Next

Nova now knows when a team beats a soloist, and that the art is routing context to the right specialist. Next episode she assembles the team — the handful of coordination patterns (routing, orchestrator-worker, parallelization, and sub-agents) that turn a pile of agents into a system that actually works together.