Nova Learns to Think in Steps

Part of the free Generative AI course on LogicWiz, module: Nova Goes to Production.

Episode 21: Nova Learns to Think in Steps

"'Compare our refund policy to what you told this customer last week.' One question — but two lookups and a comparison. Nova's straight-line pipeline just... stops after the first."

The Question a Straight Line Can't Answer

Priya throws Nova a curveball: "Compare our refund policy to what you told this customer last week."

Answering that takes three moves: (1) retrieve the refund policy, (2) retrieve last week's conversation, (3) compare the two. But everything Nova has built is a fixed pipeline — Query → Retrieve → Generate. It retrieves once and answers. It has no way to look at what it found, realise it needs a second lookup, and go back. A straight line can't loop.

Some questions need Nova to decide her own steps. That's the leap from a RAG pipeline to a RAG agent.

Direct RAG vs. Agentic RAG

flowchart LR
    subgraph Direct["Direct RAG — a fixed line"]
        Q1["Query"] --> E1["Retrieve"] --> G1["Generate"]
    end
    subgraph Agentic["Agentic RAG — a reasoning loop"]
        Q2["Query"] --> T["Agent thinks"]
        T --> P["Pick a tool"]
        P --> O["Observe result"]
        O -->|need more| T
        O -->|done| A2["Answer"]
    end

{{visual:direct-vs-agentic}}

⚠️ Warning: Agentic is not an upgrade you apply everywhere. Every extra reasoning step is another LLM call — more latency, more cost, more to debug. For "how much is the Pro plan?" a direct pipeline wins. Reserve the agent for questions a straight line genuinely can't answer.

Retrieval Becomes a Tool

Back in Chapter IV, Nova learned the ReAct loop — think → act → observe, repeat until done. Agentic RAG points that loop at retrieval. But before that clicks, we have to clear up the one word everyone trips on: "tool."

First, a question Nova simply can't answer

Start with a real one. A customer messages Nova: "What's the status of my order #4471?"

Nova is a language model — she learned from a giant pile of public text, but she was never trained on your company's live order database. So she has no way to know the answer. Ask her and one of two bad things happens: she admits "I don't have access to that," or — worse — she invents a plausible-sounding status. Either way, the case fails.

Sit with why it fails. It isn't that Nova isn't smart enough. It's that the answer lives somewhere she can't reach — a database, a live API, the web. She can reason and she can talk, but on her own she cannot go and fetch anything. Every question whose answer lives outside her training — an order status, today's weather, a stock price — slams into the same wall.

The fix is a tool — and a tool is just a function

So we give Nova a helper she can call on — a small piece of code that can reach the database for her. When a question needs an order status, Nova stops trying to answer from memory and instead asks for that helper to run; it does the real lookup and passes the answer back to her. Now order #4471 is answerable.

That helper is a tool — and here's the part that surprises everyone: it's nothing exotic. A tool is just an ordinary function, the same kind you've written all course, that goes and gets what Nova can't reach:

def look_up_order(order_id):
    return db.get_order(order_id)   # your normal database query

Then you wrap that function with a plain-English description so Nova knows when to use it — "Look up the status of a customer order by its ID" — and hand it to her. Function + description = tool. That's the whole idea.

How Nova actually uses it

Now the part beginners trip on: Nova never runs this code herself, and never even sees it. She only ever sees three things — the tool's name, its description, and what arguments it takes. From the description alone she thinks "this question needs look_up_order" and says so, filling in order_id="4471". The framework (LangGraph) runs your actual function, gets back "Shipped — arrives Tuesday," and feeds that to Nova — who finally answers the customer.

So every tool is exactly three parts:

• Name — the identifier the agent calls (e.g. look_up_order).
• Function — your ordinary code that does the real work.
• Description — plain text saying what it does and when to use it. It's the agent's only guide for choosing the right tool.

So retrieval becomes one tool among many

Here's the "aha": retrieval is just a function too — "search the ticket archive, return the closest matches." Wrap it with a description and it becomes just another tool on the agent's desk, sitting right next to look_up_order, check_ticket_status, and escalate_to_human.

That's the whole shift. In direct RAG (Lessons 18–20) retrieval always ran, first, no matter the question. As a tool, the agent decides per question: do I even need to search? what should I search for? did that answer it, or do I need a second lookup?

And since the agent picks tools off their descriptions alone, a sloppy description derails the whole loop:

First, watch the whole loop end to end — how a question turns into an answer by choosing and running a tool:

{{visual:tool-call-flow}}

Now zoom into the code that makes it happen. Watch an agent read those descriptions and pick the right tool for a query:

{{visual:tools-walkthrough}}

{{cell:l21-tools}}

Tool-design rules that make agents work:

• Single-purpose — one tool, one job. Vague, do-everything tools confuse the agent.
• Precise descriptions — the agent selects entirely off the text; be specific about what, when, and what it returns.
• 3–7 tools — too few and it can't do the job; too many and it confuses similar ones. (If you truly need 50, the rule becomes: each must be clearly distinct.)
• Fail gracefully — a tool that errors should return a helpful message ("no ticket found; try rephrasing"), not crash — so the agent can react and try something else.

The Loop, With a Leash

LangGraph hands you the ReAct loop prebuilt: give create_react_agent a model and a list of tools, and it runs think → call tool → observe → repeat until the model produces a final answer. Nova can now chain two retrievals and compare them — no hand-wired branching.

{{visual:agent-walkthrough}}

{{cell:l21-agent}}

But an autonomous loop can run away. Two guardrails are non-negotiable:

• A max_iterations cap — so a confused agent can't spin forever, burning tokens on the same failing tool.
• Retry limits — a few attempts, then give up gracefully instead of hammering a dead endpoint.

📌 Summary: Direct RAG is a fixed line; agentic RAG is a reasoning loop where the agent picks tools — and retrieval is just one of them. Design tools single-purpose with precise descriptions (3–7 of them), make them fail gracefully, cap the loop with max_iterations, and reach for an agent only when the question truly needs multiple steps.

Chapter VI Complete — Nova Ships

Look how far Nova has come this chapter. She retrieves with MMR and orchestrates with LangChain; she holds a multi-turn conversation; she's measured — precision, recall, groundedness — and gated before every release; and now she can reason in a loop, choosing her own tools when one lookup isn't enough. That's not a prototype anymore. That's a production RAG system.

From here, Nova stops working alone. In Chapter VII, one agent becomes many — a team of specialists that plan, critique, and hand work to each other. Nova is about to build a crew.