The Two Walls

Part of the free Generative AI course on LogicWiz, module: The Agent Awakens.

Episode 11: The Two Walls

"A brilliant mind with no eyes, no hands, and no memory is a very expensive paperweight."

Nova Hits a Wall (Twice)

Nova can hold a conversation now, and with a sharp prompt she's genuinely useful. So Anjali raised the bar: "Make Nova the front desk for LogicWizNews. Readers should be able to ask her anything."

Arjun launched it. Within an hour, two questions broke her.

Reader 1: "What's the biggest story on the site right now?" Nova answered confidently — with a story from her training data, two years stale. She had no idea what was actually published today.

Reader 2: "Tell me about the EU AI Act." (then, a moment later) "When does that take effect?" Nova replied: "When does what take effect?" She'd already forgotten the previous message.

Two failures, two hard walls.

The first is like a brilliant professor who's been stranded on a desert island for two years. Everything she learned before is razor-sharp — but ask about anything since, and she's blank. That "stranded since" date is the model's training cutoff. Let's watch it:

The second wall is pure amnesia. Every message arrives as if it's the first — like the film Memento, whose hero wakes into each scene with no memory of the last. Neither wall is a prompting problem: you can't word your way around information the model doesn't have or a memory it doesn't keep. Watch the amnesia play out in full:

The Four Limits of a Lone LLM

Step back and the pattern is clear. A standalone LLM, however large, has four hard limits:

No fresh knowledge — it's frozen at its training cutoff. Ask for today's weather, a live stock price, or this morning's headline and it simply can't know.
Hallucinations — when it lacks the facts, it doesn't shrug and say "I don't know." It generates a confident, plausible-sounding answer that happens to be wrong. (This is the real reason lawyers have been sanctioned for filing AI-written briefs that cited court cases which never existed.)
No actions — it produces text, full stop. It can write the email, but it can't click send, run a database query, or update a ticket on its own.
No memory — every call starts from a blank slate. It's like a call centre where each call reaches a brand-new agent with none of your history — you re-explain everything, every time.

Here's the good news: each limit has a fix, and together the fixes turn a chat model into an agent.

The LLM's limit	The agentic fix
No fresh knowledge	Grounding — feed it real, up-to-date data instead of relying on training
Hallucinations	Groundedness — answers stay anchored to that provided data, so it stops guessing
No actions	Agency — let it actually do things on the user's behalf
No memory	Context retention — carry the conversation and history forward

Two upgrades deliver all four fixes: tools and memory.

Tools: Giving Nova Hands

A tool is a function you make available to the model so it can reach beyond its own text — to fetch something or do something in the real world. If an LLM is a brilliant brain in a jar, tools are its hands, eyes, and phone. It's the same leap ChatGPT made when it gained web browsing and a calculator: suddenly it could look things up and compute, not just talk.

Tools fix two of the four limits at once:

Give Nova a get_todays_headlines() tool, and "no fresh knowledge" disappears — she's no longer trapped in her training snapshot; she can read today's front page.
Because she's now answering from real, retrieved data instead of her own weights, she's far less likely to hallucinate. That's grounding — the single biggest lever on reliability.

Tools come in a few flavours:

Tool type	What it does	A Nova tool
Retrieval	Fetch data — from an API, a database, or a vector store	`get_todays_headlines()`
Action	Change the world — send an email, book a slot, update a record	`send_newsletter()`
Logic	Run deterministic code — sort, filter, calculate	`word_count(article)`

💡 Tip: The most famous retrieval pattern, RAG (Retrieval-Augmented Generation), is really just a tool: it looks up relevant documents and hands them to the model as context. We'll build RAG properly in a later chapter — for now, file it under "tools."

The LLM never runs the tool itself. It decides a tool is needed and asks for it; the surrounding system executes it and hands back the result. Holding that leash is the job of the piece we're about to name.

Memory: Short-Term and Long-Term

The other upgrade is memory, and it comes in two tiers — a distinction worth burning into your brain. It mirrors how you remember: what you're holding in your head this second, versus what you've written down to look up later.

Short-term memory is like a computer's RAM — or the phone number you repeat in your head until you dial it. It's the conversation happening right now: the running list of messages you pass back to the model on every call. Fast and immediate, but bounded by the context window — a whiteboard of fixed size, so once it fills, the oldest notes get wiped to make room.
Long-term memory is like a hard drive — or your notebook and filing cabinet. It's information that must survive beyond this conversation: a reader's preferences, past chats, a whole knowledge base. It lives in a database, and the agent pulls the relevant bits into short-term memory (using a tool) exactly when they're needed.

Why not just cram everything into short-term memory? Because the whiteboard is finite, and you can't predict what will matter. Long-term memory lets Nova know far more than she can hold at once: a reader once said "I only want 3-bullet summaries," Nova files that away, and weeks later she fetches it the moment that reader returns.

The Anatomy of an Agent

Now we can name the thing we've been building. When you wrap an LLM with tools and memory and put it in charge, you get an agent. There's even a formula:

Agent = ( Prompt + Tools + Memory ) × LLM

Read it as: the prompt defines the behaviour, tools grant abilities, memory grants context — and the LLM multiplies them all together as the reasoning engine that decides what to do. (It's a mnemonic, not literal maths — but it captures the shape exactly.)

There's a crucial split hiding in that formula, and it's the same one on any film set. The LLM is the controller — the director who reasons, breaks a goal into steps, and decides which tool to call next. The agent is the executor — the crew (the surrounding code, often called the orchestrator) that actually runs the tools, manages the memory, and hands results back to the director. The orchestrator isn't inside the model; it's the structure you build around it — and that structure is exactly what makes "an agent" more than "an LLM."

flowchart TB
  U[User query] --> O[Orchestrator]
  O <--> L[LLM · the brain]
  O <--> M[Memory · chat history + database]
  O <--> T[Tools · APIs, search, code]
  O --> R[Answer]

📌 Summary: A lone LLM has four limits — no fresh knowledge, hallucinations, no actions, no memory. Tools fix knowledge and hallucination (via grounding) and add agency; memory (short-term RAM + long-term hard drive) adds context retention. Wrap an LLM with prompt + tools + memory and put an orchestrator in charge, and you have an agent: Agent = (Prompt + Tools + Memory) × LLM.

What Nova Learns Next

Nova now has a brain, hands, and a memory. But one question remains, and it's the hardest one: how does she decide what to do? When a reader asks something messy — "what's the top story, and is it good news for the markets?" — Nova has to figure out the steps herself: think, act, look at the result, think again.

Next episode we build exactly that engine — the agent loop. You'll meet the Think → Act → Observe cycle (the pattern the whole industry calls ReAct) and watch Nova reason her way through a real task, one tool call at a time. This is the moment she stops being a smart component and starts being an agent.