The Machine That Learned to Improvise

Part of the free Generative AI course on LogicWiz, module: The Agent Awakens.

Episode 8: The Machine That Learned to Improvise

"You can hardcode a hundred answers. Intelligence is what happens on the hundred-and-first question."

The Chatbot That Only Knew Five Sentences

Arjun shipped his first version of Nova on a Friday.

She was a chatbot for the LogicWizNews homepage — a friendly little box that answered reader questions. Under the hood, she was a wall of if statements. If the message contained "hello," say hi. If it mentioned "subscribe," paste the pricing link. Simple. Fast. Shippable.

Think of her like a tourist holding a phrasebook. As long as the conversation sticks to the exact sentences printed on the page, she does fine. The moment anyone goes off-script, she's stranded.

Then Anjali, the product manager, demoed her live to the leadership team.

"Nova, what's the biggest story today?"

Bot: Sorry, I cannot help with that.

"Okay… Nova, summarise the election coverage for me."

Bot: Sorry, I cannot help with that.

Arjun watched his chatbot fail in real time, in front of everyone, on every question he hadn't personally predicted. The room went very quiet.

Here is Nova, version one. Run her, then try typing a question the author never thought of.

{{visual:rule-chatbot-walkthrough}}

{{cell:l8-try-rule-chatbot}}

Notice what's happening. The bot doesn't understand anything. It matches keywords you hardcoded, and the instant a reader phrases things their own way — or asks about something genuinely new — it falls off a cliff.

And you can't just keep adding if statements. A reader can ask for the top story a thousand ways: "biggest story?", "what's big today?", "anything major happening?", "give me the headline." You'd be writing rules forever and still miss the next one.

That's the wall every builder hits. You cannot hardcode intelligence. Arjun needed a machine that could handle the hundred-and-first question — one it had never been shown. That is a completely different kind of software.

Where AI Actually Lives: The Map

Before we rebuild Nova, let's get our bearings. "AI" is a big, fuzzy word, and it helps to see how the pieces nest inside one another — like a set of Russian dolls, each smaller and more specialised than the last.

{{visual:ai-hierarchy}}

Working from the outside in — with an everyday example of each:

• Artificial Intelligence (AI) — the widest circle. Any technique that makes a machine act "smart." A chess program, the route Google Maps picks for you, a spam filter, even Nova's if statements all count as AI.
• Machine Learning (ML) — a subset of AI. Instead of hand-writing the rules, you show the machine thousands of examples and it works out the rules itself. Nobody wrote "if the email says 'you won a prize,' flag it as spam" — the filter learned that pattern from millions of labelled emails. Netflix suggesting your next show works the same way.
• Deep Learning (DL) — a subset of ML built on neural networks: layered systems loosely inspired by how the brain wires neurons together. The extra layers let it learn richer, more abstract patterns. This is what powers Face ID unlocking your phone and voice assistants understanding what you say.
• Generative AI (GenAI) — a niche inside deep learning whose whole job is to create new content — text, code, images, audio. ChatGPT writing an email, DALL·E painting a picture, GitHub Copilot suggesting code: all GenAI.

And cutting across all of them:

• Natural Language Processing (NLP) — the field of getting machines to work with human language: reading it, understanding it, generating it. Autocomplete on your phone, Google Translate, and that same spam filter are all NLP. It draws on ML, DL, and GenAI depending on the job, which is why it doesn't sit in one neat ring.

Notice the nesting in action: a spam filter is AI (it acts smart), and more specifically ML (it learned from examples), and more specifically NLP (it works with language). Each label is a tighter circle around the last.

💡 Tip: The word that sets GenAI apart from everything above it is generate. A spam filter classifies (spam or not spam). A sales model predicts (tomorrow's revenue). GenAI creates — a sentence, a paragraph, a working function — something that never existed before you asked.

One Model, Many Jobs

Here's the shift that makes the new Nova possible.

Traditional machine-learning models are specialists. Picture hiring a different expert for every task: one person who only detects fraud, another who only forecasts demand, a third who only tags images. Each is excellent at exactly one thing and useless at the others — and every new task means recruiting and training a whole new specialist.

Generative AI is a generalist. It's one remarkably versatile hire who can summarise an article, answer a reader's question, translate a headline, and draft code — without being rebuilt for each task.

Watch what that means when four different jobs show up:

{{visual:specialist-vs-generalist}}

For Nova, this is the difference between building, hosting, and maintaining four separate models and pointing one model at all four jobs. That generality is exactly why Arjun can finally retire his hundred hand-tuned if statements.

The Engine Under the Hood: Foundation Models

The thing doing all this generating has a name: a foundation model.

A foundation model is a single, very large neural network trained on a massive pile of data — and it serves as the base engine you build everything else on. Here's the analogy: it's like a broadly-educated university graduate. They've absorbed an enormous amount of general knowledge, so you can hand them almost any job and they're useful on day one. You don't raise a new person from scratch for each role — and you don't train a foundation model from scratch either. You take one that already exists and put it to work.

And modern foundation models are multimodal. A modality is simply a type of data — text, image, audio, video. A multimodal model can take one type in and hand a different type back.

{{visual:modality-flow}}

• Text in, text out — "summarise this article."
• Audio in, text out — drop in a podcast, get a transcript (speech-to-text).
• Text in, image out — "a watercolour of the city skyline at dawn" becomes an illustration.
• Image in, text out — upload a photo, get back "a red bicycle leaning against a brick wall."

The key idea: the input type does not have to match the output type. A model can hear and write, or read and draw. It pulls this off by chopping every kind of data into a sequence of small pieces — a paragraph into words, a video into frames, audio into tiny sound samples — so it can process a picture the same way it processes a sentence.

For Nova, one multimodal foundation model is a big deal: the same engine could eventually read an article, answer a question about it, and narrate the answer aloud for a commute-home podcast — no three separate systems required.

What Makes a Model "Large"

The specific foundation models that power text assistants like Nova have their own name: Large Language Models, or LLMs. The name is basically the definition — break it into three words:

• Large — trained on an enormous dataset (much of the public internet) and built from a huge number of internal dials called parameters.
• Language — it works with human language: reading it and generating it.
• Model — at its core it's a mathematical model with one deceptively simple job: predict what comes next.

That last point is worth pausing on, because it's the whole trick. An LLM is, fundamentally, an extraordinarily good autocomplete. Give it "The capital of France is" and it predicts "Paris." Give it "Dear team, following up on" and it predicts a sensible next word, then the next, then the next. Do that with a big enough model trained on enough text, and the result looks and feels like genuine understanding.

Parameters: the dials of intelligence

A parameter is a single adjustable number inside the neural network. Picture a colossal mixing board with billions of tiny knobs. During training, the model checks its own guesses against real text and nudges every knob a hair — over and over, across trillions of words — until its predictions get good. The number of knobs sets how much nuance it can capture.

{{visual:training-loop}}

And the count of those knobs is what scales as models grow larger:

{{visual:param-scale}}

• More parameters → more capacity to absorb varied patterns → a smarter generalist.
• Fewer parameters → a smaller, cheaper, faster model — perfect for a narrow job, weaker on open-ended ones.

The growth has been staggering. To put real numbers on it: GPT-2 (2019) had about 1.5 billion parameters; GPT-3 (2020) leapt to 175 billion; today's frontier models are larger still. Each jump in scale, fed more and more varied data, is a step toward the industry's north star: AGI (Artificial General Intelligence) — a single system that could tackle any intellectual task a human can, from writing code to diagnosing an illness to composing music, instead of being brilliant at just one.

📌 Summary: An LLM is a foundation model specialised for language. It's "large" because of its sheer number of parameters — the dials it tunes during training — and that scale is what lets one model generalise across many tasks at once.

Why any of this scales: the Transformer

None of this would be practical without one architectural breakthrough from 2017: the Transformer. It underpins every modern LLM, and its superpower is parallelism.

Older models read a sentence one word at a time, like reading a page through a narrow straw — slow, and by the end they'd half-forgotten the beginning. The Transformer takes in the whole passage at once and can weigh how every word relates to every other. That's what lets these models train on internet-scale text in a reasonable time and still answer you in seconds.

(We'll open up the Transformer properly in a later chapter. For now: it's the engine that makes "large" possible.)

From Autocomplete to Agent: Two Ways to Use AI

Arjun now has the raw material — a generalist LLM. But how he wires it into LogicWizNews changes everything. There are two fundamentally different postures, and telling them apart is the mental model for this entire chapter.

AI-Assisted is human-driven. You sit in the loop at every step: you write a prompt, the AI suggests something, you decide whether to keep it, then you prompt again. It's a very smart autocomplete, and you hold the steering wheel the whole way. You already use this — GitHub Copilot proposing a line of code you accept or reject, or ChatGPT drafting an email that you then edit and send.

Agentic AI is goal-driven. You hand the system a high-level goal and a set of tools, and it manages the reasoning, the actions, and the checking on its own — looping through think → act → observe by itself, coming back only when it's done or genuinely stuck. Think "book me a table for four near the office this Friday": it checks your calendar, searches, reserves a table, and reports back.

{{visual:assisted-vs-agentic}}

Here's the same split in Nova's world:

• Assisted Nova — Arjun asks, "draft a two-line summary of this story." She writes it; he tweaks a word and hits publish. He drives every step.
• Agentic Nova — Arjun says, "keep the homepage fresh tonight." She watches the news feed, summarises each new story as it lands, updates the page, and pings Arjun only when something is genuinely ambiguous. She drives; he reviews.

The difference is who closes the loop. In assisted AI, the human closes every loop. In agentic AI, the agent closes the loop, and the human's job shifts to reviewing the result.

Autonomy vs Authority

Handing over the wheel raises an obvious question: how much should you actually let the agent do on its own?

The answer tracks the sensitivity of the task. A good rule of thumb: the harder something is to undo, the more a human should stay in the loop.

• Low sensitivity → high autonomy. Sorting today's articles into the right sections is cheap and easily reversed — let Nova run start to finish.
• High sensitivity → keep a human in the loop. Emailing a correction to 50,000 subscribers, or publishing a retraction, is public and hard to take back — Nova drafts it, but a human presses send.

It's the same instinct you'd use with a sharp new intern: let them reorganise the filing cabinet unsupervised, but have them run any message to a client past you first.

⚠️ Warning: Autonomy is a dial, not a switch. The more irreversible or public an action, the more you keep a human in the loop. We'll build these guardrails explicitly later — for now, remember that "agentic" never has to mean "unsupervised."

Nova's Road Ahead

That posture — from smart autocomplete to goal-driven agent — is the arc of this whole chapter. Here's where we're headed:

flowchart LR
  A[Plain LLM<br/>she can talk] --> B[Prompted<br/>she follows orders]
  B --> C[Tools + Memory<br/>she acts & remembers]
  C --> D[Reasoning Loop<br/>she thinks for herself]
  D --> E[Agent Family<br/>reflex to utility]

Right now, Nova is still just an idea with a very good engine sitting in a box. She can't actually say anything yet — Arjun hasn't connected her to a real LLM.

In the next episode we fix exactly that. You'll make Nova's first real API call to a live language model, learn to speak to it in the format it expects, and even swap between providers — OpenAI, Anthropic, Google — without rewriting your code. Nova is about to find her voice.