Nova Finds Her Voice

Part of the free Generative AI course on LogicWiz, module: The Agent Awakens.

Episode 9: Nova Finds Her Voice

"A model in a box is potential. A model behind an API is a colleague."

Nova's First Words

Last episode, Arjun met the engine — a generalist LLM. But it's sitting on someone else's servers, in a data centre humming with GPUs. He can't import it like a normal library, and he certainly can't fit hundreds of billions of parameters on his laptop.

So how does Nova actually speak?

The same way every production AI app does — from the ChatGPT app on your phone to Notion AI to the smart reply in your inbox. The app doesn't contain the model. It sends a request over the internet to the hosted model and gets text back.

Here's the mental model: it's like ordering at a restaurant. You don't march into the kitchen and cook — you hand your order to a waiter, and a finished dish comes back. The model's servers are the kitchen; you never set foot in there.

The waiter has a name. The model's creator ships a small helper library — an SDK (Software Development Kit: a package of ready-made functions that wrap the raw web request for you) — so you can place your order in a few lines of Python instead of hand-crafting network calls.

You create a client (your authenticated connection to the model), hand it a message, and read the reply. Let's give Nova her very first words:

That's it. That's a real language model answering a real question, live. No if statements, no hardcoded replies — Nova is generating an answer she's never seen before.

💡 Tip: The client needs an API key — think of it as a membership card. It proves you're allowed in and tells the provider whose account to bill for the meal. Never paste a key straight into your code, where it could leak into version control and let a stranger run up your bill. In production you load it from an environment variable or a secrets manager; here, it's already wired up so you can focus on the ideas.

The Shape of a Conversation: Messages & Roles

Look again at what we sent. It wasn't a bare string — it was a list of messages, and each message carries a role. That structure is the heart of every chat model.

Think of it like a film script. Every line is tagged with who is speaking — and one of those speakers is the director, whose notes the audience never sees but which shape the whole performance.

Two roles matter right now:

system — the director's note. It sets Nova's persona, tone, and rules before the user ever speaks. This is where you write: "You are Nova, a concise news assistant for LogicWizNews. Never guess about stories that aren't published yet." (If you've ever set "custom instructions" in ChatGPT, that's a system message.)
user — the actual question or request from the person: "What's the top story today?"

(There's a third role, assistant, for the model's own past replies — we'll lean on it heavily once Nova needs memory.)

Change the system message and you change who Nova is — same question, completely different voice. A "serious financial analyst" and a "cheerful sports commentator" will describe the exact same market news in wildly different ways. Try it:

The Creativity Dial: `temperature`

One more knob worth knowing. temperature controls how adventurous the model is when picking its next word. Picture a spice dial on a stove.

temperature=0 → follow the recipe exactly. Focused and near-deterministic: ask twice, get almost the same answer. This is your setting when Nova extracts a date, returns a category, or answers yes/no — anything that has to be reliable.
Higher temperature (up to ~2) → improvise and add surprises. More variety, more creativity: ask twice and you'll get two different answers. This is your setting when Nova brainstorms ten headline options.

⚠️ Warning: For anything another system depends on — a category label, a JSON field, a yes/no — reach for a low temperature. Randomness is a feature for creativity and a bug for reliability.

Reading the Reply: Peeling the Response

When you call the model you don't get a plain string back — you get a structured response object, and the text you want is tucked a few layers deep. Think of it as a parcel with packing material around the thing you actually ordered.

Why the nesting? Because the model can return several alternative replies at once — like asking a designer for three logo options. You almost always want the first one, so you dig down to it:

In code, that path is:

response.choices[0].message.content

Read it left to right — it's the same Russian-doll idea from last episode: choices (the list of candidate replies) → [0] (the first, usually only, candidate) → message (the reply object) → content (the actual text). Memorise that chain — you'll type it in every OpenAI call you ever write.

One Code, Many Models

Here's a problem Arjun hits fast. OpenAI's GPT models are one option, but there's also Anthropic's Claude and Google's Gemini — and each provider ships its own SDK, with its own function names and its own response shape. Want to switch Nova from GPT to Claude to compare their summaries? Rewrite your integration code. Every time.

It's the old travel-plug problem: every country has a different socket, so you'd need a different adapter for each.

There's a cleaner way. OpenRouter is the universal travel adapter. It's a single service that sits in front of hundreds of models and speaks the OpenAI format for all of them. You keep using the exact OpenAI SDK you already know — you just point the client at OpenRouter's address and name the model you want.

Only two things change in your client setup — the address it calls (base_url) and the key it uses. Your message-building and response-parsing code stays identical:

# Same OpenAI SDK — just re-aim it at OpenRouter
client = OpenAI(
    base_url="https://openrouter.ai/api/v1",
    api_key=OPENROUTER_API_KEY,
)

# Now name any model, provider-first: "provider/model"
response = client.chat.completions.create(
    model="anthropic/claude-sonnet-4.5",   # or "openai/gpt-5", "google/gemini-2.5-pro", "x-ai/grok-4"
    messages=[{"role": "user", "content": test_prompt}],
)

To swap models, you change one string. That's a genuine superpower. When Arjun wants to know whether GPT or Claude writes punchier LogicWizNews summaries — or whether a cheaper model is good enough to save money — he runs the same code twice with two different model strings and compares. No rewrite, no lock-in.

📌 Summary: To call an LLM you use the provider's SDK, authenticated with an API key. You send a list of role-tagged messages (system sets the persona, user asks), tune temperature for reliability vs. creativity, and read the reply at response.choices[0].message.content. OpenRouter lets one codebase reach many models by swapping a base URL and a model name.

SDK or raw API?

Under the hood, the SDK is just making a web request — an API call — that bundles up the model name, your settings, and your messages and ships them to the provider's servers. You could write that raw request yourself (like calling the taxi company and negotiating the fare), but the SDK is the ride-hailing app: same ride, far less hassle. Either way, the model lives in the cloud, so you never manage any infrastructure. Get a key, and you're calling a frontier model from your laptop.

What Nova Learns Next

Nova can speak now — but she says whatever she likes, however she likes. A system message nudges her, yet the moment a task gets specific ("return only JSON," "answer in exactly three bullet points"), gentle nudging isn't enough.

Next episode is where Nova learns discipline. We'll dig into prompt engineering — the craft of writing instructions so precise that the model does exactly what you need, every time. It's the difference between a model that's impressive in a demo and one you can actually build a product on.