I Built a Suite of AI Apps. Here's What Every AI Concept Actually Means.

Over the last couple of years I've built and shipped a small pile of AI products. A suite of consumer apps — study help, a legal advisor, a content creator, a farming assistant in Hindi and Punjabi — plus a phone-based assistant for a company that manages tens of thousands of buildings, and a Slack bot that keeps my team in sync.

Along the way I kept bumping into the same handful of concepts. They have intimidating names and most explanations make them sound harder than they are. So here's my version: each idea, the story of the app that taught it to me, and what it actually means — no jargon, I promise.

1. Sometimes you want a robot. Sometimes you want a poet.

I built a feature in my legal app that pulls the exact section of a law and explains it. It has to be exact. No creativity, no "interpretation," no surprises — just the right section, the same way, every time.

Then I built a content tool that writes social posts and marketing copy. That one needs the opposite: personality, flair, word choices you didn't see coming.

Same underlying model. Opposite needs. And early on, the legal one kept "getting creative" with the wording while the content one kept writing safe, boring copy. The fix was one single dial: temperature.

So, what is temperature? It's a single setting that controls how safe versus creative an AI's answers are. Turn it down and the model becomes a reliable robot. Turn it up and it becomes a poet. Most people never touch it — and then wonder why their app feels either dull or unhinged.

2. The agent has the memory of a goldfish

My lecture app takes a 90-minute class and turns it into clean notes, a summary, and exam-style questions. The first time I tried it, I did the obvious thing: shove the entire transcript at the model and ask for notes.

It choked. Long recordings simply didn't fit, and when they almost fit, the model would "forget" the beginning of the lecture by the time it reached the end — like a student who slept through the first hour.

So, what is a context window? It's the model's short-term memory — the total amount of text it can hold in its head at one time. Go past it and the earliest stuff falls out the back. The real skill isn't dumping everything in; it's deciding what the model actually needs to see for the question at hand. For a long lecture that meant breaking it into chunks, summarising each, then summarising the summaries.

3. You're not paying per question. You're paying per word.

The first month one of my consumer apps got popular, I felt great — right up until I saw the bill. AI isn't like normal software, where one more user costs almost nothing. Every single answer costs real money, and the meter runs on tokens.

So, what is a token? Roughly, a chunk of a word — the unit a model reads and writes in. "Hello" is one token; a long word might be two or three. And here's the part that bites you: you pay for everything you send in (the user's question, the document, your instructions) and everything the model sends back. A chatty assistant answering a long question is quietly expensive.

This is why my apps run on yearly plans with sensible limits, and why I instrument cost per request from day one. Going "viral" while every free user racks up an unbounded bill isn't a success story — it's an invoice you can't pay.

4. Don't make it memorise. Give it the textbook.

The hardest thing I've built is a voice assistant that takes work orders over the phone. A caller half-remembers a building's name on a noisy line, and the agent has to pin down the right one out of tens of thousands. My first version asked the model to just "know" the answer. It confidently picked the wrong building constantly.

The fix wasn't a smarter model. It was giving the model the textbook. Before it answers, I search a proper index for the buildings that actually match, hand the model just those few, and ask it to choose from them. Accuracy shot up overnight.

So, what is RAG? It's an ugly acronym (retrieval-augmented generation) for a simple idea: look it up first, then answer. Instead of trusting the model's memory, you fetch the relevant facts from a source you trust and feed them in. It's the difference between an open-book exam and a from-memory one — and it's the single biggest weapon against made-up answers.

5. The job description you write before anyone says a word

The same model powers my farming assistant and my legal advisor, but they behave like completely different people. The farming one talks like a helpful neighbour, in simple Hindi or Punjabi, patient with someone who's never used an app. The legal one is precise, careful, and never guesses about the law.

I didn't fine-tune two different models for that. I just wrote two different system prompts.

So, what is a system prompt? It's the standing instruction you give the model before the user ever speaks — its job description, its tone, its rules, the things it must never do. Get it right and the same engine becomes a calm teacher, a cautious lawyer, or a punchy copywriter. Get it vague and you get a generic chatbot that sounds like every other one.

6. When you want an answer, not an essay

I built a tool that reads a messy PDF invoice and pulls out the numbers — vendor, date, total, line items. The trouble is, models love to chat. Ask for the total and you'd get "Sure! The total on this invoice appears to be ₹12,400, though you may want to double-check…" — lovely for a human, useless for code that just needs a number.

So, what is structured output? It's telling the model to reply in a strict, machine-readable shape — usually JSON, basically a tidy labelled box — instead of prose. "Give me only this: vendor, date, total, items." Now my code gets clean fields it can drop straight into a database, with no fishing the number out of a paragraph. The moment an AI feature has to talk to the rest of your software, this is the concept that makes it reliable.

7. Teaching it to press buttons, not just talk

My team's Slack bot can answer "what did everyone ship last week?" That sounds like a chat question, but the model can't answer it from thin air — the information lives in a database, not in its training. So I let it use tools: when it sees that question, it doesn't make something up, it calls a function that actually queries our data, gets real numbers back, and then explains them.

So, what is function calling (or "tool use")? It's giving the model a set of buttons it's allowed to press — look up the weather, search a database, send a message — and letting it decide when to press them. This is the leap from a chatbot that talks to an agent that does things. My farming app uses it to fetch live weather; the Slack bot uses it to pull real activity. The model stays the brain; the tools are its hands.

8. It will lie to you with a completely straight face

There's a specific kind of dread in watching an AI tell a paying user something that's flatly wrong — with total confidence. In a legal app, that's not just embarrassing, it's dangerous. Early on I treated this as a "later" problem. By the time I noticed I needed safeguards, the agent had already said things it shouldn't have.

So, what is hallucination? A model is built to produce text that sounds right, not to know when it's actually wrong. So sometimes it just… makes things up, fluently. There's no built-in alarm bell.

The answer isn't a magic setting — it's guardrails you build yourself: give it real sources to quote from (back to retrieval), check what it's allowed to claim, and let it say "I'm not sure — let me get you a human" instead of inventing an answer. Your users can't tell confidence from correctness. Your guardrails have to.

The whole thing in one breath

If you remember nothing else: temperature sets robot-vs-poet. Tokens are what you pay for, in and out. The context window is its short-term memory. RAG means look it up before you answer. The system prompt is its job description. Structured output is for when code needs an answer, not an essay. Function calling turns a talker into a doer. And hallucination is why you build guardrails before you need them.

None of this requires a PhD. It requires building something real, watching it fail in an interesting new way, and learning the name of the thing that just bit you. Eight ideas. That's the whole vocabulary behind every AI agent I've shipped.

Frequently Asked Questions

What does temperature mean in an AI model?

Temperature is a single setting that controls how safe versus creative an AI's answers are. Low temperature makes it predictable and repeatable — good for exact tasks like quoting a law or pulling a number off an invoice. High temperature makes it varied and surprising — good for writing marketing copy or brainstorming. Same model, opposite behaviour, one dial.

What is a token and why does it cost money?

A token is roughly a chunk of a word — the unit an AI model reads and writes in. You don't pay per question; you pay per token, for everything you send in and everything it sends back. Long documents and chatty answers cost more, which is why apps that scale have to plan limits and tiers before they grow, not after.

What is a context window?

The context window is the model's short-term memory — the total amount of text it can hold in mind at once. Anything beyond it gets forgotten. For a 90-minute lecture or a long document, you can't just dump everything in; you have to decide what the model actually sees, which is its own design problem.

What is RAG (retrieval-augmented generation)?

RAG means giving the model the right reference material to answer from, instead of hoping it memorised the answer. You search a trusted source first, hand the model only the relevant passages, and ask it to answer from those. It's the difference between an open-book exam and one from memory — and it dramatically reduces made-up answers.

What is a system prompt?

A system prompt is the standing instruction you give the model before the user ever speaks — its job description, tone, rules and boundaries. A farming assistant gets told to talk like a helpful neighbour in simple Hindi; a legal assistant gets told to be precise and cautious. Same engine, completely different employee.

Why do AI models hallucinate, and what can you do about it?

Models are built to produce plausible-sounding text, not to know when they're wrong, so they will state false things with total confidence. You manage it with guardrails: give the model real sources to quote from, validate what it's allowed to claim, and let it say "I don't know" instead of inventing an answer. Users can't tell confidence from correctness, so your checks have to.