196: AI 101: Tokens, Context Windows & Why AI Makes Dumb Mistakes Explained

Get AI-Ready with Kyle’s 5-Day Challenge: https://aiwithkyle.com/join

Subscribe and turn on notifications to catch the next live stream: https://www.youtube.com/channel/UChlLglbHDASnoGkbjDeHnQg

Summary:

Tokenization is one of those concepts that sounds technical until you realise it explains basically everything. Why AI can't count the letters in "raspberry." Why a bigger context window sometimes makes things worse. Why your API bill might be quietly spiralling. This is a 101 lesson on tokens: what they actually are, how models use them, and why the common assumption that a token equals a word is close but not quite right.

The second half gets into context windows, the working memory of a model, and why the "just give it everything" instinct is both expensive and counterproductive. Whether you're building on the API or just a heavy user wondering why long threads start getting weird, understanding this stuff changes how you work. Also, token maxing is a trend and it is, in my view, a bit dumb. We get into that too.

—— Time Stamps ——

0:00 Intro: What Tokenization Is & Why It Matters 1:27 Tiktokenizer Demo: How Words Break Into Tokens 4:18 Token IDs: How Models Store Words as Numbers 5:35 Tokens Across Languages: Why English Is Cheapest 7:59 The Raspberry Problem: Why AI Can't Count Letters 9:49 Context Windows Explained: What "1 Million Tokens" Means 11:40 Context Compaction: Why Chats No Longer Cut You Off 13:27 Why a Bigger Context Window Isn't Always Better 16:18 API Pricing: How Tokens Drive Your AI Costs 18:42 "Token Maxing": The Wasteful AI Trend to Avoid 20:13 Finding Real Context Limits & DeepSeek vs GPT Pricing 22:36 Rules of Thumb: Optimize for Useful Context 25:12 Wrap-Up & Andrej Karpathy Recommendation 26:20 Newsletter & Sign-Off

— Useful Resources ——

Find everything else at https://aiwithkyle.com/