The smallest unit of text that an AI language model reads and generates. Before a model processes any input, it breaks the text into tokens — chunks that might be a whole word, part of a word, or even a single character. The word "darkness," for example, becomes two tokens: "dark" and "ness." Every interaction with a chatbot, from your prompt to its reply, is measured in tokens.

Tokens matter for data reporters in two practical ways. First, they're how AI companies charge for API access: prices are quoted per million tokens, so understanding token counts helps you budget when you're using an AI service to, say, summarize thousands of public records or extract data from court filings. Second, tokens determine how much text a model can handle at once — its context window. A model with a one-million-token context window can process roughly 750,000 words in a single prompt, enough to hold an entire book or a year's worth of city council minutes.

Because models see tokens rather than letters, they can stumble on tasks that seem simple to humans — like counting the r's in "strawberry" or doing arithmetic — since a number like "380" might be one token while "381" is split into two. This quirk is worth keeping in mind when asking an AI to work with structured data where precision matters.