What is an LLM (large language model)?

Strip away the chat interface and the branding, and a large language model does one thing: it reads a stretch of text and predicts what comes next. Everything a person experiences as answering questions or reasoning through a problem is built on top of that single mechanical act, repeated one fragment at a time.

What a large language model actually is

An LLM is a neural network trained on a very large body of text. Almost all of them share one architecture, the transformer, and one training goal: given some text, predict the next unit of text. That unit is a token, usually a word or a word fragment rather than a whole word. During generation the model predicts a token, appends it, and predicts again, so a paragraph is thousands of these guesses laid end to end.

The “large” is literal on two counts. The training data runs to a large fraction of the public web and digitised books. The model’s learned settings, its parameters, number in the billions for the biggest systems.

Large language model (LLM): a neural network, almost always a transformer, trained on a large text corpus to predict the next token given the preceding text. At generation time it strings those predictions together into fluent output.

How next-token prediction works

Text goes in as tokens. For each position, the model outputs a probability for every token it could produce next, one is chosen, and the process repeats. The model is never told the rules of grammar or the facts of history. It only adjusts its parameters to make the real next token more likely across the training data. Fluent grammar and apparent world knowledge both fall out of getting very good at that one prediction task.

Why the transformer changed things

The behaviours people find striking rest on the architecture underneath. The transformer, introduced by Vaswani and colleagues in 2017, replaced the earlier recurrent designs that read text strictly left to right, one step at a time. Its central idea is attention: for each token, the model weighs how much every other token in the context should influence the prediction. The paper’s own framing is that the architecture is “based solely on attention mechanisms, dispensing with recurrence and convolutions entirely” (Vaswani et al., 2017). Attention made it practical to train on enormous corpora, because the computation parallelises in a way the older sequential models could not.

Where the useful behaviours come from

The model was trained only to predict text, yet at scale it does far more. Brown and colleagues showed that a sufficiently large model could perform a new task from nothing but a few examples written into the prompt, “without any gradient updates or fine-tuning” (Brown et al., 2020). This is in-context learning, and it was not designed in. It appeared as the model grew.

That pattern has a name. Wei and colleagues define an ability as emergent when it “is not present in smaller models but is present in larger models” (Wei et al., 2022). Step-by-step reasoning is one such ability: prompting a model to work through a problem in intermediate steps improves its answers, and that gain “emerge[s] naturally in sufficiently large language models” via chain-of-thought prompting (Wei et al., 2022). No one wrote a reasoning module. Scale plus attention produced the behaviour.

Why it predicts rather than knows

The single most useful thing to understand about an LLM is that it has no store of facts to look up. When it answers a question correctly, it is producing the most plausible continuation of your prompt, and the correct answer happens to be that continuation. When it answers wrongly with total confidence, the same machinery is running. A fluent falsehood is just a plausible continuation that is not true. This is why models hallucinate, and it is not a defect a bigger model fully removes.

It also explains the tooling built around these models. Retrieval-augmented generation, or RAG, fetches real documents and places them in the prompt so the model predicts from grounded text rather than from memory alone. An AI agent wraps the model in a loop that lets it act and check each result before moving on, compensating for the fact that any single prediction carries no guarantee of being right. Both exist because prediction, however fluent, is not retrieval.

Sources

• Vaswani, A. et al. Attention Is All You Need. arXiv:1706.03762, 12 Jun 2017.
• Brown, T. et al. Language Models are Few-Shot Learners. arXiv:2005.14165, 28 May 2020.
• Wei, J. et al. Emergent Abilities of Large Language Models. arXiv:2206.07682, 15 Jun 2022.
• Wei, J. et al. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. arXiv:2201.11903, 28 Jan 2022.