The mechanics under the API: tokens, context windows, sampling, and the knobs that actually change output.
17 articles
Listed inTokensHow LLMs Workon
Why character and word counts are wrong by enough to matter, how BPE actually splits your text, and how to budget a request so the model never gets cut off mid-sentence.
Listed inHow LLMs WorkHow LLMs Workon
The transformer paper. Worth reading once even if you never train anything — everything downstream inherits its shape.
Listed inPrompt CachingHow LLMs Workon
How cache breakpoints work, what invalidates a prefix, and the latency and cost numbers to expect.
Listed inSampling ParametersHow LLMs Workon
How decoding settings turn one distribution into many possible responses.
Listed inStreaming ResponsesHow LLMs Workon
Token-by-token delivery, and the UX and error handling it forces you to design for.
Listed inTokensHow LLMs Workon
The BPE paper — why tokenizers merge frequent byte pairs, and why your rare identifiers cost so much.
Listed inTokensHow LLMs Workon
The unit models actually read and bill for, and why character counts mislead you.
Listed inFine-tuningHow LLMs Workon
Adapting a model's weights to your task, and the far cheaper things to try first.
Listed inPre-trained ModelsHow LLMs Workon
Why you almost always start from someone else's weights, and what that inherits.
Listed inTop-KHow LLMs Workon
Restricting the candidate pool to the k most likely tokens, and how it interacts with top-p.
Listed inTemperatureHow LLMs Workon
The randomness dial: what it does to the distribution, and sane defaults per task.
Listed inContext WindowHow LLMs Workon
The hard budget every prompt shares, and what gets dropped when you exceed it.
Listed inPrompt CachingHow LLMs Workon
Reusing a stable prefix across calls to cut latency and cost on long system prompts.
Listed inTypes of ModelsHow LLMs Workon
Base, instruction-tuned, reasoning, embedding, and multimodal models, and when each fits.
Listed inRepetition PenaltiesHow LLMs Workon
Frequency and presence penalties, and why they are the wrong fix for most loops.
Listed inHow LLMs WorkHow LLMs Workon
Next-token prediction, attention, and the path from raw text to a streamed response.
Listed inTop-PHow LLMs Workon
Nucleus sampling — trimming the tail by probability mass instead of by rank.