Fundamentals of Large Language Models

top_p defines the cumulative probability mass (nucleus) from which tokens are sampled. A very low top_p (0.1) restricts the candidate pool to only the most likely tokens regardless of how temperature reshapes the overall distribution, so the effective diversity introduced by high temperature is largely negated.

Encoder-decoder architectures (e.g., T5-style models) are designed exactly for sequence-to-sequence tasks: the encoder produces a full contextual representation of the source sequence, and the decoder attends to that representation while autoregressively generating the target sequence.

Subword tokenizers (e.g., BPE) build a fixed vocabulary from training data. Words not seen during training, such as invented brand names, cannot be represented as a single token and are decomposed into multiple smaller subword or byte fragments, increasing the token-to-character ratio.

The core defense against prompt injection from retrieved content is architectural: the system must clearly delineate untrusted retrieved text as data to be referenced, not as executable instructions, and enforce that boundary via system prompts, instruction hierarchies, and sanitization rather than trusting model behavior alone.

Parameter-efficient fine-tuning methods like T-Few insert and train a small number of additional parameters (or a subset of existing layers) while freezing the bulk of the pretrained weights, directly matching the goal of minimizing trainable parameters and cost while still adapting the model to the domain task.

Chain-of-thought prompting explicitly instructs the model to generate intermediate reasoning steps before the final answer. This decomposition lets the model condition each step on prior correct reasoning, which measurably improves accuracy on multi-step arithmetic and logic tasks compared to direct-answer prompting.