LLM Foundations and Architecture
Free NVIDIA-Certified Professional: Generative AI and LLMs practice — 6 questions on LLM Foundations and Architecture, with explanations. No sign-up. Full 12-question mixed test →
A team deploying a 70B-parameter decoder-only LLM for high-throughput inference finds that KV cache memory dominates GPU memory when serving long contexts at batch size 64. They need to cut KV cache size substantially while retaining quality close to full multi-head attention. Which architectural choice best addresses this?
GQA groups query heads to share a smaller set of K/V projections, cutting KV cache memory roughly by the grouping factor while preserving quality much closer to full MHA than MQA does.
A pretrained model uses RoPE (rotary position embeddings) with a native training context of 4K tokens. The team must serve 32K-token contexts without retraining the model from scratch. Which approach is most appropriate?
RoPE scaling methods (NTK-aware interpolation, YaRN) adjust the rotation base/frequencies so position encodings remain well-conditioned at longer lengths, and brief fine-tuning adapts the model to the extended range cheaply compared to full retraining.
Profiling a transformer training run on H100 GPUs shows the attention operation is bottlenecked not by FLOPs but by repeated reads/writes of the O(n^2) attention score matrix to HBM. Which technique specifically targets this bottleneck?
FlashAttention is an IO-aware algorithm designed specifically to reduce HBM read/write traffic by computing attention in tiles within fast on-chip SRAM, directly addressing memory-bandwidth-bound attention rather than compute-bound bottlenecks.
A sparse Mixture-of-Experts model uses top-2 routing among 8 experts per layer with a capacity factor of 1.25. During training, many tokens are dropped due to expert overflow, hurting quality. Which configuration change most directly reduces token dropping while keeping the same number of experts?
Capacity factor directly sets the per-expert token buffer size in a training batch; raising it (e.g., from 1.25 to 2.0) gives experts more headroom to accept tokens before overflow forces drops, directly reducing dropped-token quality loss.
Comparing byte-pair encoding (BPE) with SentencePiece's unigram language model tokenizer for a multilingual LLM handling low-resource, morphologically rich languages, which statement is most accurate?
The unigram model treats tokenization probabilistically over multiple candidate segmentations and picks the most likely one under a learned vocabulary, which tends to produce more linguistically coherent subwords for complex morphology than BPE's deterministic greedy merge process.
Given a fixed training compute budget C, Chinchilla scaling law research found that many earlier large models (e.g., GPT-3 175B) were compute-suboptimal because they were:
Chinchilla's analysis showed that for a fixed compute budget, model size and training token count should scale together roughly proportionally; earlier large models allocated too much compute to parameters and too little to training tokens, making them undertrained relative to compute-optimal allocation.
Ready for the real thing?
The full course: two full-length practice tests, video lessons for every exam domain, hands-on labs and detailed explanations.
$129.99 $34.99 with code FREETEST33 — valid through September 7.
Get my $34.99 deal →