Prompt Engineering and Adaptation
Free NVIDIA-Certified Professional: Generative AI and LLMs practice — 6 questions on Prompt Engineering and Adaptation, with explanations. No sign-up. Full 12-question mixed test →
A team implements self-consistency decoding for a multi-step arithmetic reasoning task: they sample 15 chain-of-thought completions at temperature 0.7 and take the majority-vote final answer. Accuracy improves over greedy decoding but inference cost rises significantly. Which statement BEST explains the mechanism behind the accuracy gain?
Self-consistency samples multiple independent CoT reasoning paths at nonzero temperature, then aggregates (typically majority vote) over the final answers. Because reasoning errors tend to be inconsistent across samples while the correct answer path recurs, marginalizing over samples reduces the impact of any single faulty chain and improves accuracy at the cost of extra inference compute.
An engineer builds a ReAct agent that interleaves Thought/Action/Observation steps to answer questions using a search tool and a calculator tool. During testing, the agent frequently emits a syntactically valid Action call to the calculator with a malformed expression, causing a tool error, but the agent then loops repeatedly retrying the same malformed call instead of recovering. What is the MOST likely root cause and fix?
ReAct relies on the Observation step surfacing tool results (including errors) back into the context so the model can reason about what went wrong and adjust its next Action. If the error/observation isn't clearly returned and the prompt doesn't instruct the model to use it for correction, the model has no signal to change behavior and will repeat the same malformed call.
A production system must guarantee that every LLM response is a syntactically valid JSON object matching a fixed schema (specific keys, enum-constrained string values, integer fields), with zero tolerance for malformed output reaching downstream services. Prompt-only instruction to "respond in JSON" produces a 92% valid-output rate. Which approach provides the strongest guarantee of schema compliance?
Constrained/structured decoding enforces validity at the token-sampling level by masking out tokens that would violate the grammar or schema at each generation step, guaranteeing syntactic and structural compliance regardless of prompt phrasing — this is the only listed option that provides a hard guarantee rather than a probabilistic improvement.
A team must answer customer questions about a rapidly changing product catalog (prices and stock updated hourly) with source citations required for compliance, using a budget that precludes any model fine-tuning and requires sub-2-second latency. Which adaptation strategy is MOST appropriate?
RAG is designed exactly for this scenario: frequently changing factual data, need for verifiable source citations, and no budget/tolerance for retraining. Indexing updates hourly keeps the retrieval corpus current without touching model weights, and retrieved passages can be directly cited, all while maintaining low latency since retrieval + generation is faster than any training cycle.
During prompt evaluation for a summarization task, a team A/B tests two prompt variants against a held-out set of 200 documents, using ROUGE-L score as the sole evaluation metric. Variant B scores higher on ROUGE-L but human reviewers rate Variant A's summaries as more useful and factually accurate. What is the MOST likely explanation and appropriate corrective action?
ROUGE-L measures n-gram/longest-common-subsequence overlap with reference text, which correlates loosely with fluency and coverage but does not directly measure factual accuracy or task usefulness. A prompt can game overlap metrics (e.g., by closely mirroring reference phrasing) while still hallucinating or omitting key facts. Best practice for prompt iteration is to combine automated metrics with human or LLM-as-judge evaluation targeting faithfulness/usefulness directly.
A team designs a few-shot prompt with 8 in-context examples for a classification task and finds that swapping the order of the examples (same examples, different sequence) causes classification accuracy to vary by over 15 percentage points on the same test set, with no change to model or examples' content. What does this behavior primarily indicate about few-shot in-context learning, and what is the recommended mitigation?
Research on in-context learning has documented that LLMs are sensitive to the order of few-shot exemplars due to positional/recency biases in attention, causing significant variance in output quality purely from reordering. Practitioners mitigate this by empirically testing several orderings on a validation set, ensembling predictions across orderings, or using calibration techniques, rather than assuming order is irrelevant.
Ready for the real thing?
The full course: two full-length practice tests, video lessons for every exam domain, hands-on labs and detailed explanations.
$129.99 $34.99 with code FREETEST33 — valid through September 7.
Get my $34.99 deal →