Prompt Engineering, LLM Workflows, and RAG
Free GSDC Certified Forward Deployed Engineer practice — 6 questions on Prompt Engineering, LLM Workflows, and RAG, with explanations. No sign-up. Full 12-question mixed test →
A team needs a chain-of-thought math reasoning system to reduce output variance for numeric answers where the LLM is right about 70% of the time on a single attempt. The latency/cost budget allows up to 5x the base call cost. Which technique BEST fits this constraint?
Self-consistency exactly matches the pattern of sampling N reasoning paths and aggregating via majority vote, and 5 samples fits neatly within a 5x cost budget while directly reducing variance-driven errors.
A RAG system retrieves the top-10 chunks (about 500 tokens each) and concatenates them into a 16K-token context window, with the highest-scoring chunk placed first and the lowest last. Users report that answers frequently miss facts clearly present in the chunk ranked 5th or 6th. Which of the following best explains this failure mode?
This is the well-documented 'lost in the middle' effect: recall degrades for facts placed in the middle of long contexts regardless of their relevance ranking, exactly matching the symptom of a confirmed-relevant chunk at position 5-6 being missed.
A legal document search RAG system must retrieve results based on both exact citation matches (e.g., 'Section 12.3(a)') and semantic similarity to natural-language queries. Pure dense (embedding-based) retrieval alone performs poorly on citation lookups. Which retrieval architecture BEST addresses this requirement?
Hybrid retrieval combines BM25's strength at exact token/citation matching with dense embeddings' strength at semantic similarity, fused via a method like reciprocal rank fusion — the standard fix for this exact class of problem.
A RAG pipeline retrieves the top-50 candidate chunks from a vector database (optimizing for recall), and must then select a small, highly relevant subset to fit a tight LLM context budget. Which architecture correctly applies a cross-encoder re-ranker in this pipeline?
The correct pattern is retrieve-broad-then-rerank-narrow: a bi-encoder/vector search maximizes recall over the full corpus, then a cross-encoder scores each query-chunk pair for precision, and only the top-N re-ranked chunks are forwarded to the LLM.
A team builds an automated evaluation pipeline using an LLM-as-judge to score 10,000 chatbot responses for helpfulness, calibrated against 200 human-labeled examples. The judge model shows a systematic bias toward favoring longer responses regardless of actual quality. Which mitigation is MOST appropriate?
Length bias in LLM-as-judge setups is a known failure mode; the standard mitigation is rubric engineering that explicitly decouples the scored dimension (helpfulness) from length, reinforced with contrastive calibration examples.
Which statement correctly distinguishes the ReAct prompting pattern from the Reflection pattern?
ReAct follows a reason-act-observe loop where the model takes actions (typically tool/environment calls) and incorporates observations into further reasoning; Reflection is a self-critique loop where the model evaluates and revises its own prior output, generally without needing external tools.
Ready for the real thing?
The full course: two full-length practice tests, video lessons for every exam domain, hands-on labs and detailed explanations.
$109.99 $34.99 with code FREETEST33 — valid through September 14.
Get my $34.99 deal →