Evaluation and Responsible AI
Free NVIDIA-Certified Professional: Generative AI and LLMs practice — 6 questions on Evaluation and Responsible AI, with explanations. No sign-up. Full 12-question mixed test →
A financial services company deployed a customer-facing LLM assistant. They need the assistant to refuse specific stock buy/sell recommendations while still allowing general financial education topics, and must catch this both when users ask indirectly and when generated responses drift toward recommendations mid-conversation. Which NeMo Guardrails configuration BEST meets this requirement?
Combining dialogue/topical rails (conversation-level intent control) with output rails (catching drift in generated content) addresses both indirect requests and responses that drift into prohibited territory during generation.
A team must evaluate a summarization LLM's output for factual consistency against 50,000 source documents, but budget does not allow full human review of every summary. They still need confidence that automated scores correlate with human judgment. Which evaluation strategy is BEST?
A hybrid approach scales via automated judging while a calibration subsample validates reliability by measuring agreement with human raters — the standard practice for trustworthy large-scale LLM-as-judge evaluation.
Which technique is most effective for detecting hallucinations in a RAG-based LLM system that must ground its responses in retrieved documents?
Attribution/grounding scoring (e.g., NLI entailment or citation checking between generated claims and retrieved context) directly measures whether output is supported by sources, which is the core mechanism for detecting RAG hallucinations.
You are configuring NeMo Guardrails to detect and block jailbreak attempts (e.g., 'ignore previous instructions' prompt-injection patterns) before the LLM generates a response. Where should this primary detection logic be implemented?
Input rails intercept and evaluate the user message before it is passed into the core LLM prompt, making them the correct place to catch prompt-injection and jailbreak patterns before generation occurs.
In responsible AI evaluation for LLMs, 'allocational bias' specifically refers to:
Allocational bias refers to disparities in how resources, opportunities, or outcomes (e.g., loan approvals, hiring recommendations) are distributed across groups as a result of model decisions, distinct from representational bias.
During error analysis of an LLM used to generate resume screening summaries, the team wants to detect whether the model exhibits differential performance across gender-associated names in input resumes. What is the BEST evaluation approach?
Counterfactual/stratified perturbation testing (swapping only the sensitive attribute while holding content constant) isolates the effect of that attribute and is the standard technique for detecting demographic disparate treatment.
Ready for the real thing?
The full course: two full-length practice tests, video lessons for every exam domain, hands-on labs and detailed explanations.
$129.99 $34.99 with code FREETEST33 — valid through September 7.
Get my $34.99 deal →