Application Development
Free Databricks Certified GenAI Engineer Associate practice — 6 questions on Application Development, with explanations. No sign-up. Full 12-question mixed test →
A GenAI application uses MLflow to track multiple LLM experiments with different prompt templates. The team needs to programmatically retrieve the run that achieved the highest F1 score for a specific prompt template version and deploy it to production. Which MLflow API combination accomplishes this?
mlflow.search_runs() supports filter_string for querying runs by parameters (like prompt template version), can sort results by any metric column including metrics.f1_score, and returns a DataFrame making it easy to select the top run. The run_id can then be passed directly to mlflow.register_model() for model registry and deployment. This is the most efficient programmatic approach.
A company deploys a RAG application where a Databricks vector search index is queried to retrieve context, which is then passed to a foundation model API. Users report that responses sometimes contain information contradicting the retrieved documents. The development team suspects the issue occurs when the LLM ignores retrieved context. Which implementation change BEST ensures the model strictly adheres to provided context?
Explicit grounding instructions in the system prompt combined with validation is the standard RAG pattern to prevent hallucination. Instructing the model to acknowledge when context is insufficient prevents fabrication, and response validation can detect when the model deviates from provided context. This addresses the root cause: the model generating information not present in retrieved documents.
An application uses LangChain with Databricks SQL to query structured customer data and generate natural language summaries. The chain executes: SQLDatabaseChain → LLM generates SQL → Execute query → LLM summarizes results. During testing, some generated SQL queries fail with syntax errors. Which LangChain configuration reduces SQL generation errors?
Few-shot prompting with schema-specific examples is proven to dramatically improve SQL generation accuracy. Providing 3-5 concrete examples of correct queries for the specific database schema helps the LLM understand table relationships, column names, and query patterns. This is the standard approach for improving Text-to-SQL accuracy in production.
A chatbot application uses Databricks Model Serving to host a fine-tuned LLM with conversational memory. The team implements session management where each user's conversation history is stored in a Databricks SQL table with columns: session_id, user_id, timestamp, role (user/assistant), message. The endpoint currently loads the full conversation history for context on every request. As conversations grow longer, inference latency increases significantly. Which optimization maintains conversation quality while reducing latency?
The sliding window + summarization pattern is the industry-standard approach for managing long conversation context. It keeps recent exchanges for immediate context (which matter most for response quality) while compressing older history into summaries, preventing context window overflow and maintaining low latency. This is how production chat systems like ChatGPT handle long conversations.
A Databricks workflow orchestrates a GenAI pipeline: Step 1 preprocesses documents, Step 2 generates embeddings using a Model Serving endpoint, Step 3 updates a vector search index, Step 4 runs evaluation metrics. Step 2 occasionally fails with 'Model endpoint temporarily unavailable' errors. The team needs the workflow to automatically retry Step 2 without re-running Step 1. Which Databricks Jobs configuration achieves this?
Task-level max_retries in Databricks Jobs retries only the specific failed task (Step 2) without re-executing upstream tasks (Step 1). The retry_on_timeout parameter specifically handles transient availability errors like model endpoint failures. Databricks preserves task outputs, so retried tasks can use results from previously completed tasks.
A development team builds a compound AI system where an LLM agent uses multiple tools: a calculator tool (Python function), a web search tool (external API), and a database query tool (SQL execution). The agent uses ReAct prompting to decide which tool to use. During testing, the agent sometimes calls the database tool with user-provided input directly in the SQL query, creating SQL injection risks. Which implementation pattern BEST mitigates this security vulnerability?
Parameterized queries are the gold-standard defense against SQL injection. By designing the tool to enforce parameter binding at the implementation level (not relying on the LLM's behavior), you create a hard security boundary. The tool description guides the agent to provide inputs in the correct format, but the code enforcement ensures even if the agent misbehaves, injection is impossible. Defense in depth: technical control > prompt instruction.
Ready for the real thing?
The full course: two full-length practice tests, video lessons for every exam domain, hands-on labs and detailed explanations.
$199.99 $66.99 with code FREETEST33 — valid through August 23.
Get my $66.99 deal →