Model Development
Free Databricks Certified Machine Learning Associate practice — 6 questions on Model Development, with explanations. No sign-up. Full 12-question mixed test →
A data scientist tunes a gradient boosting model using Hyperopt's fmin with SparkTrials on a 32-worker Databricks cluster. They set max_evals=100 and parallelism=32 to fully utilize the cluster. After tuning, the resulting model performs worse than a previous run that used parallelism=8. What is the MOST likely explanation?
TPE (the default Hyperopt algorithm) is adaptive — it uses results from completed trials to choose better points to try next. When parallelism is close to max_evals, most trials launch before any results come back, so the algorithm can't learn from prior trials and effectively degrades to random search. Databricks documentation recommends setting parallelism no higher than roughly sqrt(max_evals) to preserve adaptive optimization quality.
A team is building a binary classifier to detect fraudulent transactions where only 2% of transactions are fraudulent. When comparing MLflow runs to select the best-performing model, which metric should be prioritized?
AUC-PR focuses on the model's ability to correctly identify the minority (positive) class and is far more informative than ROC-based metrics when classes are severely imbalanced, since it isn't inflated by the large number of true negatives.
A dataset of 5 GB fits comfortably into the memory of the driver node on a Databricks cluster with 64 GB of RAM. The team needs to train a random forest classifier and wants the fastest development iteration with full access to scikit-learn's hyperparameter options. Which training approach is BEST suited for this scenario?
When a dataset fits comfortably in the memory of a single node, using scikit-learn directly is simpler, faster to iterate on, and gives full access to scikit-learn's API and hyperparameters — the overhead of a distributed framework is unnecessary and would slow down development for data this size.
After running Hyperopt's fmin with SparkTrials with MLflow autologging enabled, a data scientist wants to programmatically retrieve the single best child run by validation F1 score using the MLflow Python API. Which approach correctly retrieves it?
With MLflow autologging enabled, each Hyperopt trial launched by SparkTrials is automatically logged as a nested child run under the parent fmin run. Calling search_runs against the experiment and ordering by the desired metric column correctly retrieves the best individual trial run.
Two regression models are compared for predicting house prices. Model A has a lower RMSE than Model B, but Model B has a lower MAE than Model A. Which statement correctly explains this discrepancy?
Because RMSE squares each error before averaging, it disproportionately penalizes large errors/outliers. A model that avoids a few large mistakes (Model A) can therefore have a lower RMSE, while a model that is more consistently accurate on typical cases but occasionally has a larger miss can have a lower MAE (Model B). The two metrics weight error magnitudes differently and can disagree on ranking.
A data scientist defines a Hyperopt search space for the max_depth hyperparameter of a gradient boosted tree model, which must be an integer between 3 and 15. Which search space definition is CORRECT?
hp.quniform samples a continuous value over the given range and rounds it to the nearest multiple of the quantization step (q=1 here), producing integer-valued samples appropriate for an integer hyperparameter like max_depth (the value should be cast to int in the objective function).
Ready for the real thing?
The full course has two full-length practice tests, video lessons for every exam domain, hands-on labs and detailed answer explanations.
Start my full course on Udemy →