ML Solution Monitoring, Maintenance, and Security
Free AWS Certified Machine Learning Engineer - Associate practice — 6 questions on ML Solution Monitoring, Maintenance, and Security, with explanations. No sign-up. Full 12-question mixed test →
A fraud-detection endpoint on SageMaker receives labeled ground truth data from a downstream system with an average delay of 21 days after inference. The MLOps team wants automated alerts when the model's precision degrades using SageMaker Model Monitor. The team configures a Model Quality monitor with a daily monitoring schedule, but every execution fails with 'insufficient merged records.' What is the MOST likely cause and correct fix?
SageMaker Model Quality Monitor merges captured inference data with ground truth labels within a specified analysis window; when labels arrive later than the schedule's lookback period, no records merge successfully. The fix is to configure the monitor's ground truth input S3 location and schedule such that the merge window (e.g., 24-25 days) covers the actual label delay.
A team enables SageMaker Clarify to detect feature attribution drift on a live endpoint. They capture inference input/output via Data Capture and configure a Model Bias/Explainability monitoring schedule referencing a baseline. Six weeks in, alerts fire showing high NDCG divergence between live SHAP values and the baseline, even though overall prediction accuracy has not changed. What does this indicate, and what should the team do?
SageMaker Clarify's feature attribution drift monitor compares SHAP-value-based feature importance rankings (using NDCG) between a baseline and live traffic. A significant shift means the model is relying on different features than during training/baseline, which is a leading indicator of concept drift even if accuracy metrics haven't yet degraded -- exactly the scenario the exam tests for proactive monitoring.
A security team requires that all data at rest in SageMaker Studio's attached EFS volume, as well as all SageMaker endpoint storage volumes, be encrypted with a customer-managed KMS key (not the AWS managed key). The Studio execution role currently has kms:Decrypt permission only. After configuring the KMS key ID in the Studio domain settings, users cannot launch notebooks and see an access denied error referencing the KMS key. What is the MOST likely missing permission?
When SageMaker provisions encrypted EFS or EBS volumes using a customer-managed key, it needs to call kms:CreateGrant (to delegate temporary permissions to the AWS service) and kms:GenerateDataKey (to create the data encryption key), in addition to kms:Decrypt for ongoing access. Missing CreateGrant is the classic cause of this specific failure.
A company hosts an ML inference workload with highly sporadic traffic -- some models receive zero requests for hours, then a burst of dozens of requests, and cold-start latency of a few seconds is acceptable. They currently run always-on real-time endpoints with multiple instances per model, resulting in high idle-capacity costs. Which change BEST eliminates idle capacity cost while meeting the workload pattern?
SageMaker Serverless Inference is purpose-built for intermittent, unpredictable traffic: it scales down to zero when there are no requests and you pay only for the compute used during invocation processing, directly eliminating idle-capacity cost while tolerating a few seconds of cold-start latency -- matching the stated requirements exactly.
A financial services company runs SageMaker training jobs that must never traverse the public internet, even to reach Amazon S3 for input/output data, per a compliance mandate. The training jobs run inside a customer VPC with private subnets and no NAT gateway. Which combination of configurations satisfies this requirement?
To keep all traffic within the AWS network, the training job must run in VPC-attached private subnets, use an S3 Gateway VPC Endpoint so S3 traffic never traverses the internet, and use Interface VPC Endpoints (AWS PrivateLink) for the SageMaker API/runtime so control-plane and data-plane calls also stay off the public internet -- this is the standard fully private SageMaker networking pattern tested on the exam.
A team is rolling out a new fraud-detection model version. They want to validate the new model's predictions against real production traffic without exposing customers to any risk if the new model performs poorly, since it must have zero impact on live decisions during evaluation. Which deployment approach should they use?
Shadow deployment (invoking the candidate model in parallel with production traffic, logging its predictions for offline comparison, but never returning its output to end users) is the only approach with zero customer-facing risk, since production traffic and responses are entirely served by the existing validated model.
Ready for the real thing?
The full course: two full-length practice tests, video lessons for every exam domain, hands-on labs and detailed explanations.
$109.99 $34.99 with code FREETEST33 — valid through September 7.
Get my $34.99 deal →