AWS Certified Machine Learning - Specialty Advanced Practice Exam: Hard Questions 2025

A fintech company ingests 3 TB/day of clickstream events into Amazon S3 (data lake). Data is partitioned by dt/hour and stored as JSON. Data scientists query the latest 7 days using Amazon Athena and build features for a SageMaker training pipeline. Queries are slow and costs are high. The company also needs schema evolution support and the ability to perform time-window joins efficiently. Which approach provides the BEST performance and maintainability with minimal operational overhead?

A media company trains a recommendation model daily in SageMaker using features built from user events stored in S3. They notice that yesterday’s training set differs when rebuilt today from the same date range, causing reproducibility issues and A/B test inconsistencies. Events can arrive late by up to 48 hours, and the feature pipeline currently reads directly from the raw S3 prefix for the date range. Which design MOST effectively ensures reproducible training datasets while still incorporating late-arriving data?

A team builds a churn model in SageMaker. During EDA they find a strong lift from a feature called "days_since_last_login" computed from login timestamps. After deployment, production performance degrades sharply. Investigation shows that in training, the feature was computed using the full dataset (including future logins after the prediction timestamp) due to a windowing bug in a Spark job. Which remediation BOTH fixes the root cause and reduces the likelihood of similar leakage in the future?

A healthcare NLP team trains a multi-class text classifier. The dataset is highly imbalanced: one class is 0.2% of samples but is clinically critical. They report 98% accuracy and a strong macro F1 on a random split. In production, the rare class recall is near zero. The dataset contains multiple notes per patient, and notes from the same patient appear across train and test. Which change MOST directly addresses the evaluation flaw that likely caused the mismatch?

A retailer trains a regression model for demand forecasting using SageMaker. EDA shows heavy-tailed errors and frequent extreme demand spikes during promotions. The team currently optimizes RMSE and observes the model underpredicts spikes, causing stockouts. The business wants to penalize underprediction much more than overprediction during promotions. Which approach BEST aligns training with the business objective while remaining robust to outliers?

A company is building an item-to-item similarity service. There are 50 million items and 500 million interaction events. The goal is to retrieve top-K similar items with sub-50 ms latency. They can accept approximate results, but the system must support daily incremental updates. Which architecture is MOST appropriate on AWS?

A bank trains an XGBoost model for credit risk. Regulators require explanations for adverse actions and evidence that the model is not using prohibited attributes. The training data includes a ZIP code feature that is highly predictive but may be a proxy for protected classes. The bank needs local explanations for individual decisions and ongoing monitoring for feature drift and bias. Which solution BEST meets these requirements using AWS-native capabilities?

A team trains a large model in SageMaker using distributed training. Training frequently fails after several hours due to transient network interruptions, and the team loses all progress. They want to minimize wasted compute while ensuring that partially completed training can resume. The training script uses a standard deep learning framework. Which change is MOST effective?

A company deploys a SageMaker endpoint for real-time fraud detection. After a new model rollout, p99 latency increases and intermittent 5xx errors appear only under traffic spikes. The model container logs show timeouts when downloading artifacts at startup and occasional CPU saturation. The deployment uses a single production variant and shifts 100% traffic at once. Which deployment strategy MOST reduces customer impact while enabling safe rollout and fast rollback?

A regulated enterprise builds ML pipelines using SageMaker Pipelines and stores datasets in S3. Auditors require end-to-end lineage: which raw data, code, container image, and hyperparameters produced a specific model artifact currently deployed. The team also needs to ensure models are only promoted if evaluation metrics meet thresholds, and all artifacts are immutable and traceable. Which approach BEST satisfies these requirements?