Machine Learning Engineer Advanced Practice Exam: Hard Questions 2025

A retail platform wants to reduce chargebacks by flagging potentially fraudulent orders in near real time. Fraud labels arrive 30–60 days after an order, and fraud patterns drift significantly during holiday seasons. The business requires a clear decision policy for manual review capacity, and leadership is concerned about “accuracy” being misleading due to extreme class imbalance. Which approach best frames the ML problem and success criteria?

Your team is building an LLM-powered customer-support assistant. Compliance requires that responses must be grounded only in approved internal documents, and every answer must include citations. Latency must be low and data must not leave Google Cloud. You also need to support frequent document updates without retraining the model. Which architecture best meets these requirements on Google Cloud?

A global ridesharing company trains a model using user-level trip histories. Training data is in BigQuery and includes nested/repeated fields. Data science wants feature parity between training and online serving, and governance requires that PII never reaches the feature store. Online predictions must be served in <50 ms. Which design best satisfies feature parity, governance, and latency?

You operate a batch training pipeline that ingests clickstream events from Pub/Sub, processes them with Dataflow, and writes training tables to BigQuery. Recently, model performance dropped. Investigation shows duplicate events and occasional out-of-order arrivals, causing label leakage and incorrect aggregations. You need a robust fix with minimal operational overhead. What should you do?

A healthcare ML team must train on structured EHR data stored in BigQuery. Compliance requires data minimization, lineage, and reproducibility. The team also needs to perform point-in-time correct joins between patient events and labels (to avoid using future information). Which approach best meets these constraints?

You are preparing image data for a multi-region training job. The dataset is 200 TB of small files in Cloud Storage. Training on Vertex AI custom training shows poor throughput due to file listing and per-file overhead. You need to improve I/O efficiency without changing the model. What is the best approach?

A model trained on imbalanced data achieves strong offline metrics but fails in production because the predicted probabilities are poorly calibrated, leading to unstable threshold-based decisions across regions. You need a solution that improves probability calibration and supports region-specific thresholds without retraining the base model from scratch. What should you do on Vertex AI?

You train a tabular model with strong performance offline, but in production the feature distributions differ due to changes in upstream data encoding (a new category mapping). You want to reduce training-serving skew and make preprocessing consistent and versioned. Which approach is most appropriate?

Your org runs a multi-step Vertex AI Pipeline: data extraction (BigQuery), preprocessing (Dataflow), training (custom training), evaluation, and deployment to a Vertex AI Endpoint. A recent incident deployed a model trained on an incomplete data partition due to a transient upstream failure; the pipeline still succeeded because downstream steps used cached artifacts. You need to prevent unsafe deployments while keeping caching benefits for expensive steps. What should you do?

After deploying a new model to a Vertex AI Endpoint, you observe a gradual drop in conversion rate. You suspect both data drift and a broken upstream feature. You need rapid isolation of the issue and a sustainable monitoring strategy that can trigger rollbacks. What is the best approach?