Machine Learning Engineer Intermediate Practice Exam: Medium Difficulty 2025

A product team wants to build an ML model to reduce customer support load by automatically answering incoming tickets. They have historical tickets with agent-written responses, but many tickets are duplicates and some categories are rare. The team must define a success metric aligned to business value and robust to class imbalance. What should they do FIRST?

A retailer trains a demand forecasting model. During evaluation, they notice strong performance overall but poor results for new stores with limited history. They want a problem framing that reduces cold-start impact while still producing daily forecasts per store-SKU. Which approach is most appropriate?

A healthcare startup needs to serve an image model for triage in multiple regions. Data residency requires that images never leave each region, but the model must be consistent across regions and updated monthly. They want a managed approach with minimal operational overhead. What architecture best meets these requirements?

A team is building a feature store for real-time fraud detection. They need low-latency online lookups during prediction and also need offline training datasets generated from the same feature definitions to reduce training-serving skew. Which approach is best on Google Cloud?

A data engineering team ingests clickstream events into BigQuery and wants to build training examples for a conversion model. Events arrive late and can be duplicated. They need a reproducible training dataset for a given time window and to avoid label leakage. What is the best design?

A team uses Vertex AI Pipelines to train a model from data in BigQuery. They must ensure the exact training dataset can be reconstructed for audits and that training code changes are traceable. Which combination best satisfies reproducibility?

You are training a model on imbalanced fraud labels (0.2% positive). The business cares about catching as many fraud cases as possible while keeping false positives within a manageable review capacity. Which modeling and evaluation approach is most appropriate?

A team trains a text model for sentiment analysis and sees excellent validation results. After deployment, performance drops significantly. Investigation shows that the training split randomly mixed comments from the same users across train and validation. What should the team change to reduce this issue?

A team wants to orchestrate a weekly training pipeline that: (1) extracts features from BigQuery, (2) trains on Vertex AI Training, (3) runs validation, and (4) conditionally deploys to an endpoint only if metrics exceed a threshold. They also need lineage tracking and repeatable runs. What is the best approach?

A recommendation model is deployed on Vertex AI. Business stakeholders report a gradual drop in click-through rate over several weeks. You suspect data drift in user behavior and want to detect it and trigger retraining while minimizing false alarms. What should you do?