IBM A1000-080: Assessment: Data Science and AI Advanced Practice Exam: Hard Questions 2025

A data science team is experiencing severe data leakage in their credit risk model, resulting in unrealistically high validation scores (AUC = 0.98) that drop dramatically in production (AUC = 0.72). They use time-series financial data with customer transactions, and their pipeline includes feature engineering with rolling statistics, followed by train-test split, then standardization. What is the MOST likely cause of this leakage?

An ML engineer is deploying a gradient boosting model for real-time fraud detection with strict latency requirements (<50ms p99). The model has 500 trees with depth 8. In production, they observe that 95% of predictions are legitimate transactions. Which optimization strategy would provide the BEST latency improvement while maintaining model accuracy?

A deep learning team is training a transformer model for document classification on IBM Watson Machine Learning. They observe that training loss decreases steadily but validation loss remains flat after epoch 3, while training accuracy is 95% and validation accuracy plateaus at 68%. The learning rate schedule uses warmup followed by cosine decay. What combination of interventions would MOST effectively address this issue?

A financial services company is building a customer churn prediction model using highly imbalanced data (3% churn rate). After training with class_weight='balanced' in scikit-learn, they achieve 94% accuracy but their business stakeholders report the model is 'useless' in production. Upon investigation, the model predicts 'no churn' for 98% of customers. What is the ROOT CAUSE of this failure and the appropriate solution?

A data scientist is implementing a convolutional neural network for medical image analysis in Watson Studio. During training, they observe that validation loss oscillates wildly with magnitude ±0.5 around an increasing trend, while training loss decreases smoothly. The batch size is 8 due to GPU memory constraints, and they're using batch normalization after each conv layer. What is the MOST likely explanation and solution?

An organization is architecting an end-to-end ML pipeline in IBM Cloud Pak for Data that processes streaming IoT sensor data for predictive maintenance. They need to retrain models weekly, perform real-time inference (<100ms), handle concept drift, and maintain model lineage for compliance. Which architectural approach BEST satisfies these requirements?

A data science team is performing feature selection on a dataset with 500 features and 10,000 samples for a regression task. They first apply univariate feature selection using correlation with the target, selecting the top 50 features, then train a Lasso regression model which zeros out 30 more features. In production, the model performs poorly on a specific customer segment that was well-represented in training data. What statistical issue MOST likely caused this failure?

An NLP team is fine-tuning a pre-trained BERT model for multi-label classification of customer support tickets into 45 categories. After training, they observe that the model achieves 82% micro-F1 but only 31% macro-F1 on the validation set. The category distribution follows a power law with the top 5 categories representing 60% of samples. What strategy would MOST effectively improve macro-F1 while maintaining micro-F1?

A company is deploying an ensemble of ML models in Watson Machine Learning for fraud detection, combining XGBoost, neural network, and logistic regression predictions. They're evaluating three ensemble strategies: simple averaging, weighted averaging with weights [0.5, 0.3, 0.2], and stacking with a meta-learner. The base models have validation AUCs of 0.89, 0.87, and 0.82 respectively. During cross-validation, stacking achieves 0.92 AUC while weighted averaging achieves 0.91. For production deployment with concept drift concerns, which approach should they choose and why?

A data science team is implementing bias mitigation for a loan approval model in Watson OpenScale. They discover their model has demographic parity difference of 0.18 (threshold: 0.10) and equalized odds difference of 0.08 (threshold: 0.10) between two demographic groups, while maintaining strong predictive performance (AUC: 0.88). The business requires maintaining model accuracy for risk management. What is the MOST appropriate bias mitigation strategy considering the fairness metrics and business constraints?