IBM A1000-108 - Assessment: Foundations of AI and Machine Learning Advanced Practice Exam: Hard Questions 2025

A retail company deploys a binary classifier to approve instant credit at checkout. After launch, approvals increase overall, but a post-deployment audit shows the false negative rate (denying qualified applicants) is significantly higher for one protected group than others. The business wants to improve fairness while keeping default risk stable and maintaining a clear decision trail for regulators. Which approach best addresses the issue with the most defensible trade-offs?

A bank builds a model to predict fraud within the next 24 hours using transaction history. During validation the AUC is excellent, but in production performance collapses and investigators discover the feature set includes "chargeback filed" which only becomes known days later. What is the most accurate diagnosis and the best corrective action?

A healthcare provider wants to use an AI model to assist triage. A model with higher overall accuracy is available, but it is less interpretable and shows unstable behavior under small input perturbations. Another model is slightly less accurate but provides stable, clinically plausible explanations and allows robust monitoring. Given safety-critical context and governance requirements, what is the best recommendation?

A team is training a multi-class image classifier and notices that validation accuracy is high, but calibration is poor: predicted probabilities are overconfident, leading to bad downstream decisions that depend on confidence thresholds. Which technique best addresses this problem without retraining the entire model from scratch?

A global manufacturer builds a predictive maintenance model using sensor data aggregated daily. They randomly split rows into train/test and get strong metrics. However, when deployed per machine, performance is inconsistent and worse on new machines. What is the most appropriate validation redesign to estimate real-world performance and why?

A customer-support chatbot uses a large language model (LLM) with retrieval-augmented generation (RAG). In production, it sometimes provides plausible but incorrect policy details. The team wants to reduce hallucinations while keeping answers concise and auditable. Which change is most effective and aligned with responsible AI best practices?

A data science team is building a churn model. The dataset includes a high-cardinality categorical feature "customer_id" and a derived feature "avg_support_calls_last_30_days" which is missing for new customers. After one-hot encoding and mean imputation, the model achieves unusually high offline performance but fails on new users and new IDs. What is the most likely root cause and best fix?

An insurer uses a model to price policies. Regulators require an explanation for each decision and evidence that protected classes are not being unfairly penalized. The model includes a feature correlated with a protected attribute (e.g., neighborhood). The business insists the feature improves risk estimation. What is the best governance-aligned path forward?

A model is deployed to predict demand weekly. After a supply-chain disruption, the model’s error increases sharply. The team suspects concept drift. Which monitoring signal most directly indicates concept drift (change in relationship between features and target), rather than just data drift (change in feature distribution)?

A team prepares training data from multiple sources (CRM, web analytics, billing) for a supervised learning model. They join tables using a monthly snapshot key. Later, they discover that some features were computed using future information relative to the label window due to late-arriving billing events. Which data management design best prevents this class of issue going forward?