Databricks Machine Learning Professional Question 266
Select 3You are deploying a machine learning model for a financial institution to detect fraudulent transactions. To ensure the model remains effective over time, you want to implement a comprehensive drift detection and mitigation solution within Databricks. Which of the following steps should you include as part of the solution?
- A
Monitor feature distributions in incoming data and compare them to training data distributions.
- B
Automatically retrain the model daily without evaluating the model's performance.
- C
Use a combination of data drift detection and concept drift detection techniques.
- D
Establish alerting mechanisms to notify stakeholders when drift thresholds are exceeded.
- E
Rely solely on historical training data to evaluate drift and avoid using new data.
Show answer and explanation
Correct answers: A, C, D
Explanation
A comprehensive drift solution should include monitoring for both data drift and concept drift, as these can lead to model performance degradation. In addition, setting up alerting mechanisms ensures that stakeholders are made aware of significant drift issues. Retraining models should be done based on evidence of performance degradation, not on a fixed schedule, and relying solely on historical data overlooks important signals from new data.
- A. Correct.
Monitoring feature distributions is an essential step in detecting data drift, which occurs when the incoming data distribution changes compared to the training data.
- B. Incorrect.
Automatically retraining the model daily without evaluating its performance is not a good practice, as it can lead to unnecessary retraining or even degradation of the model's performance.
- C. Correct.
Using both data drift (input feature changes) and concept drift (target distribution changes) detection techniques ensures a comprehensive approach to identifying potential issues.
- D. Correct.
Establishing alerting mechanisms ensures that stakeholders are promptly informed of significant drift, enabling timely action to mitigate its impact.
- E. Incorrect.
Relying solely on historical training data without considering new data is insufficient, as it ignores potential changes in the environment or data patterns that could impact model performance.