Databricks Generative AI Engineer Associate Question 272
Select 3You are tasked with deploying a generative AI model into production on Databricks. To ensure the model performs reliably, you must implement an evaluation and monitoring strategy. Which of the following steps should you include in your monitoring pipeline?
- A
Set up drift detection to identify changes in input data distribution over time.
- B
Design a feedback loop to collect user interactions for retraining purposes.
- C
Disable logging to minimize storage costs for monitoring data.
- D
Track latency and resource utilization metrics during model inference.
- E
Evaluate the model only once during initial deployment to confirm its performance.
Show answer and explanation
Correct answers: A, B, D
Explanation
Monitoring a generative AI model in production involves tracking key performance indicators, detecting drift, and ensuring that the model remains performant under real-world conditions. A robust evaluation and monitoring strategy allows early detection of potential issues and enables iterative improvements through feedback loops.
- A. Correct.
Correct: Drift detection is essential for identifying changes in the input data that may degrade the model's performance over time.
- B. Correct.
Correct: A feedback loop helps collect real-world data, which can be used for model retraining and improving the model's performance.
- C. Incorrect.
Incorrect: Disabling logging removes critical visibility into the model's behavior and would hinder effective monitoring.
- D. Correct.
Correct: Tracking latency and resource utilization ensures the model performs efficiently and meets production requirements.
- E. Incorrect.
Incorrect: Continuous evaluation and monitoring are necessary to maintain model reliability, as performance can degrade over time.