Databricks Generative AI Engineer Associate Question 286
Select 4You are deploying a Large Language Model (LLM) to serve customer support queries in real-time. The deployment should ensure high-quality responses while maintaining low latency and efficient resource utilization. Which key metrics should you monitor to ensure the deployment meets its objectives?
- A
Latency of responses
- B
Model training loss
- C
Token utilization per query
- D
User satisfaction score
- E
CPU and GPU usage
Show answer and explanation
Correct answers: A, C, D, E
Explanation
In a real-time LLM deployment for customer support, key metrics include latency (to ensure quick responses), token utilization (to track efficiency), user satisfaction score (to measure response quality), and resource usage (to monitor cost and performance). Model training loss, however, is not relevant in the deployment phase as it pertains to the training process.
- A. Correct.
Monitoring latency of responses is crucial for ensuring the model provides real-time answers, which is essential for customer support scenarios.
- B. Incorrect.
Model training loss is not relevant during the deployment phase, as it is a metric used during the training process, not for monitoring live performance.
- C. Correct.
Token utilization per query helps track the efficiency of the model in generating responses, which can impact both latency and cost.
- D. Correct.
User satisfaction score is an important metric to assess the quality of responses and overall customer experience in this scenario.
- E. Correct.
CPU and GPU usage should be monitored to ensure resource utilization is efficient and does not lead to system bottlenecks or unnecessary costs.