Databricks Generative AI Engineer Associate Question 282
Select 3You are deploying a large language model (LLM) to power a customer support chatbot. The chatbot is expected to handle a high volume of queries with responsive and accurate answers. Which key metrics should you monitor to ensure the deployment meets performance and reliability expectations?
- A
Latency: The time taken for the model to generate a response.
- B
Token Usage: The number of tokens used in each response.
- C
Model Accuracy: The percentage of responses that are factually correct and relevant.
- D
User Engagement Rate: The number of users interacting with the chatbot over a given period.
- E
GPU Utilization: The percentage of GPU resources being used during inference.
- F
Response Diversity: The variety of unique responses generated for similar queries.
Show answer and explanation
Correct answers: A, B, C
Explanation
Latency, token usage, and model accuracy are the key metrics in this scenario as they directly influence the chatbot's performance, cost-efficiency, and reliability. While other metrics like user engagement or GPU utilization provide additional insights, they are not specific to the core requirements of deploying an LLM for customer support.
- A. Correct.
Latency is critical for a customer support chatbot as users expect quick responses. High latency can degrade user experience.
- B. Correct.
Monitoring token usage is important to control costs, as LLMs often charge based on the number of tokens processed in input and output.
- C. Correct.
Model accuracy is essential to ensure the chatbot provides correct and helpful responses, which directly impacts user satisfaction.
- D. Incorrect.
User engagement rate is a useful business metric but is not directly tied to monitoring the LLM's performance or reliability.
- E. Incorrect.
GPU utilization is an infrastructure metric but does not directly measure the effectiveness of the LLM deployment in this scenario.
- F. Incorrect.
Response diversity may be relevant in creative applications but is less critical in a customer support context where consistent and accurate responses are prioritized.