Databricks Generative AI Engineer Associate Question 169
Single answerYou have trained three different generative AI models for text summarization on a dataset. The evaluation metrics for the models are as follows:
- Model A: ROUGE-L = 0.85, BLEU = 0.65, Latency = 1.2 seconds
- Model B: ROUGE-L = 0.88, BLEU = 0.70, Latency = 2.5 seconds
- Model C: ROUGE-L = 0.82, BLEU = 0.67, Latency = 1.0 seconds
Your task is to deploy the best model for use in a real-time summarization application, where both high-quality summaries and low latency are critical. Which model would you select?
- A
Model A
- B
Model B
- C
Model C
- D
None of the models meet the requirements
Show answer and explanation
Correct answer: A
Explanation
For a real-time summarization application, both high-quality output and low latency are essential. While Model B has the highest quality metrics, its high latency disqualifies it for real-time use. Model C has the lowest latency but lower quality metrics than Model A. Model A offers the best balance of quality and latency, making it the optimal choice.
- A. Correct.
Model A has a good balance of high-quality metrics (ROUGE-L = 0.85, BLEU = 0.65) and low latency (1.2 seconds), making it suitable for real-time applications.
- B. Incorrect.
Model B has the highest ROUGE-L (0.88) and BLEU (0.70), but its high latency (2.5 seconds) makes it unsuitable for real-time applications where low latency is critical.
- C. Incorrect.
Model C has the lowest latency (1.0 seconds), but its ROUGE-L (0.82) and BLEU (0.67) are lower than Model A, making it less suitable in terms of quality.
- D. Incorrect.
This option is incorrect because at least one model (Model A) meets the requirements for both quality and latency.