Databricks Generative AI Engineer Associate Question 165
Single answerYou are tasked with selecting a pre-trained model from a model hub for summarizing long documents. The requirements for the model include: (1) capability to handle long input sequences (e.g., up to 4096 tokens), (2) a high ROUGE score on summarization tasks, and (3) a permissive license for commercial use. Which model should you choose based on the following metadata/model cards?
- A
Model A: Input token limit 1024, ROUGE-1 score 40.1, licensed under Apache 2.0.
- B
Model B: Input token limit 4096, ROUGE-1 score 45.8, licensed under MIT.
- C
Model C: Input token limit 2048, ROUGE-1 score 50.2, licensed under Creative Commons Non-Commercial 4.0.
- D
Model D: Input token limit 4096, ROUGE-1 score 43.5, licensed under Apache 2.0.
Show answer and explanation
Correct answer: B
Explanation
Model B is the best choice as it meets all the stated requirements: it can handle long input sequences (4096 tokens), has a high ROUGE-1 score (45.8), and uses the MIT license, which permits commercial use. Other models either fail in token limit, licensing, or have lower evaluation scores.
- A. Incorrect.
Model A does not meet the requirement of handling long input sequences, as its token limit is only 1024.
- B. Correct.
Model B meets all the requirements: it supports input sequences up to 4096 tokens, has a high ROUGE-1 score (45.8), and is licensed under MIT, which is permissive for commercial use.
- C. Incorrect.
Model C has a high ROUGE-1 score (50.2) but is licensed under Creative Commons Non-Commercial 4.0, which does not allow commercial use, violating the requirements.
- D. Incorrect.
Model D supports input sequences up to 4096 tokens and has a permissive license (Apache 2.0), but its ROUGE-1 score (43.5) is lower than Model B, making it a less optimal choice.