Databricks Generative AI Engineer Associate Question 112
Select 2You are evaluating the responses from a generative AI model deployed on Databricks for a customer-facing chatbot. During your qualitative assessment, which of the following issues should you primarily look for to ensure both quality and safety of the responses?
- A
The response includes accurate and factually correct information.
- B
The response contains harmful or offensive language.
- C
The response is overly lengthy and verbose without adding value.
- D
The response provides sensitive customer information that should have been anonymized.
- E
The response is grammatically incorrect but conveys the right intent.
Show answer and explanation
Correct answers: B, D
Explanation
When qualitatively assessing responses from a generative AI model, safety issues like harmful language and exposure of sensitive information must be prioritized. These directly impact the trustworthiness and ethical compliance of the AI system. Other factors like factual accuracy, verbosity, or grammatical errors are secondary concerns for this specific evaluation focus.
- A. Incorrect.
While factual accuracy is important, it is not directly related to identifying common issues of quality and safety during a qualitative assessment. This is more about content correctness.
- B. Correct.
Harmful or offensive language is a safety issue and must be identified as a critical problem during response evaluation.
- C. Incorrect.
Overly lengthy responses could affect the user experience but are not directly tied to core quality and safety concerns.
- D. Correct.
Providing sensitive customer information is a significant safety issue as it violates privacy guidelines and data protection rules.
- E. Incorrect.
Grammatical errors may affect the perceived quality, but they are not critical safety or core quality issues if the intent remains clear.