Databricks Generative AI Engineer Associate Question 114
Select 3You are tasked with evaluating the responses generated by a Large Language Model (LLM) integrated into a customer support chatbot. During your qualitative assessment, you identify several issues. Which of the following issues should you prioritize to ensure the quality and safety of the responses?
- A
The responses occasionally provide incorrect factual information.
- B
The responses use language that could be interpreted as offensive or discriminatory.
- C
The responses are too concise and lack sufficient detail to answer customer queries.
- D
The responses include sensitive customer information that should remain confidential.
- E
The responses occasionally do not adhere to the company's brand tone.
Show answer and explanation
Correct answers: A, B, D
Explanation
Qualitative assessment of LLM responses must prioritize addressing issues that affect response safety (such as offensive language or privacy violations) and quality (such as factual correctness). While other issues like tone or detail can be important, they do not pose the same level of risk or impact as the prioritized concerns.
- A. Correct.
Incorrect factual information can erode user trust and lead to poor decision-making based on faulty data, making this a critical issue to address.
- B. Correct.
Offensive or discriminatory language in responses is a significant safety and ethical concern, as it can damage a company's reputation and harm users.
- C. Incorrect.
While concise responses may sometimes be less helpful, this issue is not critical when compared to safety and factual correctness.
- D. Correct.
Including sensitive customer information is a severe safety issue, as it violates privacy standards and can lead to legal and ethical consequences.
- E. Incorrect.
Although maintaining the company's brand tone is important, it is not as critical as ensuring safety, factual correctness, and the protection of sensitive information.