AIF-C01 Question 133
Select 4A company is developing a chatbot using a Large Language Model (LLM) and has chosen to utilize prompt engineering to enhance the quality of responses. However, the team is concerned about potential risks associated with this approach. Which of the following are valid risks of prompt engineering that the team should be aware of?
- A
Exposure of sensitive information due to poorly crafted prompts
- B
Model 'poisoning,' where malicious data alters the model's predictions
- C
Improved accuracy of responses through iterative prompt refinement
- D
Prompt 'jailbreaking,' which leads to bypassing ethical or safety constraints
- E
Model hijacking, where prompts are manipulated to generate harmful outputs
Show answer and explanation
Correct answers: A, B, D, E
Explanation
Prompt engineering is a powerful tool for optimizing the performance of LLMs, but it also carries risks such as the exposure of sensitive data, model poisoning, jailbreaking, and hijacking. These risks can compromise the security, safety, and ethical behavior of the AI system. Understanding these limitations is critical for mitigating potential vulnerabilities when designing AI applications.
- A. Correct.
Sensitive information can be exposed if prompts unintentionally reveal proprietary or personal data, particularly if the model's responses are shared publicly.
- B. Correct.
Prompt engineering can inadvertently allow an attacker to 'poison' the model by crafting inputs that influence the model's behavior, leading to inaccurate or biased outputs.
- C. Incorrect.
This option describes a benefit of prompt engineering and not a risk or limitation, so it is incorrect.
- D. Correct.
Jailbreaking occurs when prompts are designed to bypass safety mechanisms, potentially leading the model to generate unethical or unsafe outputs.
- E. Correct.
Model hijacking refers to using carefully designed prompts to manipulate the model into producing harmful or misleading outputs, which is a valid risk of prompt engineering.