AIF-C01 Question 134
Select 4A company is using a generative AI model for customer support automation. During testing, the team finds that certain inputs reveal sensitive company data, and other inputs yield harmful or inappropriate outputs. What potential risks related to prompt engineering could be causing these issues?
- A
Exposure of sensitive data due to improperly handled training data
- B
Prompt hijacking, where adversarial prompts manipulate the model's behavior
- C
Model drift caused by changes in the underlying training data over time
- D
Jailbreaking, where prompts bypass safeguards to produce harmful outputs
- E
Data poisoning, where malicious data corrupts the training process
Show answer and explanation
Correct answers: A, B, D, E
Explanation
Prompt engineering involves crafting inputs to guide AI models effectively, but it comes with risks. Exposure can occur if sensitive information is improperly managed during training. Prompt hijacking and jailbreaking involve adversarial or exploitative prompts that manipulate the model to behave in unintended or unsafe ways. Data poisoning corrupts the training process by injecting malicious data, leading to degraded or harmful outputs. These risks highlight the importance of robust safeguards and ethical considerations in AI development.
- A. Correct.
Exposure of sensitive data can occur if the model inadvertently memorizes and reveals private or confidential information from its training data.
- B. Correct.
Prompt hijacking involves crafting adversarial prompts to manipulate the model's behavior in unintended ways, which could explain unexpected or harmful outputs.
- C. Incorrect.
Model drift refers to performance degradation over time due to changing data patterns, but it is unrelated to prompt engineering risks.
- D. Correct.
Jailbreaking refers to crafting prompts that bypass safety or ethical guidelines set during the development of the AI model, leading to inappropriate outputs.
- E. Correct.
Data poisoning occurs when malicious actors insert harmful data into the training set, resulting in compromised or biased model behavior.