Databricks Generative AI Engineer Associate Question 252
Select 3You are developing a Generative AI application on Databricks that allows users to generate custom text responses. To ensure the application is secure against malicious user inputs, such as prompt injections or harmful requests, which guardrail techniques should you implement?
- A
Sanitize user inputs using regular expressions or predefined filtering rules
- B
Implement rate limiting to control the frequency of user requests
- C
Use a content moderation model to review generated outputs
- D
Allow users to directly access the model's token probability outputs for transparency
- E
Apply prompt augmentation techniques to include safety instructions for the model
Show answer and explanation
Correct answers: A, C, E
Explanation
To protect a Generative AI application from malicious user inputs, you should implement multiple guardrail techniques. Input sanitization (Option 1) prevents harmful inputs from reaching the model, while content moderation (Option 3) ensures that generated outputs meet safety standards. Prompt augmentation (Option 5) further guides the model to behave safely. These techniques work together to mitigate risks at different stages of the input and output pipeline. Other options, like rate limiting (Option 2), address performance but not security, and exposing token probability outputs (Option 4) may introduce new vulnerabilities.
- A. Correct.
Sanitizing user inputs helps prevent malicious prompt injections by removing harmful or unexpected content before it reaches the model. This is a key guardrail technique.
- B. Incorrect.
Rate limiting helps control traffic but does not directly address malicious inputs or prevent harmful content generation. It focuses on performance, not security or safety.
- C. Correct.
Using a content moderation model ensures that generated outputs are reviewed for harmful or inappropriate content, acting as a post-generation safety measure.
- D. Incorrect.
Allowing users to directly access the model's token probability outputs can expose sensitive information and increase the risk of exploitation. This is not a recommended guardrail technique.
- E. Correct.
Prompt augmentation, such as including safety instructions or constraints in the input prompt, helps guide the model to generate safer outputs and reduces the risk of malicious input exploitation.