Databricks Generative AI Engineer Associate Question 4
Select 3You are tasked with designing a generative AI application that uses a fine-tuned large language model (LLM) to assist customer service agents by generating real-time responses. The application will be deployed on Databricks. Which considerations are most critical to ensuring the design is scalable, efficient, and secure?
- A
Use a feature store to manage and serve the fine-tuned model's embeddings for faster inference.
- B
Implement token-level filtering to prevent sensitive information from being included in generated responses.
- C
Design the application to use Databricks' MLflow integration for experiment tracking and model versioning.
- D
Optimize the application for batch inference rather than real-time inference to reduce infrastructure costs.
- E
Ensure the Databricks workspace is configured with role-based access control (RBAC) to limit unauthorized access to the model and data.
Show answer and explanation
Correct answers: B, C, E
Explanation
When designing applications on Databricks, considerations for scalability, efficiency, and security are crucial. For this scenario, token-level filtering addresses privacy concerns, MLflow integration ensures robust model management, and RBAC enhances security. While cost optimization is important, batch inference is not suitable for real-time applications, and feature stores are unrelated to serving LLM embeddings.
- A. Incorrect.
Incorrect: A feature store is used to manage and serve features for training and inference, not embeddings of a fine-tuned model. This is not directly relevant to the design of a scalable and secure LLM-based application.
- B. Correct.
Correct: Token-level filtering is critical for ensuring the application adheres to data security and privacy requirements by preventing sensitive information from being included in responses.
- C. Correct.
Correct: MLflow integration is essential for managing the lifecycle of machine learning models, including tracking experiments and maintaining model versions, which is key to a reliable and scalable application.
- D. Incorrect.
Incorrect: Optimizing for batch inference would reduce costs but is unsuitable for a real-time application where low-latency responses are critical.
- E. Correct.
Correct: Configuring role-based access control (RBAC) in the Databricks workspace ensures that only authorized users can access sensitive resources like the model and data, enhancing security.