AI-102 Question 473
Select 3Your organization is deploying a large-scale generative AI model using Azure OpenAI Service for creating marketing content. The model is exhibiting slower-than-expected response times during inference. Which of the following actions should you take to optimize the performance of the generative AI model while maintaining cost efficiency?
- A
Increase the compute size of the Azure OpenAI resource.
- B
Enable token batching during API requests.
- C
Switch to a smaller model version if it meets the application's requirements.
- D
Utilize Azure Cognitive Services instead of Azure OpenAI Service for generative AI.
- E
Implement caching for frequently requested generations.
Show answer and explanation
Correct answers: B, C, E
Explanation
To optimize the generative AI model's performance in Azure OpenAI Service, you can take specific measures such as enabling token batching for better resource utilization, using a smaller model to reduce computational overhead, and implementing caching to avoid redundant processing. These actions ensure better performance while maintaining cost efficiency. Increasing compute size can improve performance but may not be cost-efficient, and switching to an entirely different service is not a relevant optimization for this scenario.
- A. Incorrect.
Increasing the compute size might improve performance but can significantly increase costs. While it may work in some scenarios, it is not necessarily the most cost-efficient approach.
- B. Correct.
Enabling token batching during API requests is a recommended optimization technique to reduce latency and make better use of resources during inference.
- C. Correct.
Switching to a smaller model version can reduce computational load and improve response times, provided it still meets the application's requirements.
- D. Incorrect.
Azure Cognitive Services is not a direct replacement for Azure OpenAI Service for generative AI tasks, so this option is not relevant to optimizing the existing generative AI solution.
- E. Correct.
Implementing caching for frequently requested generations can significantly reduce redundant processing, improving response times and overall performance.