AZ-400 Question 158
Select 2You are the lead DevOps engineer for an e-commerce platform that is introducing a new recommendation algorithm. The marketing team wants to measure how this algorithm affects user engagement by exposing it to a small subset of real users first. They also need the ability to revert quickly if user satisfaction or performance metrics degrade. Which two deployment strategies should you implement to achieve this goal?
- A
Use blue-green deployment to maintain two identical production environments and then switch all traffic to the new environment at once.
- B
Implement a canary release that gradually increments traffic to the new recommendation engine while monitoring key performance indicators.
- C
Perform a ring-based deployment by rolling out the new functionality to internal test rings before going global.
- D
Adopt A/B testing to compare user behavior metrics between the old and new recommendation algorithm on partial user segments.
- E
Use feature flags to toggle the new recommendation algorithm for selected user groups and revert quickly if issues arise.
Show answer and explanation
Correct answers: B, D
Explanation
Combining a canary release with A/B testing satisfies both the need to limit initial exposure of the new recommendation algorithm and to compare user behavior metrics. Canary releases help you introduce changes to a small percentage of traffic and scale up if results are good, while A/B testing provides a robust framework for measuring the differences in user engagement and performance metrics between the old and new versions. For more details, refer to Microsoft's documentation on deployment strategies in Azure DevOps (https://docs.microsoft.com/azure/devops).
- A. Incorrect.
Blue-green deployment is ideal for quick switches between two identical environments but does not inherently support gradually exposing a small subset of users to the new algorithm for comparative metrics. It is an 'all-or-nothing' strategy once you flip the traffic to the new environment.
- B. Correct.
Canary releases allow you to route a small percentage of production traffic to the new version initially. As you gather performance data and user feedback, you can gradually increase traffic if metrics remain positive, or roll back quickly if issues arise. This aligns well with the requirement for incremental exposure and simple reversion.
- C. Incorrect.
Ring-based deployment typically phases releases via internal or restricted user groups (e.g., alpha, beta, canary, production rings). While it can be effective, it� not as focused on directly comparing user metrics in a controlled A/B scenario. It� more about incrementally releasing to progressively larger user rings, rather than direct metric comparisons between old and new versions at the same time.
- D. Correct.
A/B testing is designed to compare key metrics (such as click-through rates or conversion rates) between two versions of a service. By directing a subset of user traffic to the new algorithm and comparing results to the old algorithm, you gather precise data about improvements or regressions. This meets the need to measure user satisfaction and engagement metrics in real time.
- E. Incorrect.
Feature flags are powerful for toggling features on or off for specific user groups, but by themselves, they do not provide the same structured user metric comparison as A/B testing. While they enable quick reversion, the question specifically requires a controlled and measurable experiment to compare user engagement, making canary release plus A/B testing more appropriate in this scenario.