AZ-400 Question 203
Select 2You manage an Azure DevOps pipeline that runs multiple stages of tests (unit, integration, and UI). Recently, your team has observed intermittent test failures, which sometimes pass on re-runs, indicating flaky tests. Moreover, the pipeline duration has been steadily increasing over the last few weeks, suggesting potential performance bottlenecks. You want to establish a comprehensive strategy to monitor your pipeline health, focusing on identifying flaky tests, tracking failure rate, and measuring overall pipeline duration. Which two actions should you take to effectively monitor and address these issues?
- A
Enable the built-in test analytics in Azure DevOps and examine pass/fail trends to diagnose potential flaky tests.
- B
Configure Azure DevOps� Flaky Test Management feature to automatically identify and categorize tests prone to intermittent failures.
- C
Reduce the pipeline runs to once per month to limit the impact of flaky tests and performance issues.
- D
Integrate a script that forcibly retries every test until it passes to minimize pipeline failures.
- E
Reject all pull requests on the first sign of test failure without any further investigation.
Show answer and explanation
Correct answers: A, B
Explanation
To effectively monitor pipeline health in Azure DevOps, it is essential to use actionable analytics and flaky test detection features. Analyzing pass/fail trends highlights issues in both code quality and pipeline performance. The Flaky Test Management feature isolates tests that fail sporadically, enabling targeted remediation. For more details, refer to Microsoft� official documentation on Azure DevOps Test Analytics and Flaky Test Management at https://learn.microsoft.com/azure/devops/pipelines/test/test-reporting.
- A. Correct.
Option 1 is correct. Azure DevOps provides built-in test analytics features, including pass/fail trend charts, which help identify patterns in test failures over time. Monitoring these trends can highlight tests that intermittently fail or tasks that contribute to increased pipeline duration.
- B. Correct.
Option 2 is correct. Azure DevOps includes a Flaky Test Management feature that automatically flags tests with inconsistent results. By tracking how often specific tests fail and then pass without changes, this feature aids in identifying and addressing flaky tests.
- C. Incorrect.
Option 3 is incorrect. Reducing the frequency of runs does not solve the underlying issue of flaky tests or increased duration. Instead, it delays gathering valuable data and complicates troubleshooting because of less frequent feedback.
- D. Incorrect.
Option 4 is incorrect. Repeatedly retrying all tests until they pass masks real issues and inflates pipeline duration. This approach fails to distinguish genuinely flaky tests from actual code defects and undermines confidence in test results.
- E. Incorrect.
Option 5 is incorrect. Automatically rejecting pull requests on the first sign of a test failure does not help you identify flakiness or address the root cause. It also risks halting development activities and frustrating contributors without clear diagnostics.