AZ-400 Question 202
Select 3Your team has noticed an increase in pipeline failures in Azure DevOps, and some tests that fail sporadically pass on rerun, indicating possible flaky tests. You want to monitor overall pipeline health by tracking failure rate, average duration, and identifying flaky tests so you can take appropriate remediation steps. Which of the following actions or features should you implement to effectively address these monitoring needs?
- A
Set up Azure DevOps Dashboards to track build success/failure rates, average duration, and test results across multiple runs in near real time.
- B
Enable parallelization for all test runs and ignore inconsistently failing tests unless they block a release.
- C
Implement a process to rerun failed tests automatically and flag them as flaky if they succeed on rerun, recording this status in the pipeline summary.
- D
Disable all tests that have failed more than once in the last week to reduce pipeline run duration until the issue is identified.
- E
Use the built-in Test Flakiness analytics in Azure DevOps to track how often individual tests fail or succeed across multiple pipeline runs.
Show answer and explanation
Correct answers: A, C, E
Explanation
Monitoring pipeline health effectively involves observing build success/failure rates, tracking average duration, and detecting flaky tests for targeted remediation. Azure DevOps Dashboards, test rerun mechanisms, and built-in flaky test analytics help teams gain visibility into pipeline quality and stability. For more information, refer to Microsoft� documentation on flaky test management in Azure DevOps.
- A. Correct.
Option 1 is correct. Azure DevOps Dashboards can aggregate build and test metrics (including success/failure rates and durations) for each pipeline run, allowing you to monitor trends and quickly spot issues.
- B. Incorrect.
Option 2 is incorrect. Simply enabling parallelization without addressing flaky tests could mask underlying issues. Parallel tests may finish faster, but ignoring inconsistent failures likely leads to undiscovered or unresolved test problems.
- C. Correct.
Option 3 is correct. Automatically rerunning failed tests is a common method to identify flakes. If a test passes on a second attempt, it may be flaky. This process logs additional context, making it easier to diagnose and fix.
- D. Incorrect.
Option 4 is incorrect. Disabling suspect tests outright reduces coverage. While it might speed up builds, it does not provide insight into underlying failures and can introduce risk by ignoring potential regressions.
- E. Correct.
Option 5 is correct. Azure DevOps offers test analytics that can identify flaky tests by tracking pass/fail sequences over time. Leveraging these insights helps pinpoint problematic tests more quickly.