Search topics, questions, scenarios…

Practice

Automation Scenario Lab

The situational questions that separate a scripter from an SDET. Each one is answered at two levels: solid engineer, and senior.

Most asked in senior rounds
TestNGCI/CDIntermediate

100 tests executed, 25 failed — rerun only the failed ones

A nightly regression of 100 tests finishes with 25 failures. Rerunning everything wastes 40 minutes and hides whether the failures are real.

Approach

  1. 1TestNG writes testng-failed.xml into target/surefire-reports after every run
  2. 2Add a second pipeline stage that executes only that suite file
  3. 3Tag reruns in the report so flakiness stays visible
  4. 4Only tests still failing after the rerun mark the build red

Implementation

Stage 1 runs the full suite and never fails the build. Stage 2 runs testng-failed.xml. A post step merges both reports and decides the build result.

groovyImplementation
1stage('Regression') {
2  steps { sh 'mvn test -Dsurefire.suiteXmlFiles=suites/regression.xml || true' }
3}
4stage('Rerun failed') {
5  when { expression { fileExists('target/surefire-reports/testng-failed.xml') } }
6  steps { sh 'mvn test -Dsurefire.suiteXmlFiles=target/surefire-reports/testng-failed.xml' }
7}

Interview answer

TestNG generates testng-failed.xml automatically; I run it as a second Maven execution so only the 25 failures rerun, and I keep the retry visible in the Extent report.

Senior-level answer

I treat reruns as a diagnostic, not a pass mechanism. The pipeline reruns failures once, tags them 'flaky' in the dashboard, and any test flaky twice in a week is auto-quarantined and raised as a defect against the framework — otherwise reruns quietly hide real product bugs.