Selenium test reporting

Selenium has no report format, because Selenium is not a test runner — your runner writes the report and Selenium just drives the browser. Qualflare reads whichever format that turns out to be and turns it into hosted, historical reporting: AI clustering of failures by root cause, flaky scores from run history, and — where your reporter provides it — flakiness attributed to the browser or grid node it actually happens on.

Selenium doesn’t produce a report

This is the thing worth getting straight before anything else, because it dissolves most “Selenium reporting” confusion. Selenium provides the WebDriver bindings that let code drive a browser. It has no test runner, no assertion library, and no reporter. Everything about how your tests are structured, executed and recorded comes from the framework wrapped around it:

• Java — JUnit or TestNG. Surefire writes JUnit-format XML to target/surefire-reports/. This is the most common Selenium setup by a wide margin.
• Python — pytest. --junitxml=report.xml, or pytest's own richer output.
• JavaScript — Mocha or Jest. Whatever reporter you configured.
• C# — NUnit or MSTest. TRX, or a JUnit-format converter.

One consequence is worth naming: switching runners throws away your test history, because test identity is derived from the runner's naming, not from Selenium. A team moving from JUnit to TestNG keeps every line of test code and still starts from zero on flake data. Worth knowing before the migration, not after.

Send Selenium results to Qualflare

So in practice you upload your runner's output, with the matching format:

# Most Selenium suites: upload whatever your RUNNER wrote
qf my-project collect target/surefire-reports/ --format junit     # JUnit / TestNG
qf my-project collect report.xml --format pytest                  # pytest
qf my-project collect results.json --format mocha                 # Mocha

There is also a dedicated --format selenium, for reporters that emit the generic WebDriver JSON shape rather than a runner-native format:

# Or the generic WebDriver JSON, if your reporter emits it
qf my-project collect selenium-results.json --format selenium

In CI that's one added step. Authenticate the CLI once with your Qualflare access token, stored as a CI secret — see the CLI docs.

# .github/workflows/tests.yml
- name: Run Selenium suite
  run: mvn -B test

- name: Upload results to Qualflare
  if: always() # upload even when tests fail — that's the point
  run: qf my-project collect target/surefire-reports/ --format junit

The if: always() line is the one people leave out. Without it the upload step is skipped whenever the test step fails, so only green runs ever reach the platform and failure history — the entire point — stays empty.

The field that makes Selenium flakiness tractable

The generic WebDriver JSON carries something most runner formats do not: the browser the run happened in.

{
  "browser": "chrome",          // ← the fields that make Selenium different
  "platform": "linux",
  "version": "129.0",
  "total": 412, "passed": 398, "failed": 11, "skipped": 3,
  "suites": [
    {
      "name": "CheckoutTest",
      "className": "com.example.CheckoutTest",
      "tests": [
        { "name": "expired card is rejected",
          "className": "com.example.CheckoutTest",
          "methodName": "expiredCardIsRejected",
          "status": "failed" }
      ]
    }
  ]
}

That context changes what a flake rate means. A test failing 30% of the time reads as “unreliable, needs a rewrite”. The same test failing 90% of the time on Firefox and never on Chrome is not unreliable — it is a browser-specific bug, and rewriting the test is the wrong fix. The same logic applies to grid nodes: a suite that only flakes on one worker is usually telling you about that worker, not about the tests. JUnit XML has no vocabulary for any of this, which is why a Selenium team that can emit the richer shape should.

Statuses map as follows, and the last line is the important one:

passed    → passed
failed    → failed          (an assertion did not hold)
error     → error           (the test blew up)
broken    → error
timeout   → timeout         (kept distinct — not folded into error)
aborted   → aborted
<anything else> → error     (never "passed")

An unrecognised status becomes an error, never a pass. That is a deliberate correction: the parser previously defaulted unknown statuses to passed, so a broken test could upload as green. Timeouts and aborts are also kept distinct rather than folded into a generic error, because reporters emit that difference and a timeout and an exception call for different investigations.

What you get on top of your runner

• History across runs. Whatever your runner writes is per-run and local; the analysis that matters needs many runs.
• Per-browser flake attribution. Where the reporter provides browser and platform, a flake can be pinned to where it actually happens.
• AI failure clustering. When one downed environment takes out 60 UI tests, they group into one root cause instead of 60 stack traces — the single biggest time saver on a large Selenium suite.
• Flaky scoring from history. Selenium suites are the archetypal flaky suite; scoring each test from its pass/fail record is what separates “fix this” from “ignore this”.
• Grid and sharded jobs merged. One launch per run, not one per worker.
• History, trends & defects. Pass rate, slowest tests, flakiness over time across branches.

Runner output vs Qualflare

Complementary: keep your runner's report for the local loop, add Qualflare for hosted, historical CI observability. Qualflare does not run tests and is not a browser grid.

Qualflare works the same with JUnit, TestNG, pytest, Playwright, TestCafe and 20+ more frameworks. Chasing flakes? See flaky or a real bug? and the JUnit XML format guide. Weighing tools? See how it compares to other test management platforms, or browse all framework reporting guides. Every reporter is open source.

Frequently asked questions