WebdriverIO test reporting

@qualflare/webdriverio reports your suite from inside each WebdriverIO worker — statuses, every attempt of a retried test, the screenshots WebdriverIO takes, and the browser or device each test ran on — for web runs and Appium alike. The part most reporters get wrong is the workers: each spec file runs in its own process and writes its own report, and those reports have to be recognised as one run. This guide covers the setup, how the workers agree on one run with nothing to configure, and the hosted, historical analysis — AI failure clustering, flaky-test scoring and per-launch risk — that sits on top.

One run, many workers

WebdriverIO's local runner starts a separate worker process for every spec file, and constructs a fresh instance of each reporter inside that worker. So a run of three spec files is three reporters, each knowing about one file, each writing its own report:

qualflare-results/
├── 3f1c…e2a.json      # worker 0-0: specs/login.spec.js
├── 9b07…41d.json      # worker 0-1: specs/checkout.spec.js
├── a4e2…77c.json      # worker 0-2: specs/search.spec.js
└── 5d9f…0be.png       # a screenshot, referenced from a report

qf collect merges that directory into one launch, and it decides what belongs together by a run id stamped into every report. That check is deliberate: it is what stops a report left over from yesterday from being merged into today's launch. When the files disagree, it keeps the newest run and ignores the rest. Which means the workers have to agree.

In CI they already do: the provider's run id is shared by every worker, including across the machines of a sharded job. Outside CI, every worker of one run has something else in common — the wdio run process that started it. Each worker derives the run id from that launcher process, its process id and start time, so a run's workers agree and the next run gets a new id. That holds on Linux, including inside the xvfb-run wrapper WebdriverIO 9 puts around each headless worker, on macOS and on Windows. If a worker ever has to fall back to an id of its own, it says so on stderr.

Send WebdriverIO results to Qualflare

npm install --save-dev @qualflare/webdriverio

Add the reporter to wdio.conf, keeping whichever reporter prints your terminal output:

// wdio.conf.js
export const config = {
  // ...
  maxInstances: 4,
  // The reporter alone is enough: every worker derives one run id
  // from the `wdio run` launcher process.
  reporters: ['spec', ['@qualflare/webdriverio', { environment: 'staging' }]],
  // The idiomatic screenshot-on-failure hook. Its screenshot is
  // attached to the test it was taken for, even though it runs
  // after the test's verdict.
  afterTest: async function (test, context, { passed }) {
    if (!passed) await browser.takeScreenshot();
  },
};

Run the suite, then upload the directory — never a single file, since each worker wrote its own:

npx wdio run wdio.conf.js
qf myapp collect ./qualflare-results

Reports go to ./qualflare-results by default. The option that moves them is resultsDir, not outputDir: WebdriverIO reads a reporter option named outputDir as the directory for the reporter's own log file, and sharing the name would put WebdriverIO's logs among your reports. Upload with @qualflare/cli v0.1.37 or newer, the first release that labels these launches as WebdriverIO.

What gets captured

• Retries, attempt by attempt. With Mocha's this.retries(n), a failed attempt keeps its own error, stack, start time and duration, and a test that passed on retry is marked flaky rather than collapsing into a green result.
• Screenshots, on the right test. Every takeScreenshot(), saveScreenshot() and element screenshot is picked up from WebdriverIO's command stream — including one taken in afterTest, after the verdict. A retried test keeps each attempt's screenshots, named by attempt.
• Where it ran. The browser name and version, or for Appium the platform, device, OS version and driver, are recorded on every case, and the launch's platform is derived from the session.
• One history per capability. A suite run on Chrome and Firefox keeps two histories, not one merged one — otherwise the second browser's result would silently replace the first's.

Annotate tests from inside the suite

A small runtime API, importable on its own so a spec file does not load the reporter, lets a test label itself, group its actions into steps, and record parameters — masked ones are dropped before they are written anywhere:

import { qualflare } from '@qualflare/webdriverio/runtime';

it('checks out', async () => {
  qualflare.label('team', 'payments');
  qualflare.parameter('card', '4242…', { masked: true });

  await qualflare.step('fill the card form', async () => {
    await $('#card').setValue('4242 4242 4242 4242');
  });
  await $('button=Pay').click();
});

CI: upload even when the suite fails

# .github/workflows/e2e.yml
- name: WebdriverIO
  run: npx wdio run wdio.conf.js
  continue-on-error: true

# Runs even when the suite failed: a red run is the one
# whose screenshots you actually want.
- name: Upload results
  if: always()
  run: |
    qf login "$QF_PROJECT" --force   # token from QF_TOKEN
    qf "$QF_PROJECT" collect ./qualflare-results

Common WebdriverIO reporting problems

WebdriverIO is one of fifteen frameworks with a native Qualflare reporter. Running Appium through it? See Appium test reporting. Weighing tools? See how Qualflare compares to other test management platforms, or browse all framework reporting guides.

Frequently asked questions