XCTest & XCUITest test reporting
A practical guide to reporting iOS test results: what Xcode writes — an
.xcresult bundle, not JUnit-XML — how to convert
a usable bundle for the released Qualflare CLI, and CI patterns for macOS runners. Add
hosted test history
and statistical flaky-test detection, with optional asynchronous AI analysis where enabled and available.
Worth saying up front: Qualflare doesn’t run your XCTest or XCUITest suite. It’s a results and observability layer, not a simulator or device farm — it ingests supported, usable reports from your existing CI, simulator run, or device cloud.
How XCTest and XCUITest results actually work
XCTest is Apple’s unit- and integration-testing framework; XCUITest is the UI-testing layer built
on top of it for driving a running app through the accessibility tree. Both run the same way —
through Xcode’s build system, via xcodebuild test —
and both produce the same output format:
an .xcresult bundle,
a binary, database-like package, not XML.
# macOS with Xcode selected; shared scheme and simulator already available.
# Choose a new result path for each run; keep the resulting bundle.
xcodebuild test \
-scheme MyApp \
-destination 'platform=iOS Simulator,name=iPhone 15' \
-resultBundlePath TestResults.xcresult
The documented XCTest route
for Qualflare CLI 0.6.0 uses converted JUnit XML, not the raw bundle.
Apple’s xcresulttool reads the bundle, but its
JSON interface is version-specific. Before Xcode 16, use the older
get form. Xcode 16 introduced the
get test-results family; the deprecated object
interface requires --legacy.
Check your installed tool’s help and
Apple’s release notes
for compatibility rather than assuming every Xcode version accepts the same syntax:
# Before Xcode 16: the older JSON interface
xcrun xcresulttool get --path TestResults.xcresult --format json
# Xcode 16+: compatibility form for the deprecated object interface
xcrun xcresulttool get object --legacy --path TestResults.xcresult --format json # Xcode 16+ test-results interface; check your installed tool's help
xcrun xcresulttool get test-results summary --path TestResults.xcresult
xcrun xcresulttool get test-results tests --path TestResults.xcresult Either way, what comes out is JSON in Apple’s own schema — still not the JUnit-XML that CI dashboards and platforms like Qualflare consume. That gap is this page’s central topic.
Converting .xcresult to JUnit-XML: what to actually use
The standalone trainer repository
was retired when the code moved into fastlane in November 2021. That is not the same as the
maintained fastlane trainer action,
whose current documentation includes modern xcresulttool commands and a
force_legacy_xcresulttool override.
Another option is a7ex/xcresultparser:
release 2.2.0 shipped in July 2026.
It uses Apple’s tools and JSON internally, but writes JUnit-XML without a manual JSON transformation.
Use a version compatible with the Xcode that produced your bundle:
# Convert on macOS with compatible Xcode and xcresultparser installed.
# Use a new directory so no older XML can be collected accidentally.
report_dir=$(mktemp -d)
if xcresultparser -o junit TestResults.xcresult > "$report_dir/junit.xml" \
&& test -s "$report_dir/junit.xml"; then
echo "Current report: $report_dir/junit.xml"
else
echo "No usable conversion; do not collect this XML" >&2
exit 1
fi
Only collect junit.xml after conversion succeeds
and the file is nonempty. These checks reject missing or failed conversion output, not every
possible semantic defect; the importer can still reject an unusable report.
Install the converter from its maintainer’s Homebrew tap or GitHub releases.
CI patterns: GitHub Actions, Xcode Cloud, Bitrise-style CI
GitHub Actions job fragment. This is not a full workflow
or a verified iOS run. It assumes Homebrew, selected compatible Xcode, an available iPhone 15
simulator/runtime, a shared MyApp scheme, and project dependencies/signing configured before testing.
Supply an Editor-or-higher project access token as QF_TOKEN;
secrets may be unavailable for untrusted pull requests. Adapt the scheme, destination and checkout
metadata to your workflow. The fragment records reporting errors separately and returns the original
test failure first:
# Job fragment under jobs:, not a complete workflow.
# Configure on:/permissions: and project dependencies outside this fragment.
ios-tests:
runs-on: macos-14
env:
QF_TOKEN: ${{ secrets.QF_TOKEN }}
steps:
- uses: actions/checkout@v4
- name: Install reporting tools and save project alias
run: |
brew install qualflare/tap/qf
brew install a7ex/homebrew-formulae/xcresultparser
qf login myapp --force
- name: Test, then report if a usable bundle exists
shell: bash
run: |
set -euo pipefail
report_dir=$(mktemp -d "$RUNNER_TEMP/xctest.XXXXXX")
bundle="$report_dir/TestResults.xcresult"
echo "Retain raw evidence from: $report_dir"
test_status=0
xcodebuild test \
-scheme MyApp \
-destination 'platform=iOS Simulator,name=iPhone 15' \
-resultBundlePath "$bundle" || test_status=$?
report_status=0
if [ ! -d "$bundle" ]; then
echo "Missing current-run bundle; no upload" >&2
report_status=1
elif xcresultparser -o junit "$bundle" > "$report_dir/junit.xml" \
&& [ -s "$report_dir/junit.xml" ]; then
qf myapp collect "$report_dir/junit.xml" --format xctest \
--platform ios --branch "$GITHUB_REF_NAME" \
--commit "$GITHUB_SHA" || report_status=$?
else
echo "Incomplete bundle or conversion failed; no upload" >&2
report_status=1
fi
echo "xcodebuild=$test_status reporting=$report_status"
if [ "$test_status" -ne 0 ]; then exit "$test_status"; fi
exit "$report_status" The script attempts conversion after a test failure only if this run left a bundle. An existing directory may still be incomplete; conversion can fail. A failed collect is a reporting failure, not evidence the tests failed. Add your CI’s artifact-retention step for the raw bundle and logs before this temporary runner is discarded.
Xcode Cloud post-action script fragment. CI_RESULT_BUNDLE_PATH belongs to test actions,
not every workflow action. Install both tools and configure the secret in the action’s environment;
make the script executable. This Bash fragment only handles
test-without-building and preserves its
CI_XCODEBUILD_EXIT_CODE:
#!/bin/bash
# ci_scripts/ci_post_xcodebuild.sh — reporting script fragment
# Install qf/xcresultparser and expose QF_TOKEN in this action's environment.
set -euo pipefail
if [ "${CI_XCODEBUILD_ACTION:-}" != "test-without-building" ]; then exit 0; fi
: "${CI_XCODEBUILD_EXIT_CODE:?Requires post-action test exit code}"
test_status="$CI_XCODEBUILD_EXIT_CODE"
report_status=0
report_dir=$(mktemp -d)
bundle="${CI_RESULT_BUNDLE_PATH:-}"
if [ -z "$bundle" ] || [ ! -d "$bundle" ]; then
echo "Missing test-action bundle; no upload" >&2
report_status=1
elif xcresultparser -o junit "$bundle" > "$report_dir/junit.xml" \
&& [ -s "$report_dir/junit.xml" ]; then
if qf login myapp --force; then
metadata=(--commit "$CI_COMMIT")
# Branch variables depend on the workflow's start condition.
branch="${CI_PULL_REQUEST_SOURCE_BRANCH:-${CI_BRANCH:-}}"
if [ -n "$branch" ]; then metadata+=(--branch "$branch"); fi
qf myapp collect "$report_dir/junit.xml" --format xctest \
--platform ios "${metadata[@]}" || report_status=$?
else
report_status=$?
fi
else
echo "Incomplete bundle or conversion failed; no upload" >&2
report_status=1
fi
echo "xcodebuild=$test_status reporting=$report_status"
if [ "$test_status" -ne 0 ]; then exit "$test_status"; fi
exit "$report_status" Apple documents CI_COMMIT as always available, but
CI_BRANCH is start-condition-specific.
The script uses the PR source branch when present, otherwise the branch variable, and omits
--branch when neither exists. For tag/manual/scheduled
workflows, configure your intended metadata explicitly rather than inventing a branch label.
See Apple’s custom-script setup.
Bitrise and similar mobile CI. Use the same boundaries: run tests, retain the current bundle, attempt conversion when usable, then collect successful nonempty XML. Configure post-failure execution without masking the test exit code. A platform’s “always run” setting schedules a step; it cannot guarantee a report exists.
Common XCTest/XCUITest reporting problems (and fixes)
The public announcement for the Bitrise Mobile Insights Report 2025 describes aggregated Bitrise-customer data from January 2022–June 2025. It reports teams experiencing flakiness rising from 10% to 26% and an association between monitoring-tool use and 25% fewer flaky reruns. These are cohort observations, not universal mobile prevalence or proof monitoring caused the difference. The announcement does not establish every comparison denominator or method; the gated full report was not reviewed for this guide.
- No .xcresult bundle at all after a crash. If the test
process crashes mid-run — a segfault in the app under test, an XCUITest host-app timeout — Xcode
can fail to finalize the result bundle, leaving nothing for
xcresultparserto read. Check the job for a non-zeroxcodebuildexit combined with a missing bundle path, and treat it as its own failure signal rather than a silent gap in the report. - Simulator boot-time variance inflates or times out CI runs. Simulator boot time on shared CI runners is inconsistent — cold caches, resource contention with parallel jobs. Consider booting the destination once before the suite and use an explicit CI timeout appropriate to your workload. A timeout can still leave an incomplete bundle; keep the xcodebuild logs and report that failure separately.
- Intermittent element-query failures need investigation. Animations, application races, network state, selectors and simulator timing can all contribute to “element not found.” Investigate the application, test and environment rather than ruling any of them out. Historical pass/fail variation helps prioritize triage; it does not establish cause.
- Xcode-version mismatches break bundle parsing. An
.xcresultbundle produced by one Xcode version isn’t guaranteed to parse cleanly with a converter built against a different one — a developer running Xcode 15 locally and CI running Xcode 16 (or vice versa) is a common source of “it works on my machine, not in CI.” Pin the Xcode version in CI (xcode-select/xcodes) to match what generated the bundles you’re testing conversion against. - Reporting is skipped or uses an old file.
Post-failure scheduling such as
if: always()does not create a bundle or validate XML. Use a new result directory per run, gate collection on successful conversion and nonempty output, and surface missing/incomplete bundles separately. Never let successful reporting turn a failed test job green.
Send XCTest/XCUITest results to Qualflare
For released CLI 0.6.0, use the conversion-first route on macOS too: a usable current-run bundle → JUnit XML → collect. First install the tools and authenticate an alias for the project that should receive the results:
# macOS setup; reviewed collect contract: qf 0.6.0
brew install qualflare/tap/qf
brew install a7ex/homebrew-formulae/xcresultparser
# CI: expose a project access token as the QF_TOKEN secret.
# Save the alias in each job that uploads; do not echo the token.
qf login myapp --force
Locally, qf login myapp can use browser approval.
In CI, QF_TOKEN supplies the project access token
to login; collect uses the saved alias, not that variable directly. Log in in each upload job.
See authentication
and collect metadata and exit codes.
Set --platform ios; choose an existing project
environment with --environment when needed,
and pass branch/commit metadata if CI auto-detection does not describe the tested revision.
# Convert on macOS with compatible Xcode and xcresultparser installed.
# Use a new directory so no older XML can be collected accidentally.
report_dir=$(mktemp -d)
if xcresultparser -o junit TestResults.xcresult > "$report_dir/junit.xml" \
&& test -s "$report_dir/junit.xml"; then
echo "Current report: $report_dir/junit.xml"
else
echo "No usable conversion; do not collect this XML" >&2
exit 1
fi # Same shell as conversion, after successful conversion and login:
qf myapp collect "$report_dir/junit.xml" --format xctest --platform ios
The XML route imports the test names, statuses, durations and failure text represented in a
supported report. It is not a lossless bundle upload: activities, attachment bytes and retry
attempts are not guaranteed to survive conversion/import. Keep the original
.xcresult and CI logs for captured evidence
outside the imported summary. Conversion needs macOS with compatible Xcode tools; a separate
supported CLI platform can collect the resulting XML after receiving that current-run artifact
and configuring its own login.
What hosted reporting adds to the converted summary
- Failure-cluster suggestions. Background clustering groups similar imported failures for investigation. A cluster needs five failures before naming/analysis; suggested causes and fixes need confirmation against the original evidence.
- Statistical flaky-case detection. Historical pass/fail variation is scored once configured sample thresholds are met. A retry-based flaky run is distinct and requires imported attempt data, which this XML route does not guarantee. Neither score identifies the cause.
- Asynchronous launch analysis. Uploaded launches can receive a background summary and recommendations. Risk is derived from the pass rate of tests that ran, not a ship/don’t-ship verdict or proof of release safety.
- History & trends. Compare imported results across launches with appropriate branch/environment metadata; a single raw bundle does not provide that hosted aggregation.
- Context & defects. Review names and failure messages represented in the imported XML, and create a defect from a failing run.
Statistical detection is separate from AI. AI Launch Analysis requires an uploaded launch, project AI enabled, an active eligible subscription and available allowance where applicable. It can be pending, failed, disabled, unavailable or skipped when the allowance is exhausted; results remain available without an analysis. All-pass/no-tests launches use fixed reports rather than a model call. Failure clustering is a separate, slower job with plan, workspace cluster-cap and shared monthly allowance gates. Open Source does not include clustering. There is no guaranteed analysis completion time.
Raw .xcresult bundle vs Qualflare
| .xcresult bundle | Qualflare | |
|---|---|---|
| Readable without Xcode / xcresulttool | — | Yes |
| History across CI runs | — | Yes |
| AI cluster suggestions (asynchronous) | — | When enabled and eligible |
| Flaky-test scoring over time | — | Yes |
| Offline inspection with compatible local tools | Yes | — |
| Captured activities, logs & attachments | If captured in bundle | Keep original CI artifact separately |
They’re complementary: keep the original test result bundle for evidence it actually captured.
An app build .xcarchive is not a substitute for a test .xcresult.
Qualflare adds hosted history for successfully imported reports, not every attempted CI run.
Build a history of your iOS test results
Start free — convert a usable .xcresult bundle
and collect its XML with your configured project alias. Explore background AI analysis where
project settings, subscription and allowance permit; clustering has separate plan limits.
Building out Android coverage too? See Espresso test reporting (JUnit-XML natively, no conversion) or Maestro test reporting for cross-platform E2E flows, Flutter test reporting if the app is built with Flutter, and how to get Espresso, XCTest, and Maestro into one dashboard. Chasing intermittent iOS failures? Read fixing flaky mobile tests or the mobile testing complete guide. Qualflare supports XCTest alongside 23+ other frameworks — see the full list, or browse all framework reporting guides. Weighing tools? See how it compares to other test management platforms.
Frequently asked questions
Does Qualflare run my XCTest/XCUITest tests on simulators or real devices?
No. Qualflare is a results-management and observability layer, not a simulator or device-execution cloud. Run your suite locally or in your existing CI, then supply a supported, usable report. For XCTest and XCUITest, convert the current .xcresult bundle to JUnit XML on macOS with compatible Xcode tools before collecting it.
Do I need to convert .xcresult before uploading?
Yes. For released CLI 0.6.0, convert a usable .xcresult bundle to JUnit XML, then run qf myapp collect junit.xml --format xctest --platform ios after configuring the myapp project alias. This conversion-first route applies on macOS too. Keep the original bundle: the XML import does not guarantee preservation of activities, attachments or retry attempts.
Which iOS test frameworks does Qualflare support?
XCTest and XCUITest, both listed by name in Qualflare’s framework support. Both produce the same .xcresult bundle format from xcodebuild, so the same conversion-then-collect flow covers unit tests (XCTest) and UI tests (XCUITest) alike. Maestro, which also drives iOS UI flows, writes JUnit-XML natively with no conversion step — see the Maestro test reporting guide.
Should I use xcresulttool or xcresultparser to convert my results?
Apple’s xcresulttool exposes result data as JSON, not JUnit XML. a7ex/xcresultparser converts a compatible bundle to JUnit XML without a manual JSON transformation; its 2.2.0 release shipped in July 2026. The retired standalone fastlane-community/trainer repository moved into fastlane in November 2021. The maintained fastlane trainer action is another option and documents modern xcresulttool support plus a legacy override. Check converter compatibility with your selected Xcode version.
How do I send XCTest/XCUITest results to Qualflare?
Install qf and a compatible converter, select Xcode on macOS, and run your configured shared scheme on an available destination with a new -resultBundlePath. Convert only a usable current-run bundle; collect only nonempty XML from a successful conversion with qf myapp collect junit.xml --format xctest --platform ios. First save the project alias with qf login myapp; in each CI upload job, provide a project access token through QF_TOKEN and run qf login myapp --force. Configure an existing environment and branch/commit metadata as needed. Preserve the test exit status even if reporting succeeds.
Does Qualflare detect flaky XCUITest tests?
Qualflare can flag historical cases whose pass/fail results alternate once configured sample thresholds are met. A flaky run that passes after a failed attempt is a separate concept and requires retry data in the imported report; the XCTest XML route does not guarantee that data survives conversion. Neither history nor AI proves whether the test, application or environment caused a failure. Statistical detection is separate from asynchronous AI launch analysis and clustering, which require project AI and applicable plan/allowance conditions.
Setup reflects the Qualflare CLI (docs.qualflare.com), Apple’s XCTest/XCUITest and xcresulttool docs, and a7ex/xcresultparser reviewed 17 November 2026 against CLI 0.6.0. Published 14 August 2026. These instructional fragments are not a record of an executed XCTest or Qualflare upload. Written by İbrahim Süren, founder of Qualflare.