Product news and testing tips.
GitLab's `artifacts:reports:junit:` parses JUnit XML into a merge request widget that compares your branch against its target, and a pipeline Tests tab. It enforces four separate limits — 30 MB per file, 100 MB per job, a 20 MB display cap, and 500,000 test cases on GitLab.com — silently ignores several JUnit attributes, and does not affect job status. It has no flaky-test detection and no cross-pipeline test history.
Key takeaways
- `artifacts:reports:junit:` accepts globs and arrays, but never a bare directory path.
- Four separate limits apply: 30 MB per file, 100 MB per job, a 20 MB MR display cap, and 500,000 test cases on GitLab.com.
- Unit test reports do not affect job status — your script must exit non-zero to fail the job.
- GitLab uses `testcase classname` as the suite name and ignores `testsuite name` entirely.
- GitLab ships no flaky-test detection and no test-case history; the only cross-pipeline signal is a 14-day failure counter.
GitLab does more with a JUnit XML file than GitHub Actions does. That is a low bar, but it clears it properly: point artifacts:reports:junit: at your results and you get a merge request widget that tells you which tests broke on this branch specifically, plus a browsable Tests tab on the pipeline.
It is genuinely useful, and it is also more constrained than most teams realise. There are four separate size limits documented across three different pages, a set of JUnit attributes GitLab silently discards, and two features people routinely assume exist that do not.
How to wire it up
rspec:
stage: test
script:
- bundle exec rspec --format RspecJunitFormatter --out rspec.xml
artifacts:
when: always
reports:
junit: rspec.xmlThe path accepts three shapes:
junit: rspec.xml # single file
junit: rspec-*.xml # glob
junit: [rspec-1.xml, rspec-2.xml] # array
junit: test-results/**/*.xml # recursive globGlobs and arrays combine. What does not work is a bare directory — GitLab’s documentation gives junit: test-results and junit: test-results/** as explicit non-examples. The ** only works as part of a pattern that matches files.
when: always is not optional. A failing job skips artifact upload by default, which means the run you most want a report for is the run that produces none. GitLab’s own examples all set it.
What GitLab shows you
Two surfaces, both available on Free, Premium and Ultimate — this is not a paid feature.
The merge request widget is the genuinely clever part, because it does a comparison rather than a dump. GitLab classifies every result into four buckets by diffing your source branch against the target branch:
| Bucket | Meaning |
|---|---|
| Newly failed | Passed on the target branch, fails on yours |
| Newly encountered errors | Passed on the target branch, errors on yours |
| Existing failures | Failed on both branches |
| Resolved failures | Failed on the target branch, passes on yours |
That distinction is the single most valuable thing in GitLab’s test reporting. “This branch broke 3 tests” and “this branch inherited 47 pre-existing failures” are completely different situations, and most CI systems present them identically.
The pipeline Tests tab shows all suites and cases in the pipeline, including results from child pipelines — useful for a monorepo fanning out work.
The limits, which are four different numbers
This is where people get caught, because the figures live on three separate documentation pages and are easy to conflate:
| Limit | Value | Scope |
|---|---|---|
| Individual JUnit file | 30 MB | Per file |
| All JUnit files in one job | 100 MB | Per job, total |
| MR widget display | 20 MB | Report not loaded above this |
| Test cases per report | 500,000 | GitLab.com only — self-managed documented as unlimited |
Note what these are not: none of them vary by Free, Premium, or Ultimate. The only split is GitLab.com versus self-managed.
The 20 MB display cap is the sneakiest, because it sits below the 30 MB upload limit. A report between 20 and 30 MB uploads successfully, counts against your job total, and then simply does not render in the merge request. Nothing is broken; nothing appears.
When the panel is empty, GitLab’s documentation names two causes: the report artifacts have expired, or the JUnit files exceed size limits.
Reports do not affect job status
Worth stating on its own line, because it surprises people:
Unit test reports require the JUnit XML format and do not affect job status. To make a job fail when tests fail, your job’s script must exit with a non-zero status.
GitLab parses the XML purely for display. If your test command is wrapped in something that swallows the exit code — a shell pipeline, a || true someone added to debug an unrelated problem — you get a green job and a report full of red. The report is not a gate. Making it one is the job of a quality gate you configure yourself.
What GitLab silently ignores
GitLab’s parser makes opinionated choices about the JUnit XML dialects it accepts, and the documentation is unusually candid about them:
testcase classnameis displayed as the suite name. Thetestsuite nameattribute is not displayed at all. If you have carefully named your suites, GitLab does not care.- Ignored on
testsuite:tests,failures,errors,timestamp. - Ignored on
testcase:assertions,line,status. - Duplicate test names: only the first is used; the rest are discarded.
The ignored-counters behaviour is the right call, and it matches what any careful JUnit XML consumer does — header counters are written by the producing tool and can disagree with the actual case list, so deriving everything from the cases is the only way to keep the summary honest. It is worth knowing that GitLab does this, though, because it means a discrepancy between your runner’s summary line and GitLab’s displayed counts is expected rather than a bug.
The duplicate-name rule is the one that can genuinely lose data. Parameterised tests that do not encode their parameters into the test name will collapse into a single displayed case.
The two things GitLab does not have
Here is where accuracy matters, because searching GitLab’s documentation for “flaky” produces a lot of promising-looking results that are not what they appear.
There is no flaky-test detection. The pages on docs.gitlab.com covering flaky tests, quarantining, and rspec-retry describe GitLab’s own internal engineering process for testing GitLab the product. They are developer-handbook content. They carry no tier badge because they are not a tier feature. The customer-facing feature request has been open since 2017 and remains unshipped.
If you have read those pages and concluded GitLab will find your flaky tests, it will not, on any plan.
There is no test-case history. The merge request widget compares exactly two things: your source branch and its target. That is a comparison, not a history — it tells you what this branch changed, not what this test has been doing for the past month.
The one genuine cross-pipeline signal is narrow and worth knowing precisely: on a failed test, GitLab can show “Failed {n} time(s) in {default_branch} in the last 14 days.” That counter includes failures from completed pipelines but not blocked ones. It is a real historical signal, it is 14 days deep, it is default-branch only, and it is a count with no context — no trend, no per-test view, no way to ask which tests fail most.
One naming trap: GitLab’s Test Cases feature is a different product. It is manual test-case management, unrelated to JUnit result parsing. Searching for “test case history” will lead you there.
Where this leaves you
GitLab’s test reporting is better than GitHub’s out of the box, principally because of the target-branch comparison, which is a genuinely good idea well executed. Both stop at the same boundary: GitHub Actions has no native parsing at all, and GitLab parses well but keeps almost nothing.
The questions that survive past a single merge request — has this test always been unreliable, is this suite degrading, are these failures one cause or many — need results stored over time and compared. A 14-day counter on the default branch is the closest GitLab gets, and it is a long way from an answer.
For transparency: Qualflare is our product, a test observability layer that ingests results from CI and analyses them across runs. It does not run your tests and does not replace GitLab’s widget — the target-branch comparison is genuinely good and you should keep using it. Our GitLab CI integration documents the setup if the history gap is one you actually feel. If your current problem is just that reports are not showing up, it is almost always when: always or a size limit, and both are free to fix.
Frequently asked questions
How do you add a JUnit test report to GitLab CI?
Add the report path under artifacts:reports:junit in your job definition. It accepts a single path, a glob such as rspec-*.xml, or an array of paths, and globs and arrays can be combined. A bare directory path is not valid. You should also set artifacts:when:always, otherwise a failing job skips the artifact upload — which is exactly when you wanted the results.
What are GitLab’s JUnit test report size limits?
Four separate limits apply. Each individual JUnit file is capped at 30 MB, all JUnit files in a single job at 100 MB total, and the merge request widget will not load a report larger than 20 MB. Separately, GitLab.com caps a unit test report at 500,000 test cases; self-managed instances are documented as unlimited. None of these vary by Free, Premium, or Ultimate tier.
Why does my GitLab job pass when tests fail?
Because unit test reports do not affect job status. GitLab parses the XML for display only; making the job fail is your script’s responsibility, and it has to exit with a non-zero status. A test runner invoked in a way that swallows its exit code produces a green job and a report full of failures.
Does GitLab detect flaky tests?
No. GitLab has no customer-facing flaky-test detection feature at any tier. Documentation about flaky tests on docs.gitlab.com describes GitLab’s own internal engineering process for testing GitLab itself, not a product capability, and the feature request has been open since 2017. The only cross-pipeline signal GitLab surfaces is a counter showing how many times a test failed on the default branch in the last 14 days.
Does GitLab keep test result history across pipelines?
Not in any meaningful sense. There is no test-case history feature. The merge request widget compares only the source and target branches of that merge request, and the single historical signal is a 14-day failure count on the default branch. Note that GitLab’s Test Cases feature is something different — manual test-case management, not JUnit result history.
