Product news and testing tips.
To speed up a slow CI test suite, attack it on four fronts: run fewer tests per change with smart test selection, run the rest concurrently with parallelization and sharding, stop wasting runners on flaky retries and reruns, and rebalance toward a bottom-heavy test pyramid so fast tests give early feedback.
Key takeaways
- The fastest test is the one you correctly skip — smart selection runs only the tests a change affects.
- Parallelization and sharding cut wall-clock time by running tests concurrently across runners.
- Flaky tests inflate CI time through retries and reruns — fixing them is a speed win, not just a reliability one.
- An end-to-end-heavy (inverted pyramid) suite is slow by construction; rebalance toward unit tests.
- Measure the CI feedback loop and treat it as a tracked metric, not a vibe.
A slow CI suite taxes every push. Developers wait, context-switch, and — worst of all — start batching changes to avoid the wait, which makes failures harder to localize when they finally appear. Speeding up CI isn’t one trick; it’s four levers applied together. This guide covers each. Where we reference Qualflare, we describe only what it actually does.
The short answer
A CI suite is slow for two reasons: you run too many tests, and you run them too sequentially. So you fix it from both directions — run fewer tests per change, and run the ones you keep concurrently — then remove the flakiness and structural bloat that waste time on top. Measure the CI feedback loop before and after so you know which lever paid off.
First, find out where the time actually goes
Almost every team that sets out to speed up CI starts optimising before measuring, and picks the lever that is most interesting rather than the one that is binding. Four numbers tell you which lever is yours:
| Measure | How to get it | What it tells you |
|---|---|---|
| Wall-clock vs summed test time | Pipeline duration against the sum of individual test durations | A large gap means you are serial-bound — parallelise. A small gap means you are volume-bound — select. |
| Slowest 1% of tests | Sort durations descending | A handful of tests often own a third of the runtime |
| Setup vs execution split | Time before the first test starts | If setup dominates, caching beats anything you do to the tests |
| Retry and rerun rate | Count of reruns per merged PR | Every rerun is the full pipeline cost paid twice |
The distinction in the first row is the important one. If your suite takes 40 minutes of wall-clock and the tests sum to 38 minutes, you have a concurrency problem and selection will barely help. If it takes 40 minutes and the tests sum to 400, you are already parallel and your problem is that you are running too much. Those two situations look identical from the outside and have opposite fixes.
The second row is worth checking before anything else, because it is so often the cheap win: suites frequently have a small number of pathological tests — a fixture that boots a real browser per case, a test that sleeps — where fixing three tests beats an architecture change.
The five levers at a glance
| Lever | Typical gain | Cost to adopt | When it is the right one |
|---|---|---|---|
| Smart selection | Large on big suites | Medium — needs dependency mapping and a scheduled full run | Tests sum to far more than wall-clock |
| Parallelization / sharding | Near-linear to a limit | Low-medium — needs parallel-safe tests | Wall-clock ≈ summed test time |
| Fixing flaky tests | Reclaims retries and reruns | Medium — needs run history | High rerun rate |
| Rebalancing the pyramid | Large but slow to realise | High — it is test rewriting | End-to-end heavy distribution |
| Caching | Small to large | Low | Setup dominates execution |
1. Run fewer tests: smart selection
The fastest test is the one you correctly skip. Smart test selection runs only the tests a given change is likely to affect, built on test impact analysis. Most changes are local, so most of the suite is irrelevant to any one commit — and the bigger the suite, the more there is to skip. Keep a scheduled full run as a safety net, and run the selected subset on the fast per-change path.
2. Run them concurrently: parallelization and sharding
Whatever tests remain, don’t run them one after another. Parallelization runs tests concurrently within a machine; sharding splits the suite across multiple machines or CI jobs. Ten balanced shards finish a suite in roughly a tenth of the time, minus overhead. Balance shards by historical duration so one slow shard doesn’t become the straggler that sets your total time. Most modern runners (Playwright, Jest, pytest with plugins) support sharding natively. Switching runners for speed is a related but separate decision — and a more expensive one than the benchmarks suggest, as the Jest to Vitest migration cost sets out. The mechanics — how to split, how to balance, and where the returns stop — are covered in test parallelization and sharding in CI.
Returns are near-linear only up to a point. Every shard pays the fixed setup cost again — checkout, dependency install, service boot — so at some shard count you are spending more on setup than on tests. If setup is three minutes and your suite is thirty, sixteen shards spend 48 minutes of setup to save 28 of testing. Measure the fixed cost before choosing a shard count; the optimum is usually lower than people expect.
One prerequisite: parallel-safe tests. Tests that share ports, fixtures, or data fail unpredictably under concurrency — which shows up as flakiness. So before you crank up parallelism, make tests independent.
3. Stop wasting runners on flaky tests
Flakiness is a hidden speed problem. A flaky failure triggers a retry, and a flaky red build often forces someone to re-run the entire pipeline — paying the full cost twice for a failure that was never real. And it isn’t rare: Google reported that ~16% of its tests have some level of flakiness, with about 1.5% of all test runs reporting a flaky result — at that scale, the retries and reruns add up to a steady tax on CI time. Detecting and fixing flaky tests reclaims that wasted compute and the human delay around it. This is why reliability work and speed work are the same work.
4. Rebalance toward the test pyramid
An inverted test pyramid — mostly slow, flaky end-to-end tests — is slow by construction. Push coverage down to fast, stable unit and integration tests where you can, and reserve end-to-end tests for critical paths. A bottom-heavy suite gives a trustworthy first signal in seconds, so most failures surface long before the slow tests even start.
5. Cache and skip redundant work
Beyond the tests themselves, avoid repeating work: cache dependencies and build artifacts between runs, reuse setup where safe, and don’t rebuild what hasn’t changed. In a monorepo, this pairs naturally with selection — only build and test the packages a change touched.
Runner performance itself is a variable worth checking rather than assuming — machine class, cache behaviour and concurrency limits differ substantially between providers, which is the subject of the best CI for test automation.
Measure it
Treat the CI feedback loop as a tracked metric, not a feeling. Watch the median and the tail — the worst runs are where developers actually get stuck, and a p95 of forty minutes is what people plan their day around even if the median is eight — and connect it to delivery outcomes through DORA metrics like lead time for changes. The historical result data that powers selection and flaky detection is the same test observability data you need to see where the time is going.
And keep the outcome in view rather than the proxy: CI duration is not the goal, it is an input to how quickly a change reaches production. Reducing PR cycle time with faster test feedback is the metric that actually reflects whether the work paid off — a suite that got twice as fast while reviews still take two days has not moved anything a user can feel.
Start free with Qualflare — send your CI results and see flaky tests, failure clusters, and the history that smarter, faster pipelines depend on.
Frequently asked questions
How do you speed up a slow CI pipeline?
Combine four levers: run fewer tests per change with smart test selection, run the remaining tests concurrently with parallelization and sharding, eliminate flaky tests that waste time on retries and reruns, and rebalance the suite toward fast unit tests. Cache dependencies and build artifacts to avoid repeating work, and measure the feedback loop so you know what’s actually slow.
What’s the biggest cause of slow CI?
Usually running the entire suite on every change when most of it is irrelevant, compounded by an end-to-end-heavy test distribution where the slowest, flakiest tests dominate. Running everything serially makes both problems worse. The fix is to run less (selection) and run it concurrently (parallelization).
Does fixing flaky tests make CI faster?
Yes. Flaky tests trigger retries and full-suite reruns, and a single flaky failure can force someone to re-run a 30-minute pipeline. Eliminating flakiness removes that wasted compute and the human delay around it, so reliability work is also speed work.
How do I measure CI speed?
Track the CI feedback loop — the time from pushing a change to a usable test result — as a first-class metric. Watch the median and the tail (the worst runs), and connect it to delivery outcomes through DORA metrics like lead time for changes.
