Go Flaky Tests: -shuffle, -count and synctest

Go’s testing story is minimal by design, and that extends to flakiness: there is no retry flag, no quarantine mechanism, and no built-in flake report. What Go gives you instead is a small set of flags that make non-determinism show up rather than hide, which is arguably the better trade.

The causes are also distinctly Go-shaped. A Ruby test does not flake because of map iteration order; a Go test does, routinely, and by design.

Why Go tests flake

Cause	Why it bites	What exposes it
Map iteration order	Randomized by the runtime, on purpose	-shuffle, repetition
Goroutine scheduling	Different every run; worse under CI load	-race, -count
Package-level state	Shared across t.Parallel() tests	-shuffle
Real-clock dependence	Sleeps and timeouts assume machine speed	testing/synctest
Test caching	Hides a failure behind a cached pass	-count=1

Map iteration order is randomized deliberately

The Go specification does not define an iteration order for maps, and the runtime actively randomizes it. This is not an accident that Go tolerates — it is a decision to prevent code from depending on an order that was never guaranteed.

It is easy to underestimate how aggressive it is. Iterating an eight-key map twenty times in a single process produced eight distinct orderings on Go 1.26. Any test that ranges over a map and compares the result to a fixed slice is not occasionally flaky; it is a coin flip that you happened to win the first few times.

The fix is not a flag — it is sorting keys before you assert, or comparing sets rather than sequences.

Goroutine scheduling, and why CI is worse

Goroutine interleaving differs run to run, and CI amplifies it. Runners are typically slower, busier, and have a different GOMAXPROCS than a developer laptop, which widens every timing window a test was implicitly relying on. A test that passes locally a hundred times can fail one CI run in ten purely because the machine is contended.

-race is the highest-value flag here and the most under-used:

go test -race ./...

It finds actual data races rather than their symptoms. A test that flakes because two goroutines touch the same variable will often pass under -race too — but the detector reports the race regardless of whether the assertion happened to fail this time. That converts an intermittent failure into a deterministic finding, which is the whole game.

The loop variable trap is gone — check your Go version

This deserves stating clearly because a lot of published advice is now out of date.

Before Go 1.22, a for loop reused one variable across iterations. The classic table-driven parallel test was therefore broken by default:

for _, tc := range cases {
    t.Run(tc.name, func(t *testing.T) {
        t.Parallel()
        check(tc.input) // pre-1.22: every subtest saw the LAST tc
    })
}

Everyone learned to write tc := tc at the top of the loop. Since Go 1.22, each iteration gets its own variable and the workaround is unnecessary. Verified on Go 1.26: taking the address of a loop variable across three iterations yields three distinct addresses holding 0, 1 and 2.

If your module declares an older Go version in go.mod, the old semantics still apply — this changed with the language version, not just the toolchain. That is worth checking before you delete the tc := tc lines.

Shared state and t.Parallel()

t.Parallel() is where package-level state becomes a bug. Two tests that each mutate a package-level registry, a global config, or a shared temp directory will pass when run sequentially and fail unpredictably when run in parallel — and which one fails depends on scheduling.

t.Cleanup is the right tool for teardown, because it runs in the correct order relative to parallel subtests in a way that a bare defer does not.

-shuffle: prove order dependence

Added in Go 1.17:

go test -shuffle=on ./...

It accepts off, on, or an integer seed. The critical property is that it prints the seed it used. A real run emits:

-test.shuffle 1788950163029644000

Pass that number back as -shuffle=1788950163029644000 and you get the same order again. This is what separates -shuffle from just running tests in a random order: the failures it finds are reproducible, so you can actually debug them rather than watching them vanish.

Order dependence is one of the most common sources of non-determinism in any suite that shares state, and -shuffle is close to free to adopt.

-count=1: the cache will lie to you

Go caches test results. An unchanged package returns its previous result without executing anything:

go test -count=1 ./...

This is the documented idiom for disabling the cache, and it matters enormously when hunting a flake. Without it you can spend twenty minutes “failing to reproduce” a failure against a cached pass that never ran. Any run whose purpose is to observe flakiness needs -count=1.

For actively hunting, combine repetition with shuffling:

go test -count=20 -shuffle=on -race ./...

That is the highest-yield single command in this post.

testing/synctest: fake time for concurrent code

The hardest category is code that waits — retries with backoff, timeouts, tickers, rate limiters. Testing it honestly means either sleeping for real (slow, and still timing-dependent) or injecting a clock abstraction everywhere (invasive).

testing/synctest solves it at the runtime level. Inside a bubble, the time package uses a fake clock starting at midnight UTC on 2000-01-01, and time only advances when every goroutine in the bubble is durably blocked. A test that waits two seconds completes instantly and deterministically.

The version history changed twice, and getting it wrong means writing code that does not compile:

Go version	Status	API
1.24	Experimental, behind GOEXPERIMENT=synctest	synctest.Run
1.25	Generally available	synctest.Test
1.26	Old experimental API removed	synctest.Test

Verified on Go 1.26: go doc testing/synctest Run reports “no symbol Run in package testing/synctest”. Any tutorial showing synctest.Run under a GOEXPERIMENT flag is describing a package that no longer exists.

synctest.Wait is the companion — it blocks until every other goroutine in the bubble is durably blocked, which is how you assert on state after background work has settled without a sleep.

A CI recipe

- name: Race + shuffle
  run: go test -race -shuffle=on -count=1 ./...

- name: Flake hunt (nightly, non-blocking)
  run: go test -count=10 -shuffle=on -race ./...
  continue-on-error: true

Splitting them matters. The first is a gate and should be fast enough to block a pull request. The second is a hunt — it will find real problems, it is slow, and it should not block anyone at 2am. Pair it with a report your CI can actually read rather than raw logs.

What no flag can tell you

Every technique above works within one invocation. They provoke, reproduce, and diagnose — which is genuinely most of the work.

What they cannot answer is whether a given test is unreliable, because that is a property of its behaviour over time. A test that failed once under -shuffle might be deterministically broken under one specific ordering, or might fail one run in fifty regardless. Those need opposite responses, and one run cannot distinguish them. Google’s analysis of its own corpus found almost 16% of tests showed some level of flakiness; no single command would have surfaced that number.

Answering it means storing each test’s outcomes across runs and comparing them — the general shape covered in the complete guide to flaky tests.

Our own bias, declared: Qualflare is ours. It reads go test output and scores flaky tests from history; setup is on our Go test reporting page. It does not run your tests. And it is genuinely the second step: -race -shuffle=on -count=1 costs nothing, ships today, and will find more real bugs in a neglected Go suite than any dashboard.

Frequently asked questions

How do you find flaky tests in Go?

Run the suite repeatedly with -count above one and -shuffle=on so both repetition and ordering vary, and add -race to catch data races. Those expose order dependence and concurrency bugs within a single session. To know whether a specific test is actually unreliable rather than briefly unlucky, you need its pass/fail history across many runs, which no flag can give you.

Why do Go tests pass locally but fail in CI?

The most common causes are test caching hiding a real failure locally, different GOMAXPROCS and CPU contention changing goroutine scheduling, and tests that depend on map iteration order or wall-clock timing. CI machines are usually slower and busier, which widens every timing window your test was implicitly relying on.

What does go test -shuffle do?

It randomizes the execution order of tests and benchmarks. It accepts off, on, or an integer seed, and it prints the seed it used — a run emits a line like -test.shuffle 1788950163029644000. Passing that number back as -shuffle=<seed> reproduces the same order, so a failure it finds is reproducible rather than a one-off.

Does Go still have the loop variable capture bug?

No. Before Go 1.22 a for loop reused a single variable across iterations, so a captured reference or a parallel subtest could observe the final value. Since Go 1.22 each iteration gets its own variable. Verified on Go 1.26 — taking the address of the loop variable across three iterations yields three distinct addresses holding 0, 1 and 2.

What is testing/synctest for?

Testing concurrent code that depends on time without waiting for real time. Inside a synctest bubble the time package uses a fake clock that only advances when every goroutine in the bubble is durably blocked, so a test that would sleep for seconds completes instantly and deterministically. It went GA in Go 1.25 as synctest.Test.