Flaky Tests & Reliability

Non-determinism (in tests)

Also known as: non-deterministic test

Non-determinism in tests is when the same test and the same code can yield different outcomes because the result depends on uncontrolled factors like timing, ordering, or shared state.

Non-determinism is the root mechanism behind flakiness. Sources include asynchronous timing and race conditions, dependence on wall-clock time or randomness, leftover state between tests, reliance on external services, and tests that assume a particular execution order.

The fix is to remove the uncontrolled variable: wait on conditions instead of fixed sleeps, isolate state, stub external dependencies, and make each test independent of order. Eliminating non-determinism is what turns a flaky test into a reliable one.

Same test + same code → different result, caused by an uncontrolled variable.
Common sources: async timing, shared state, external services, test order, randomness.
The fix is removing the variable — wait on conditions, isolate state, stub dependencies.

Frequently asked

What is the most common cause of non-determinism in tests?

Asynchronous waits — a test that does not wait properly for an operation to finish — are the single most common cause in Luo et al.’s empirical analysis of flaky tests, ahead of concurrency (race conditions) and test-order dependencies.

How do you remove non-determinism?

Replace fixed sleeps with waits on explicit conditions, isolate or reset shared state between tests, stub external services, seed any randomness, and make each test independent of execution order.

Sources

See it in your own test results

Qualflare detects flaky tests, clusters failures by root cause, and scores release risk from the test results you already produce in CI. Start free.

Start free with Qualflare

← Back to the testing & observability glossary.

Last reviewed June 26, 2026