Flaky Mobile Tests: Why Android & iOS Tests Fail Randomly (2026)

Mobile tests fail randomly because Android and iOS add failure surfaces that don’t exist in web or backend testing: real devices and emulators behave differently, system permission dialogs interrupt flows unpredictably, animations race against assertions, and CI runners strain to run multiple emulators or simulators in parallel. Bitrise’s analysis of 10M+ mobile builds found the share of teams hitting flakiness rose from 10% in 2022 to 26% in 2025. The causes span device/OS fragmentation, timing, network variability, and CI resource contention — and each framework (Espresso, XCTest/XCUITest, Appium) adds its own flake surface on top.

Mobile suites also get big fast. Slack’s mobile engineering team runs 16,000+ automated Android tests and 11,000+ iOS tests across 120+ developers shipping 550+ pull requests a week — at that scale, even a low per-test flake rate produces a steady stream of red builds nobody trusts. This guide covers the mechanisms specific to Android and iOS, a flake source specific to each of Espresso, XCTest/XCUITest, and Appium, and how Qualflare’s results-based approach applies once you have data instead of a hunch. It’s the mobile-specific companion to why tests pass locally but fail in CI and root-causing a single flaky test — and one spoke in the complete guide to mobile testing. Where we reference Qualflare, our own platform, we describe only what it actually does.

Why do Android and iOS tests fail randomly?

Seven mechanisms account for most of what turns a green mobile build red without any code change — the same non-determinism behind flaky tests generally, expressed through mobile-specific mechanics. Some are Android-specific, some iOS-specific, most hit both, but each is diagnosable once you know which one you’re looking at.

Why does device and OS fragmentation cause flaky mobile tests?

Android fragmentation is the widest in software testing: thousands of device/OS/manufacturer-skin combinations, each with its own background-process-killing policy — Samsung’s One UI, Xiaomi’s MIUI, and Huawei’s EMUI are all notably aggressive about killing backgrounded apps to save battery — plus chipset and screen-density differences. iOS fragmentation is narrower, since Apple controls the hardware, but still real across device generations and iOS versions, which change when the OS decides to throttle or kill a backgrounded process. A test asserting a specific timing- or memory-dependent behavior on one emulator profile can pass in CI for months, then flake the moment it runs against a real low-end device with a different kill policy. A CI matrix narrower than the real-world device population isn’t measuring what the field failure rate measures.

Why do animations and transitions cause flaky mobile tests?

Both platforms animate UI transitions by default, and a test that queries or taps an element before its animation finishes gets a stale reference or a tap on the wrong coordinates. Android’s Espresso has a named, official mitigation for exactly this: the IdlingResource pattern, which lets a test register background work — network calls, database queries, animations run through a tracked executor — so Espresso’s synchronization waits for it to go idle before proceeding, instead of guessing with a fixed sleep. iOS’s XCUITest has similar built-in waiting on the app reaching quiescence, but that produces its own timeouts when the app has a legitimate continuous animation loop or polling behavior that never quiesces. Either way, tests that don’t explicitly account for animation state are the most common source of “sometimes passes, sometimes doesn’t” on both platforms.

Do emulators and simulators behave differently than real devices?

Yes, consistently enough that testing on only one is a known blind spot. Android emulators run on a virtualized, often architecture-translated CPU with virtualized sensors, GPS, and camera, and typically render animations and handle memory pressure differently than physical silicon. iOS simulators run compiled-for-simulator binaries on the host Mac’s own CPU rather than actual ARM hardware, which changes timing and skips hardware-specific behavior — thermal throttling, real camera/sensor latency, background refresh under real memory pressure — entirely. A suite that’s rock-solid on emulators/simulators and then flakes, or fails outright, on a real-device lab isn’t contradicting itself; it’s exposing behavior the virtualized environment never had to reproduce. Emulators and simulators are the right default for fast CI feedback, but they’re a different population from what users run, not a stand-in for it.

Why do permission dialogs break mobile UI tests?

System permission prompts — camera, location, notifications, contacts, Bluetooth — are owned by the OS, not the app, and they interrupt the test’s expected element tree the moment they appear. The unpredictability comes from when they appear: a first-run-only prompt a test wrote once and forgot about starts silently failing after a CI cache reset re-triggers the “first run” state, or after an OS update changes which permissions require a runtime prompt (Android 13’s added notification-permission prompt broke UI flows that never accounted for it). XCUITest and Appium tests are especially exposed because they drive the accessibility tree directly, and a system alert sitting on top of the app’s UI is a different subtree than the script expected. The fix is deterministic, not defensive scripting: pre-grant permissions before the test runs (adb shell pm grant on Android, xcrun simctl privacy grant on a simulator) so the dialog never appears.

How does network variability cause flaky mobile tests?

Mobile apps assume an unreliable network by design — retries, spinners, cached fallbacks — and mobile tests inherit that non-determinism unless the network layer is controlled. A test running against a live backend over an emulator’s virtual network, a simulator’s host-machine Wi-Fi, or a real device on a shared CI network sees a different latency and packet-loss profile on every run, and any assertion timed against a fixed wait instead of an explicit network-complete signal will flake as that latency drifts. This hits every framework equally, because it lives below all of them, in the app’s own networking stack. The fix mobile teams reach for once they’ve been burned by it is the same one used in web testing: stub or mock the network layer in test builds so timing depends on the app’s own logic, not live network conditions.

Why do cold starts and warm starts produce different test timings?

A cold start — launching the app with no process already in memory — can take several times longer than a warm start, where the process and often cached views are already resident. Tests tuned against warm-start timing, because that’s what a developer’s machine usually has, flake in CI, where every run typically forces a fresh cold start, and flake worse on a loaded CI runner where cold start is slower still than it was locally. The fix isn’t a longer fixed wait — a longer sleep just changes which percentile of runs still time out — it’s an explicit wait tied to an app-state signal (a specific view becoming visible, an IdlingResource going idle, an XCUITest element existing) rather than a duration guess.

Why does CI resource contention make mobile tests flakier than they are locally?

Running one Android emulator or iOS simulator locally is a very different resource picture than running several in parallel on a shared CI runner. Emulators need hardware-accelerated virtualization — KVM on Linux runners, Hypervisor.framework on macOS — to perform reasonably; CI providers that don’t expose it, or that oversubscribe CPU across parallel jobs, produce emulators that boot slowly and render sluggishly, exactly the conditions that turn a marginal timing assumption into a visible flake. iOS simulators face the same contention on shared macOS runners, where Apple’s own hardware-hosting requirement already caps parallel capacity. This is why a suite stable one shard at a time can start flaking the same week CI parallelism increases — the tests didn’t change, but the CPU and memory each emulator instance gets did.

Which mobile testing framework is most prone to flaky tests?

Device- and OS-level causes sit below all three frameworks, but each one also adds a flake source specific to how it talks to the device.

Why does Espresso flake on synchronization and animations?

Espresso is designed to synchronize automatically with the app’s main thread and message queue — its whole pitch is that you shouldn’t need manual waits. That synchronization only covers work Espresso can see: UI operations and anything routed through a tracked AtomicUiController. Background threads, animations run outside the tracked queue, and async network or database calls all sit outside Espresso’s default view, so a test can tap a button while Espresso considers the app “idle” and proceeds, even though an animation or async update is still in flight. IdlingResource, and its CountingIdlingResource implementation, exists specifically to close that gap — registering it tells Espresso’s synchronization to wait on exactly the work it couldn’t see on its own, which is why unregistered async work is the most common Espresso-specific flake source, not a bug in Espresso itself.

Why does XCUITest flake on element queries and simulator boot variance?

XCUITest queries the accessibility tree to find elements, and that tree can be mid-update — a view still animating in, a table still laying out cells — when a query runs, producing an element that exists but isn’t yet hittable, or a stale reference to a view about to be replaced. waitForExistence(timeout:) covers existence but not hittability, so tests that don’t separately confirm an element has settled flake on the gap between “exists” and “settled.” Simulator boot time itself also varies run to run — a fresh boot on a loaded CI Mac can take meaningfully longer than a warm one, and a suite timed against the fast case flakes whenever CI hands it the slow one.

Why does Appium add its own flake surface on top of Espresso and XCUITest?

Appium doesn’t replace Espresso or XCUITest — under the hood, it drives Android through the UiAutomator2 driver and iOS through XCUITest itself, so every flake source native to those frameworks is still present. What Appium adds is a WebDriver-protocol layer between the test script and the device: each command travels from the test, to the Appium server, to the platform driver, to the device, and the response makes the same trip back. That’s a real network hop — even on one machine, it’s still a local HTTP round-trip per command — and it adds its own latency and failure surface: a slow or dropped response mid-interaction produces a flake that has nothing to do with the app under test and everything to do with the protocol layer relaying to it. Suites migrating from native Espresso/XCUITest to Appium for cross-platform reach should expect a nonzero flake-rate cost from this layer alone.

Mobile flaky-test root causes at a glance

The seven root causes above don’t hit every framework the same way — some are universal, others concentrate on one platform’s tooling. This maps each to where it hits hardest and the fix that actually holds:

Root cause	Hits hardest	Typical fix
Device/OS fragmentation	Espresso, Appium (wide Android matrix)	Test a real device/OS matrix, not one emulator profile
Animation/transition timing	Espresso	Register an IdlingResource; avoid fixed sleeps
Emulator vs. real-device variance	Espresso, XCUITest, Appium	Validate critical flows on real devices before release
Permission-dialog interrupts	XCUITest, Appium	Pre-grant permissions (adb shell pm grant, xcrun simctl privacy grant)
Network-condition variability	Espresso, XCUITest, Appium	Stub/mock the network layer in test builds
Cold-start vs. warm-start timing	XCUITest, Espresso	Wait on app-state signals, not fixed sleeps or duration guesses
CI emulator/simulator resource contention	Espresso, XCUITest, Appium	Match parallel instance count to available CPU/RAM; enable hardware acceleration

How does Qualflare help with flaky mobile tests?

Qualflare is a results and observability layer for mobile tests, not a device-execution cloud — it doesn’t provision real devices, run emulators or simulators, or execute your suite. What it does is ingest the results your CI already produces. Espresso emits JUnit-XML natively through Gradle, and Maestro does the same with maestro test --format junit — both upload with no conversion step. XCTest/XCUITest needs one extra hop: Xcode’s native output is .xcresult, converted to JUnit-XML with the actively maintained a7ex/xcresultparser rather than fastlane-community/trainer, unmaintained since roughly 2019–2022. Appium has no official JUnit reporter of its own, but suites built on it with JUnit, TestNG, or a pytest harness already produce JUnit-XML through those tools, so they ingest the same way.

Once results land, the same flaky-detection, quarantine, and failure-clustering approach already documented for web test flakiness applies to mobile results with no special case: every test’s pass/fail outcome is tracked across runs, and a test is scored flaky from that history, not judged off one red build. Qualflare doesn’t fix any of the seven root causes above — registering an IdlingResource, pre-granting a permission, stubbing a network call is a test-authoring fix, not a platform one. What it does is show you, from real historical data, which of your Espresso, XCTest, Appium, or Maestro tests are actually flaky, how often, and — via clustering — whether several share one root cause, so the root-cause workflow starts from evidence instead of a hunch.

Start free with Qualflare — upload your Espresso, XCTest, Appium, or Maestro JUnit-XML and see which mobile tests are actually flaky, not just red today.

Frequently asked questions

What causes flaky tests on Android and iOS?

Mobile tests flake from causes that don’t exist in web or backend testing: device and OS fragmentation, emulator/simulator behavior diverging from real devices, animations and transitions racing against assertions, permission dialogs interrupting flows unpredictably, network-condition variability, cold-start versus warm-start timing differences, and CI runners straining to run multiple emulators or simulators in parallel. Bitrise’s analysis of 10M+ mobile builds found the share of teams hitting flakiness rose from 10% in 2022 to 26% in 2025.

Why does Espresso flake more than other Android testing approaches?

Espresso synchronizes automatically with the app’s main thread, but only for work it can see — background threads, untracked animations, and async network or database calls sit outside that view. A test can proceed while that work is still in flight unless it’s registered as an IdlingResource, the official, named mitigation Google built specifically for this gap.

Is Appium more flaky than Espresso or XCUITest?

Appium inherits every flake source from the framework it drives underneath (UiAutomator2 on Android, XCUITest on iOS) and adds its own: a WebDriver-protocol network hop between the test script, the Appium server, and the device for every command. That’s a real, additional flake surface on top of whatever’s already native to the platform driver.

Do real devices reduce flaky tests compared to emulators or simulators?

They change which flakes you see rather than eliminating flakiness. Emulators and simulators run virtualized or host-CPU environments that behave differently from real silicon on timing, sensors, and memory pressure, so a suite that’s stable on emulators/simulators can still flake or fail on real devices, and vice versa. Validate critical flows on real devices before release rather than treating emulator/simulator results as the full picture.

Can Qualflare fix flaky mobile tests automatically?

No — root causes like animation timing, permission dialogs, and network stubbing are test-authoring fixes, not something a results platform can do for you. Qualflare ingests Espresso, XCTest, Appium, and Maestro results via JUnit-XML and tracks each test’s pass/fail history across runs, so you know which mobile tests are actually flaky, how often, and whether they cluster around one shared cause — the evidence a fix needs, not the fix itself.

How do you get XCTest/XCUITest results into a JUnit-XML format?

Xcode’s native test output is .xcresult, not JUnit-XML, so it needs conversion. The actively maintained tool for this is a7ex/xcresultparser; the once-standard fastlane-community/trainer has been unmaintained since roughly 2019–2022 and shouldn’t be a new pipeline’s first choice.