İbrahim Süren

Articles by İbrahim Süren

• What Espresso's JUnit XML Leaves Out of a Test Report

  Espresso's JUnit XML carries one line per test. Here is what it drops — screenshots, steps, retry history — and the Android-specific traps in capturing them.

  Sep 25, 2026 · 8 min read

• What Maestro's JUnit XML Leaves Out of a Test Report

  Maestro's JUnit XML carries one line per flow. Here is what it drops — steps, screenshots, failed runs — and how to report a Maestro run in full.

  Sep 20, 2026 · 8 min read

• Jest vs Vitest: The Migration Cost Nobody Measures

  Speed is the easy part. The real Jest to Vitest migration cost is behavioural: mockReset is inverted, __mocks__ stops loading, and hooks nest.

  Sep 15, 2026 · 10 min read

• AI in Software Testing: What Actually Works

  A practical guide to AI in software testing — what genuinely ships today, what's oversold, and how to evaluate any vendor's AI claim before you buy.

  Sep 9, 2026 · 14 min read

• CTRF: A Common Format for Test Reports

  CTRF is a JSON test report format with retries and flakiness as first-class fields. What the schema actually requires, and an honest read on its adoption.

  Sep 9, 2026 · 9 min read

• Cucumber Test Reporting: Steps, Retries & JUnit XML

  How cucumber-js maps scenarios and steps onto test reports, why the JSON formatter is in maintenance mode, and what --retry does and doesn't record.

  Sep 9, 2026 · 8 min read

• Flaky or a Real Bug? A Triage Procedure

  The thresholds everyone repeats have no source, and one was rolled back by its author. What GitLab, GitHub, Dropbox and Google actually publish instead.

  Sep 9, 2026 · 12 min read

• Test Reporting in GitHub Actions and Its Limits

  GitHub Actions has no native test-result parsing. What annotations, job summaries and artifacts actually give you — and the documented limits of each.

  Sep 9, 2026 · 10 min read

• GitLab CI Test Reports: What junit: Actually Does

  GitLab parses JUnit XML into a merge request widget and a Tests tab. The exact limits, the attributes it silently ignores, and the two features it doesn't have.

  Sep 9, 2026 · 9 min read

• Go Flaky Tests: -shuffle, -count and synctest

  Why Go tests flake — map ordering, goroutine scheduling, shared state and real clocks — and the four toolchain flags that expose each one.

  Sep 9, 2026 · 9 min read

• Go Test Reporting in CI: JSON, gotestsum & JUnit XML

  go test -json is an event stream, not a report. How to turn it into JUnit XML with gotestsum or go-junit-report — and the three ways assembly goes wrong.

  Sep 9, 2026 · 11 min read

• JUnit XML Format: The Spec That Doesn't Exist

  JUnit XML is the default test result format everywhere — and it has no official specification. A field guide to the elements, the dialects, and the gaps.

  Sep 9, 2026 · 19 min read

• Tracking k6 Results Across Runs, Not Just Thresholds

  k6 thresholds pass or fail one run. How to get results out with handleSummary, and why cross-run comparison is a Grafana Cloud feature, not an OSS one.

  Sep 9, 2026 · 9 min read

• Newman & Postman Test Reporting in CI

  How Newman maps API requests and assertions onto test results, how to get JUnit XML out of it, and the skipped-assertion gap in the built-in reporter.

  Sep 9, 2026 · 9 min read

• Open Source Test Reporting & Observability Tools

  Which OSS tools actually store test history and score flakiness — Allure, ReportPortal, CTRF — and where the open-source path genuinely stops.

  Sep 9, 2026 · 11 min read

• RSpec Flaky Tests: --bisect, --seed, let vs let!

  How to isolate order-dependent RSpec failures with --bisect and --seed, why before(:context) leaks state, and what let! actually does.

  Sep 9, 2026 · 10 min read

• Software Testing Statistics: Every Number, Sourced

  The 100x cost-of-defects claim has no source. Neither does 50% debugging. What testing statistics actually survive a check on their primary sources.

  Sep 9, 2026 · 14 min read

• Test Automation ROI: Why the Standard Formula Is Wrong

  Every vendor publishes the same ROI equation. It omits maintenance and rerun cost — and one vendor's own calculator books maintenance as a saving.

  Sep 9, 2026 · 10 min read

• Test Report Formats Compared: JUnit XML, CTRF, TRX, TAP

  Which test report formats have a real spec, and which can express retries, attachments, steps and history. A capability matrix across seven formats.

  Sep 9, 2026 · 11 min read

• How to Test LLM Applications and AI Agents

  Assertion-based tests break on non-deterministic output. What replaces them — graded evals, LLM judges and their measured biases — and how it reaches CI.

  Sep 9, 2026 · 11 min read

• Why Visual Regression Tests Flake

  Antialiasing, headless shells, font loading and GPU differences make pixel diffs unstable. The real defaults, the real research, and the two ways out.

  Sep 9, 2026 · 10 min read

• Your HTML Test Report Is a Zip Nobody Opens

  CI produces a beautiful HTML test report, zips it, and buries it. Why file:// breaks reports, what works for hosting, and what hosting can't fix.

  Sep 9, 2026 · 10 min read

• Android vs iOS Testing: Key Differences (2026)

  Android and iOS testing diverge in build tools, CI runner cost, OS fragmentation and release process — not just frameworks. What it means for CI/CD.

  Aug 14, 2026 · 13 min read

• Appium Flaky Tests: Root Causes & Fixes (2026)

  Appium's WebDriver layer is its own flake source — session and capability mismatches, wait anti-patterns, driver differences, and stale sessions.

  Aug 14, 2026 · 11 min read

• Appium vs Espresso vs XCUITest vs Maestro (2026)

  Espresso and XCUITest are platform-locked; Appium and Maestro go cross-platform. Compare language, CI setup and JUnit XML output across all four.

  Aug 14, 2026 · 14 min read

• Best Mobile Test Management & Observability Tools (2026)

  Compare 8 tools for managing Android and iOS test results in 2026 — TestRail, Zephyr, Allure TestOps, ReportPortal, Testmo, Kobiton and Qualflare.

  Aug 14, 2026 · 18 min read

• Espresso Flaky Tests: Root Causes & Fixes (2026)

  Espresso waits for the main thread to idle — but network calls, coroutines, animations and RecyclerView diffing sit outside it. Root causes and fixes.

  Aug 14, 2026 · 15 min read

• Flaky Mobile Tests: Why Android & iOS Tests Fail (2026)

  Mobile tests flake for reasons web tests don't: device fragmentation, animation timing, emulator variance, permission dialogs. Root causes and fixes.

  Aug 14, 2026 · 15 min read

• Maestro Mobile Testing: How It Compares to Appium (2026)

  Maestro is an open-source, YAML-based UI testing framework for Android, iOS, React Native and Flutter. What it is, and how it compares to Appium.

  Aug 14, 2026 · 10 min read

• Manual vs. Automated Mobile Testing (2026)

  Automation wins mobile regression testing and CI-gating; manual testing wins real-device feel, visual polish, and new-OS compatibility. How to split the budget.

  Aug 14, 2026 · 11 min read

• Mobile App Testing Checklist: Before Every Release (2026)

  A copy-paste mobile QA checklist: functional flows, device matrix, permissions, offline, deep links, store compliance, and what to automate.

  Aug 14, 2026 · 14 min read

• Mobile CI/CD Testing Pipeline: Build to App Store (2026)

  A worked mobile CI/CD pipeline: build, unit and UI tests, parallel Android and iOS jobs, a quality gate — with a real GitHub Actions workflow.

  Aug 14, 2026 · 10 min read

• Mobile Release Readiness: Quality Gates (2026)

  Mobile release gates need criteria web pipelines don't: per-platform flake thresholds, crash-free rate, device coverage, and open-defect counts.

  Aug 14, 2026 · 11 min read

• Mobile Test Reporting: What to Track Beyond Pass/Fail (2026)

  A green Espresso or XCTest run only says pass or fail. What mobile teams should track instead — flakiness trend, fragmentation, duration drift.

  Aug 14, 2026 · 10 min read

• Mobile Testing: The Complete Guide (2026)

  Qualflare ingests Espresso, XCTest/XCUITest and Maestro results — a mobile test observability layer, not a device cloud. Frameworks, flakiness, CI/CD.

  Aug 14, 2026 · 22 min read

• Espresso, XCTest & Maestro in One Dashboard (2026)

  Espresso, XCTest and Maestro produce three formats and no combined pass rate. How each reaches JUnit XML, and how to get all three into one view.

  Aug 14, 2026 · 11 min read

• What Is Mobile Test Observability? (2026)

  Mobile test observability analyzes Android and iOS results after they run — flaky detection, failure clustering, release risk. Not mobile monitoring.

  Aug 14, 2026 · 11 min read

• XCTest & XCUITest Flaky Tests: Stabilizing iOS CI (2026)

  XCTest flakiness is usually an async race; XCUITest flakiness is UI-query timing. Swift patterns for expectations, waits, and simulator variance.

  Aug 14, 2026 · 15 min read

• How to Root-Cause a Flaky Test (2026)

  A workflow for diagnosing one flaky test: measure its real flake rate, read the failure signature, correlate with deploys, confirm with debug flags.

  Jul 5, 2026 · 13 min read

• QA Metrics That Actually Matter (2026)

  Pass rate, defect density and automation rate get tracked but rarely change a decision. The QA metrics that do — and how to tell vanity from actionable.

  Jul 5, 2026 · 9 min read

• Conversational Test Analysis with Quo (2026)

  Quo is Qualflare's conversational AI agent — ask plain-language questions about your test data and get answers from its clustering and flaky analysis.

  Jul 5, 2026 · 9 min read

• Self-Healing Tests: How They Work & Fail (2026)

  Self-healing tests repair broken locators by matching similar elements or visuals. The actual mechanics, and the five ways the approach goes wrong.

  Jul 5, 2026 · 8 min read

• Test Retry Strategies: Help vs Hide Bugs (2026)

  Retries are safe for network flakiness and dangerous around business logic. A decision framework, retry-count math, and config for Playwright and Jest.

  Jul 5, 2026 · 12 min read

• The Real Cost of Flaky Tests (2026)

  Flaky tests cost more than CI minutes. A cost model — triage minutes x failure frequency x engineer rate — plus the delayed-release costs it misses.

  Jul 5, 2026 · 9 min read

• What Is Test Coverage? Line vs Branch & Good %

  Test coverage measures how much code your tests execute, not whether they check anything. Line vs branch coverage, and where the metric breaks down.

  Jul 5, 2026 · 10 min read

• AI in QA 2026: What's Real vs. What's Hype

  Every test tool now claims AI. We checked the docs behind a dozen vendors' claims — what genuinely ships, what's oversold, and what's just a rebrand.

  Jul 4, 2026 · 9 min read

• AI Quality Gates for Release Readiness (2026)

  An AI risk score isn't a quality gate by itself — it needs a policy: hard-block or advisory, how overrides are logged, how confidence is calibrated.

  Jul 4, 2026 · 8 min read

• Dashboards Aren't Analysis: Reporting vs Observability

  A dashboard only shows what you thought to graph in advance. Why that structurally limits reporting tools — and a vendor who renamed a product over it.

  Jul 3, 2026 · 8 min read

• Single Pane of Glass for Multi-Framework Tests

  Most teams run 4+ test frameworks, and each reporter sees only its own results. Why that's an observability problem, not just an aggregation one.

  Jul 3, 2026 · 8 min read

• Test Observability vs. Monitoring: What's the Difference?

  Monitoring watches a running system against thresholds you set. Test observability explains failures you didn't anticipate. The difference, with a table.

  Jul 3, 2026 · 7 min read

• Best CI for Test Automation (2026)

  GitHub Actions vs GitLab CI vs Jenkins for test automation in 2026 — matrix builds, parallel sharding, self-hosted runners, config-as-code, and real costs.

  Jul 2, 2026 · 16 min read

• Flaky Test Quarantine Strategy (and SLAs)

  How to size a flaky test quarantine SLA, tier it by severity, escalate breaches, cap the bucket, and quarantine tests in Playwright, Jest, pytest, JUnit.

  Jul 2, 2026 · 11 min read

• Quality Gates in CI/CD

  A quality gate is a CI/CD checkpoint that blocks merges unless criteria pass. How gates work, real tool examples, and DORA's evidence on approval boards.

  Jul 2, 2026 · 10 min read

• Reduce PR Cycle Time With Faster Test Feedback

  Slow, flaky CI adds real minutes-to-hours to a pull request's wait time. Here's how Shopify, Cal.com, and Uber cut CI time and shortened PR cycle time.

  Jul 2, 2026 · 8 min read

• Shift-Left Testing: A Practical Guide

  Shift-left testing means testing earlier in development. Its real 2001 origin, practical techniques, a shift-right comparison, and what the evidence shows.

  Jul 2, 2026 · 8 min read

• Test Case Template & Examples: Free CSV Download (2026)

  Free test case template — download the CSV for Excel, Sheets or Numbers with example rows filled in, plus the format and a worked example.

  Jul 2, 2026 · 9 min read

• Test Parallelization & Sharding in CI (2026)

  Exact Playwright, Jest, and pytest-xdist sharding flags, CI matrix YAML for GitHub Actions, GitLab, and CircleCI, and the math for choosing a shard count.

  Jul 2, 2026 · 10 min read

• AI Test Observability for CI/CD Pipelines (2026)

  Bring AI test observability into CI/CD: upload each run's results, cluster failures by root cause, score flaky tests, and gate releases on per-launch risk.

  Jun 26, 2026 · 12 min read

• How to Choose a Test Management Platform (2026)

  A practical 2026 guide to choosing a test management platform: define your needs, weigh the criteria that matter, and decide with a clear rule.

  Jun 26, 2026 · 14 min read

• Agentic Testing: What AI Agents Mean for QA (2026)

  Agentic testing is AI agents that plan and run testing tasks with little human direction. What it really is, what's real vs hype, and where humans stay.

  Jun 25, 2026 · 6 min read

• Best Flaky Test Detection Tools in 2026 (8 Compared)

  The 8 best flaky test detection tools in 2026, compared: history-based flaky scoring, auto-quarantine, and failure clustering, with each tool's best fit.

  Jun 25, 2026 · 18 min read

• 7 Best Test Observability Tools for 2026

  Compare the 7 best test observability tools for 2026 — flaky-test detection, failure clustering, and release-risk analysis across CI pipelines.

  Jun 25, 2026 · 18 min read

• Cypress Flaky Tests: Retries, Detection & Fixes (2026)

  Cypress tests flake from fixed cy.wait() calls, animations, and cross-test state. Fix with intercept-and-wait, isolation, and retries, then track flakiness.

  Jun 25, 2026 · 9 min read

• DORA Metrics for QA Teams (2026)

  DORA's four metrics measure delivery speed and stability. QA most affects stability — reliable, fast tests lower change failure rate and shorten lead time.

  Jun 25, 2026 · 11 min read

• Flaky Test Statistics 2026: Prevalence & Cost

  Flaky test statistics for 2026: how common they are (Google ~16% of tests), what they cost (Slack ~28 min/failure), causes, and how top teams fix them.

  Jun 25, 2026 · 10 min read

• Flaky Tests: The Complete Guide (2026)

  A flaky test passes and fails on the same code. The complete guide to detecting, quarantining, and preventing flaky tests in CI.

  Jun 25, 2026 · 11 min read

• How to Detect Flaky Tests Automatically (2026)

  You can't detect a flaky test from one run. The history-based approach to automatic flaky-test detection — signals, scoring vs flags, and tooling.

  Jun 25, 2026 · 6 min read

• Jest & Vitest Flaky Tests: Causes & Fixes (2026)

  Jest and Vitest tests flake from leaked module state, mocks, and timers. Reset state between tests, fake the clock, and detect flakiness over runs.

  Jun 25, 2026 · 5 min read

• Monorepo Test Aggregation: Unify CI Results (2026)

  In a monorepo, frameworks emit results in different formats. Test aggregation combines them into one view per run — overall health, flakiness, ownership.

  Jun 25, 2026 · 5 min read

• Playwright Flaky Tests in CI: Causes & Fixes (2026)

  Playwright tests flake in CI from timing on slow runners, parallel workers, and manual waits. Fix them with web-first assertions, isolation, and retries.

  Jun 25, 2026 · 6 min read

• Predictive Flaky Scoring: Catch Flakes Early (2026)

  Predictive flaky scoring uses a test's history to estimate its flakiness probability, so you can quarantine unreliable tests before they block a release.

  Jun 25, 2026 · 6 min read

• pytest Flaky Tests: xdist, Fixtures & Retries (2026)

  pytest tests flake from shared fixtures, parallel xdist runs, and test-order dependencies. Isolate state, expose order bugs, and detect flakiness over runs.

  Jun 25, 2026 · 5 min read

• Smart Test Selection: Cut CI Time (2026)

  Smart test selection runs only the tests a code change is likely to affect — how it works, how it relates to test impact analysis, and how it cuts CI time.

  Jun 25, 2026 · 7 min read

• Smoke vs Sanity vs Regression Testing (2026)

  Smoke testing checks the build is stable enough to test. Sanity checks a specific fix works. Regression checks nothing else broke.

  Jun 25, 2026 · 5 min read

• How to Speed Up Your CI Test Suite (2026)

  Speed up CI by running fewer tests (smart selection), running them concurrently (sharding), fixing flaky retries, and rebalancing toward the test pyramid.

  Jun 25, 2026 · 9 min read

• Test Observability vs Test Management (2026)

  Test management organizes test cases, plans, and runs. Test observability analyzes results for flaky tests, root causes, and risk — how they differ.

  Jun 25, 2026 · 6 min read

• Test Plan Template & Examples: Free Download (2026)

  Free test plan template to download — objectives, scope, approach, entry/exit criteria, risks and roles, with a worked example filled in.

  Jun 25, 2026 · 6 min read

• The Test Pyramid Explained (2026)

  The test pyramid favors many fast unit tests, fewer integration tests, and a few end-to-end tests. Why it matters and what the inverted pyramid costs.

  Jun 25, 2026 · 6 min read

• What Is AI Failure Clustering? (2026)

  AI failure clustering groups test failures that share a root cause, collapsing a wall of red into a few problems. How it works and why it speeds triage.

  Jun 25, 2026 · 7 min read

• What is Test Observability? A Complete Guide (2026)

  Test observability is understanding why your tests pass or fail over time — not just whether a run was green. How it differs from reporting and management.

  Jun 25, 2026 · 19 min read

• Why Do Tests Pass Locally but Fail in CI? (2026)

  Tests pass locally but fail in CI because the environments differ — timing, parallelism, ordering, missing env vars. The usual causes and fixes.

  Jun 25, 2026 · 6 min read

• 6 Best AI Test Management Tools for QA Teams (2026)

  Compare the 6 best AI test management tools for QA teams in 2026 — AI flaky-test detection, failure clustering, and launch risk analysis side by side.

  Jun 2, 2026 · 15 min read

• 6 Best Test Management Tools for Mid-Sized Teams (2026)

  Compare the 6 best test management tools for mid-sized teams in 2026 — from AI-powered Qualflare to Jira-native options like Zephyr and Xray.

  Jun 2, 2026 · 16 min read

• How to Evaluate Test Observability Platforms in 2026

  A practical framework for evaluating test observability platforms in 2026 — flaky test detection, quality gates, AI analysis, and what to ask vendors.

  Jun 2, 2026 · 16 min read

• 10 Test Management Adoption Challenges (& Fixes)

  The 10 most common test management platform adoption challenges — low adoption, CI/CD friction, flaky tests, reporting gaps — and a practical fix for each.

  Jun 1, 2026 · 15 min read