Got questions?
We've got answers.

Yes — and in any framework, not just Playwright. Each shard only runs part of the suite, so comparing inside a shard would report every snapshot the other shards own as missing coverage; we measured that on a real 8-shard run and every machine failed. The fix is capture-only: shards capture, one job merges the captures and compares once.

You get one exit code, one report and one PR comment instead of eight partial ones. Shards don't need baselines checked out, and comparing on a single machine removes runner-to-runner font rendering as a source of phantom diffs. If a shard crashes and never reports, the merge refuses to compare rather than quietly checking part of your suite.

Two environment variables drive it — TESTIVAI_SHARD=3/8 and TESTIVAI_CAPTURE_ONLY=1 — honoured identically by the Playwright, Selenium, Python, Java and Ruby adapters. Because it's environment rather than a framework API, the same four steps port to GitLab, Jenkins, CircleCI and Buildkite: capture per node, publish, collect, compare once. Full recipe →

Not by default. A visual change usually means someone changed something on purpose, so treating it as a build failure is the wrong default. Changes post as a neutral check run: fully visible in the pull request, and GitHub treats neutral as satisfying a required check — so it never blocks the merge, even if you mark the check required.

The diff comment on the PR is the deliverable; the check is just the signal. If you do want a hard gate, set fail-on-diff: true on the Action. The one thing worth enforcing separately is fail-on-missing — a baseline that received no capture means coverage silently disappeared, which is different from an intentional change.

Measured, not hand-waved: a full-page screenshot of a straightforward page is about 175 KB, and re-approving one stores a second full copy — two versions of the same screenshot packed to 342,845 bytes against 355,259 raw, so git's delta compression saved only 3.5%. PNG is already compressed; there's nothing left to squeeze. Identical images cost nothing extra, since git is content-addressed.

So a 50-snapshot suite is roughly 8.5 MB initially, plus about that much per round of approvals, permanently. Fine for years on a normal project. Keep it small by approving deliberately rather than --all, using ignoreSelectors so snapshots stop changing for reasons nobody cares about, and letting CI shallow-clone.

Git LFS: usually no. It costs money past GitHub's free quota, burns bandwidth on every CI run, and breaks forks — contributors don't inherit your LFS budget, which defeats the point of committing baselines. It earns its keep only when the repo is private, the suite is large, and approvals are frequent. Full guidance →

The detection isn't AI at all. The diff regions, the selector attribution, whether the DOM changed, whether it's a style-only change — all of that is deterministic comparison. Same input, same output, every run, no tokens and no network call. Connect no model and you still get every one of those signals.

A model adds interpretation: connecting the change to the edit you just made ("you changed the brand token — it reached the header, the primary buttons and every card price"), triaging thirty changed snapshots down to the two that aren't the same cause, explaining it in a PR comment, and letting an agent catch its own unintended changes before you see them.

What it does not do is decide whether something changed — that's settled before the model is involved. Which is the point: a hallucinating model can write a poor explanation, but it cannot invent a regression or hide one, because it isn't in the detection path. See a real payload →

TestivAI is built by one developer, Budi Sugianto. There's no company behind it — which is why there's no hosted service to upsell and no pricing page waiting to appear. First release was March 2026; since then 18 versions of the core package have shipped, across 70+ releases on npm, PyPI and RubyGems. The releases page is the live record and the public roadmap shows what's next.

If maintenance ever stopped, nothing of yours breaks: it's MIT licensed, your baselines are PNG files in your own git repository, results.json is a semver-governed contract, and there is no account, API key or network call involved. The worst case is that it stops gaining features — the version you pinned keeps working offline, indefinitely. Full details →

Two layers. The pixel comparison uses perceived colour distance (YIQ) with a tunable threshold, so tiny anti-aliasing shifts don't register — that part is standard. The layer that actually kills false positives is the second one: every screenshot is captured alongside the page's DOM and computed styles. When pixels differ but the DOM and styles are identical, that's render noise and the report says so instead of asking you to review it. When the DOM is identical but styles changed, you get "style-only change" with the affected selectors, not a guess. Whole-page shifts are detected as movement rather than dozens of separate changes.

Yes! You can configure ignore selectors for any CSS selector. Common use cases include timestamps, dynamic ads, user-generated content, or animated elements. These regions are masked out before comparison. You can also mask areas outright — by CSS selector or coordinates — with masked regions visibly hatched in the diff so exclusions stay auditable.

We automatically wait for animations to complete before capturing screenshots. You can also configure custom wait conditions or use CSS selector-based waits to ensure the page is stable before analysis.

Yes! TestivAI integrates with GitHub Actions. We provide detailed setup instructions in our documentation.

Yes! TestivAI is designed to work alongside your existing tests. Simply add a witness() call where you want visual coverage — Playwright, WebdriverIO, and Selenium have their own adapters, and the Python and Java packages share the same baselines and report. No need to rewrite existing tests.

Playwright gets native adapters in TypeScript/JavaScript, Python (pytest plugin), and Java (JUnit 5, experimental) — all sharing one set of baselines and one report. Selenium, WebdriverIO and Ruby (Capybara / RSpec) have their own adapters as well. Cypress, Puppeteer and Robot Framework go through the framework-agnostic Witness CLI, which is still experimental. Apps with no test suite at all can use npx testivai witness <url>.