Visual Regression Testing: Catching UI Bugs Before Users Do
Pixel-diff screenshot testing catches the UI bugs that functional tests are blind to — broken layouts, overlapping elements, invisible text. Here's how to use it without drowning in false positives.
Functional tests verify behaviour: click this button, check this text appears. What they don't verify is whether the page looks right. A button can be fully functional but overlapped by a tooltip. Text can render with the correct content but be invisible due to a colour regression. A modal can open correctly but push the layout out of alignment.
Visual regression testing catches these by comparing screenshots against a known-good baseline.
How pixel-diff testing works
The process has three steps:
- Establish a baseline — take screenshots of each page/component when the UI looks correct. These become the reference images.
- Run comparisons — on each subsequent test run, take new screenshots and diff them against the baseline pixel-by-pixel.
- Flag differences — any diff above the configured threshold (e.g., 0.5% of pixels changed) is flagged as a potential visual regression.
VigilQA stores baselines in your repo's test artifacts, diffs on every run, and presents any flagged differences in the dashboard with a side-by-side comparison and the diff overlay highlighted.
The false positive problem
The biggest challenge with visual regression testing is noise. Dynamic content (timestamps, user names, ads, animations) changes between runs and generates constant false positives. Without managing this, the tests become impossible to maintain.
Mitigation strategies:
- Mask dynamic regions — tell VigilQA to ignore specific areas of the screenshot (date fields, avatars, live data widgets) during comparison
- Set appropriate thresholds — a 0.01% threshold catches tiny regressions but generates noise; 0.5–1% filters rendering artefacts while still catching real layout breaks
- Use component-level screenshots where possible rather than full-page — smaller scope means fewer dynamic elements
When baselines need updating
When you intentionally change the UI, the baseline becomes outdated and every run will flag a diff — not a bug, just an outdated reference. This is handled through the Approvals queue: the comparison shows the old baseline, the new screenshot, and the diff. Approve it to update the baseline; reject it to investigate whether the change was intentional.
The approval workflow means baseline updates are always deliberate and logged, rather than silently overwritten.