Visual testing

Visual testing compares a screenshot taken during a run against a stored baseline, so a layout change shows up as a failed comparison rather than going unnoticed. It is off by default and switched on per step.

Note

A step with visual testing enabled needs a baseline image before it can compare anything. That baseline comes from the next successful run, so visual testing produces results from the second run onward.

Enable visual testing on a step

Click the ellipsis icon ⋮ on the step, select Enable Visual Testing, and click Update Step. This works on the Test Case Details page and in the test step recorder.

To enable it on several steps at once, select their checkboxes, click Update Settings in the menu bar, select Enable Visual Testing in the Bulk Update Settings overlay, and click Update.

Run the test case once to create the baselines, then run it again to see comparisons.

Read a comparison

The Test Case Result page marks each visual step with a camera icon. Green means the current image matches the baseline. Red means it does not.

Click the icon to open Visual Difference. To reach it from a past run, click Ad-hoc Runs in the utility panel, click View Details, then click the camera icon on the step.

The overlay offers:

• Highlight Differences: marks the variations
• Hide Visual Differences: conceals them
• Select region to ignore from visual comparison: excludes areas from the comparison
• Merge base image and current image: overlays the two
• Zoom in and Zoom out

Tip

AI Insights produces a UI Difference Analysis Report whenever a difference is found, naming the likely cause and suggesting a fix. The insights can be copied for debugging.

Update the baseline

A deliberate UI change makes every baseline for that screen wrong, so replace it rather than letting the comparison fail.

In the Visual Difference overlay, select Mark as base image on the current image. Later runs then compare against the new baseline.

Fail the test case on a difference

A visual difference does not fail a test case by itself. To make it fail, open Test Case Settings in the utility panel and turn on Fail Test Case if Visual Testing Fails.

Ignore parts of the screen

Dynamic content produces differences that are not defects, such as advertisements, timestamps, and user-specific data. Excluding those regions cuts the false positives, so only meaningful changes are flagged.

Open Visual Difference on the step, select Select region to ignore from visual comparison, choose the areas to exclude, and click Save ignore region.

Choose a comparison mode

Two modes are available, and both compare pixel by pixel. Strict requires an exact match. Lenient takes a threshold, so a threshold of 70% passes when 70% of pixels are identical and ignores the rest.

Open Visual Difference on the step, expand Visual Testing Mode, and select Strict or Lenient. For Lenient, enter the percentage in Threshold.

Note

To apply Lenient mode to one area rather than the whole image, click Select region to highlight for visual comparison, expand the option, and choose Select selection.

A mode can be combined with the region features: Select Region narrows what is compared, and Ignore Region excludes areas from it.

Compare against a Figma design

Instead of comparing a screenshot to an earlier screenshot, you can compare it to the design itself, using a Figma frame as the baseline.

Note

This works on web and mobile applications, and is limited to enterprise accounts. Contact Testsigma support to have it enabled.

1. Go to Create Tests > Test Cases and open the test case.

2. Click the ellipsis icon ⋮ next to the step, select Enable Visual Testing, and click Update Step.

3. Hover over the step and click the camera icon. Visual Difference - Figma opens.

4. Click Add Figma Designs in the Base Line Image section.

5. Select the Browser and Resolution in Add Devices. For Android and iOS, select a mobile device with the resolution you need.

6. Enter the Figma frame link in Frame Link and click Refresh to verify it.

7. Click Save.

8. Run the test case, then click the camera icon on the step result and open the Visual Difference - Figma tab.

Note

The browser and resolution set here have to match the ones selected for the run, and the Figma frame dimensions have to match the target device resolution. A mismatch produces differences that are not real.