Test Plan Run Results in Testsigma

View and download reports from the Run Results page. The page presents results at the test case, test suite, and test machine level, and shows the count of passed and failed tests along with the reason for each failure. Drill into a single step to read its analysis, check its logs, and compare it against an earlier run.

Prerequisites

Before you begin, ensure that:

1. You have referred to the documentation on creating test plans.
2. A test plan has completed at least one run.

Steps to View Test Suite Results

1. From the left navigation bar, go to Run Results and click the test plan for which you want to check the results.
2. Results open at test suite level by default.
3. Expand a test suite to check the test case results inside it.
4. The parallel indicators on each row show how many test suites run in parallel, and how many test cases run in parallel within each suite.

Steps to View Test Case Results

1. On the Run Results page, select Test Cases from the dropdown menu.
2. The page displays all test cases from all test suites.
3. Hover over a failed test case to view a brief description of the reason for failure.
4. Click a test case to view its detailed results.
5. On the Test Case Results page, the step list appears on the left, headed by the step count and the number of failed steps. Select a step to open its details on the right. Drag the divider between the two panes to resize either side.
6. Review the step header, which shows the step number, its duration, its status, and the result message. On a failed step, the header shows the error message with a Read more link.
7. Click Affected tests to see what else depends on the element this step uses. The Affected Instances panel opens with counts across Test Cases, Step Groups, Test Suites, and Test Plans.
8. Click More details to open Test Case Overview, which shows the step breakdown across Passed, Failed, Not Executed, and Stopped, along with the run message and Machine Details.

Step Details Tabs

Each step presents its details across five tabs.

Below the two panels, the Analysis tab lists the element, element name, run type, action, start time, duration, step level timeout, plan level timeout, error code, error message, test data type, and test data for each side of the comparison. The Page source section at the bottom holds the captured HTML for each side as a downloadable file.

On a failed step, the tab opens with the error code for the failure, such as #NO_SUCH_ELEMENT.

Steps to View Test Machine Results

1. On the Run Results page, select Test Machines from the dropdown menu.

2. The page displays all test machines included in the test plan.

3. Click a test machine to expand and view the test suites it contains.

Test Runs in Run Results

1. From the Test Runs panel, select a different run to view its corresponding test run results.
2. Click Rerun in the top-right corner to rerun the test plan by selecting the appropriate rerun options.

Compare Runs

Compare two runs of the same test case side by side.

1. On the Test Case Results page, click Compare Runs.
2. Select a run from the list at the top of either panel. Each panel shows the run's duration, date, step count, and failed count, followed by every step with its own duration.

Comparison also spans the steps on either side of the failure, because the step that reports an error is often not the step that caused it.

Compare a Single Step

Compare one step against the same step in another run.

1. On the Analysis tab, in the Visual Evidence section, click Compare Steps. The step comparison overlay opens. The step under comparison appears in the breadcrumb.

   

2. From the View list, select a comparison mode.

   

   Mode	What it shows
   Compare with Baseline	The baseline run and the current run side by side
   Overlay Wipe	Both screenshots in one frame, with a divider to wipe between them
   Current Step Only	The current run's screenshot on its own
   Element Source	A diff of the captured page source across the two runs, alongside the locator value for each

   In Overlay Wipe, the capture recorded at authoring time and the capture taken during execution load into a single frame separated by a vertical divider. Drag the handle on the divider to wipe between them. Stacking the two makes a shifted control, a modal that did not close, or a reflowed layout visible at once, rather than something to find by comparing two panels.

   

3. Click Screens to compare screenshots only, or Screens + Details to show the step details beneath each screenshot.

   Screens + Details lists the run type, start time, duration, step level timeout, plan level timeout, error code, error message, element name, and locator for each run. Where a value was not captured, the field shows a dash.

   NOTE:

   Screens + Details is disabled in Overlay Wipe, because that mode renders a single combined frame rather than two panels.

4. To compare a different step, select a step from the Change Step list.

   

   Each entry shows the step number, the step name, its duration, and its status, including Not Executed for steps the run never reached.

5. Click the download icon to download the comparison artifacts.

Overview of Tests in Run Results

1. The Run Overview section of the Run Results page is dynamic and provides a summary of tests at all levels for the selected run.
2. This section displays the Last Run, Test Plan Settings, and Accessibility Test Overview.
3. Click View Runs to view all runs, More Settings to access additional settings, and View Report to check the Accessibility Testing Report.

Exporting the Test Reports

1. Click the Export icon and select the download format from the dropdown menu.

2. For PDF format, the Export PDF dialog appears. Select the appropriate options and click Export.

   

3. For JUnit format, the report is downloaded instantly as an XML file.
4. For XLSX format, a downloadable link of the report is sent to your email.

Available Shortcuts in Run Results

For Mac:

• Option + C: View results at the test case level
• Option + S: View results at the test suite level
• Option + M: View results at the test machine level
• Option + H: Open/close run history
• Option + F: Open filters
• Option + O: Open/close run overview

For Windows:

• Alt + C: View results at the test case level
• Alt + S: View results at the test suite level
• Alt + M: View results at the test machine level
• Alt + H: Open/close run history
• Alt + F: Open filters
• Alt + O: Open/close run overview

How Run Results Handle Renamed and Deleted Items

The Run Results page shows the values as they were when the run executed. Renaming or deleting a test plan, test suite, test case, environment, test machine, or test data profile doesn't change what an earlier run displays.