Skip to content
Docs
Arcus
Popular questions
↑↓ to navigate↵ to selectesc to close
Book a demo

Test runs

A test run is a set of test cases you execute together, with a schedule, an assignee, and a status for every case. Create one for each round of manual testing, then record a result for each case as you go.

PropertyTest runRun template
Created fromCreate > New test runCreate > New run template
Used forOne round of testing, until you close itRecurring test plans clone a fresh run from it each cycle, or you start a run from it yourself
ExecutionManualManual or Automated
ScheduleStart and end datesNone. Runs created from it are scheduled separately
StatusesOne per test caseNone. Results live on the runs it creates
Listed underActive, then ClosedRun Templates

Open Test Runs in the left navigation. The list has three tabs: Active, Closed, and Run Templates. Each row shows the run’s ID, title, run configuration, the number of plans it belongs to, the number of tests, its schedule, its progress, and its owner.

The Test Runs list with the Active, Closed, and Run Templates tabs, and the Create button outlined.

To narrow the list, click Filter by, or press ⌘+Shift+F. You can filter by Created at, Start date, End date, Assignee, Label, Run configuration, Execution Type (Manual or Automated), and Run Kind (One-time or Cycle run). To find a run by name, enter it in Search runs, or press ⌘+F. To reorder the list, click Sort and sort by creation date, name, end date, or progress.

  1. On the Test Runs page, click the arrow next to Create and select New test run.

    The Create menu with New test run outlined, above New run template.

  2. Enter a title. It starts as “Test Run - ” followed by today’s date. A description is optional.

  3. In Select Test Cases, select Pick by hand or Match a filter:

    • Pick by hand: the run holds exactly the cases you tick, and nothing joins it later.
    • Match a filter: the run saves a rule, and cases that start matching it keep joining the run until you close it.
  4. Add the test cases:

    • For Pick by hand, tick the cases in the list, or click Select all.
    • For Match a filter, click Filter by and add at least one filter. You can filter by Folder, Priority, Automation Type, Test Case Type, Label, Assignee, Created at, and Reviewer. The list shows how many cases match.

    The New test run page with 4 cases ticked, the Create Test Run button outlined, and the Test Run Properties panel.

    The Add Test Cases section for Match a filter with Priority, Automation Type, and Folder filters outlined, 16 cases matching the rule, and Match a filter selected in Select Test Cases.

  5. (Optional) In Assign Run, choose the team member responsible for the run. It’s set to you by default.

  6. (Optional) In Schedule Run, set the start and end dates. They default to today and 7 days from today.

  7. (Optional) In Run Configuration, click + and tick one or more run configurations.

    The Run Configuration picker listing the 5 default configurations, with Create run configuration, Open run configurations, and the note that a manual run doesn't split results by device.

  8. (Optional) In Attach Test Plan, click + and select a one-time plan. In Labels, click + and select or create labels.

  9. Click Create Test Run, or press ⌘+↵.

Arcus shows “Test run created” and opens the run. Every case starts as UnTested. For a Match a filter run, the run page shows “These test cases are added using dynamic filtering.”

If you leave the page before creating the run, the Unsaved Test Run Changes dialog asks whether to Discard Changes or Save & Continue.

To change which columns the case list shows while you pick, click View and show or hide Labels, Test Case Type, Test Case Status, Automation Type, and Owner.

A manual run with several run configurations keeps one set of results. It records where the tester worked and doesn’t split results by device. To get a separate run for each configuration instead, pick at least two configurations and turn on One run per configuration before you click Create Test Run.

The Run Configuration field with Mac Chrome and Mac Safari picked, and One run per configuration turned on and outlined with its hint.

Arcus creates one run per configuration, adds the configuration’s name to each run’s title, and shows how many it created. The split happens once, when the runs are created.

The Test Run Properties panel holds the run’s settings:

  • Execution: Manual for every test run. Automated runs come from a run template during a test plan run.
  • Usage Type: One-time.
  • Select Test Cases: Pick by hand or Match a filter.
  • Assign Run: the team member responsible for the run.
  • Schedule Run: the start and end dates, and how many days are left or overdue.
  • Run Configuration: the configurations the run is meant for.
  • Attach Test Plan: the one-time plan the run belongs to.
  • Labels: tags for filtering runs.

On the run page, the panel is read-only. To change a property, edit the run.

Work through the run’s test cases and record a status for each one. The run’s progress updates with every status you set. When every case has a result, close the run.

On the Test Runs page, click the run’s name. The run page shows its folders on the left, its test cases in the middle, and Test run properties on the right.

A test run page with 4 test cases marked UnTested, and the Test run properties panel showing 0% pass rate and the status counts.

PROGRESS at the top of the panel shows the pass rate, the number of test cases, and how many cases have each status.

To find a case, enter its name or ID in Search by cases or ID. To narrow the list, click Filter by and filter by Folder, Status, Priority, Automation Type, Test Case Type, Label, Owner, or Created at. To reorder it, click Sort and select Order, Title, or Priority, then Default or Reverse.

  1. Click the arrow next to the test case’s status.

  2. Select Blocked, UnTested, In Progress, Retest, Failed, Skipped, or Passed.

    A test case's Status menu listing Blocked, UnTested, In Progress, Retest, Failed, Skipped, and Passed, with Add a comment and Add a Defect.

The status and PROGRESS update right away. To assign the case to someone, click the person icon in its row and select a team member.

The status menu also has Add a comment and Add a Defect. See Add a comment or a file to a test case and Report a defect.

Click a test case’s name to open it in a panel beside the list. The panel has the tabs Steps, Automated, Properties, Update, and Activity. To move to the previous or next case, press J or K.

  1. In the test case panel, click Update, or click the comment icon at the top of the panel.

  2. (Optional) Select a status above the comment box. The case’s status changes as soon as you select it.

  3. Enter your notes in the comment box.

  4. (Optional) To attach a screenshot or a log, click Upload a file.

  5. Click Add Comment.

    The Update tab of a test case with Retest selected, a note entered, and the Add Comment button outlined.

Arcus shows “Comment added successfully.” The case’s Activity tab lists the status change and the comment together, with who made them and when.

To leave a note on the whole run rather than one case:

  1. Scroll to Activity at the bottom of Test run properties.

  2. Enter the comment. Type @ to mention someone, or / for text functions.

  3. Click Add Comment.

You can reply to a run comment or react to it.

  1. Hover over a test case and tick the checkbox beside it. Tick more cases, or click the check mark in the bar to select them all.

    The bulk action bar with 3 Selected, Assign Test Case To, Change Status, Execute and Capture, and Remove From run.

  2. In the bar at the bottom of the page, click Change Status and select the status.

  3. Enter a note for the change. Add Comment stays unavailable until you do.

  4. Click Add Comment.

Arcus shows how many test cases changed and adds the note to each case. To assign the selected cases instead, click Assign Test Case To and select a team member. To clear the selection, click the × in the bar.

  1. Tick the test cases to remove.

  2. In the bar at the bottom of the page, click Remove From run.

  3. In the dialog, click Remove.

To attach a replay of what you saw to the result, click Capture in the run’s header, or tick test cases and click Execute and Capture in the bar at the bottom of the page. Both need the Test Capture extension for Chrome. See Test Capture.

Open the test case and click its Run history tab. The tab lists every test run the case belongs to, with the run’s ID and title, its kind and execution type, the date, and the case’s status in that run. Each entry also records who set the status and any note added with it, shown as the failure reason.

  1. On the Test Runs page, click the ellipsis icon (⋯) next to the run and click Edit Test Run. You can also open the run and click the pencil icon.

    The ellipsis menu of an active test run with Edit Test Run, Add to Plan, Clone Run, Close Run, and Delete.

  2. Change the title, description, test cases, or properties. For a Match a filter run, change the filters of the rule. The rule needs at least one filter.

  3. Click Save Changes, or press ⌘+↵.

Arcus shows “Run updated”. You can’t change Usage Type or Execution after the run is created.

To set a run configuration without opening the editor, click Add configuration in the run’s row on the Test Runs page and tick the configuration. The change saves straight away.

  1. On the Test Runs page, click the ellipsis icon (⋯) next to the run and click Add to Plan.

  2. Under ACTIVE ONE-TIME PLANS, tick the plan.

  3. Click Link.

A one-time run links to one active one-time plan. You can also set the plan in Attach Test Plan when you edit the run. For more on plans, see Test Plans.

  1. On the Test Runs page, click the ellipsis icon (⋯) next to the run and click Clone Run. You can also open the run, click the ellipsis icon (⋯) in its header, and click Clone Run.

  2. In the Clone Test Run dialog, change the title if you want. It defaults to the original title followed by “(Clone)”.

  3. Click Clone.

Arcus shows “Test Run Cloned” and opens the clone. The clone has the same test cases and assignee. Every case starts again as UnTested, and the schedule is cleared. You can clone a closed run too.

Close a run when testing is finished. Its results become final: statuses can’t change, and the run moves to the Closed tab.

  1. On the Test Runs page, click the ellipsis icon (⋯) next to the run and click Close Run. You can also open the run and click Close Run in its header.

  2. In the Close this run? dialog, click Close Run.

Arcus shows “The Test Run has been closed.” The run page then shows Closed and Reopen Run in its header. A closed run can still be added to a one-time plan, to give the plan a status overview.

A closed test run with Closed and the outlined Reopen Run in the header, and the note that results are final.

To reopen a run, open it and click Reopen Run, or click the ellipsis icon (⋯) next to it on the Closed tab and click Reopen Run. Arcus shows “The Test Run has been reopened.” and the run moves back to Active.

  1. On the Test Runs page, click the ellipsis icon (⋯) next to the run and click Delete. You can also open the run, click the ellipsis icon (⋯) in its header, and click Delete.

  2. In the Delete this run? dialog, click Delete.

If the run belongs to a test plan, Arcus doesn’t delete it. A Deletion Warning lists the plan under Linked Entities. Remove the run from the plan first, then delete it.

The Deletion Warning dialog listing the test plan the run is linked to under Linked Entities.

A public link lets people without an Arcus account see the run’s progress. They see the test cases, their statuses, and the run’s properties, and they can filter, search, and sort the list. They can’t change anything.

  1. Open the test run and click the share icon in its header. You can also click the share icon next to the run on the Test Runs page.

  2. In Share Test Run, turn on Anyone with the link can view. Arcus creates the link.

  3. Click Copy.

    The Share Test Run popover with Anyone with the link can view turned on, the link, and the Copy button outlined.

To stop sharing, turn off Anyone with the link can view. Copied links stop working. Turn it on again to restore the same link.

A run configuration saves where and how a test executes: web or mobile, a local agent or the cloud, and the operating system, browser, or device. Test runs, run templates, and test plans share the same configurations. To add, edit, or delete one, see Run Configurations.

A run template is a reusable test run. Recurring test plans clone a fresh run from it each cycle, and you can start a run from it yourself.

  1. On the Test Runs page, click the arrow next to Create and select New run template.

  2. Enter a title and add the test cases, as for a test run.

  3. In Execution, select Manual or Automated. For Automated, the list shows only automated test cases.

    The New run template page with 4 cases ticked, Execution set to Manual, the note that Schedule Run is not available on a template, and Create Run Template outlined.

  4. (Optional) Set Assign Run, Run Configuration, and Labels. In Attach Test Plan, you can pick only recurring test plans. Changes to a plan link take effect from the next cycle.

  5. Click Create Run Template.

Arcus shows “Run template created” and opens the template. It appears under Run Templates. An automated template needs a run configuration before a cycle can start.

Open the template and click Start as Active Run, or click the ellipsis icon (⋯) next to it on the Run Templates tab and click Start as Active Run. Arcus creates a one-time test run with the template’s test cases, named after the template with today’s date, and opens it. The new run has no schedule until you edit it.

A run template page with its 4 test cases, the TEMPLATE panel and its note, Usage Type set to Recurring, and the Start as Active Run button outlined.

From the same menu you can also Edit Run Template, Link to Test Plan, Clone Run Template, and Delete the template.

Archive a template to stop using it without deleting it.

  1. On the Run Templates tab, click the ellipsis icon (⋯) next to the template and click Archive Run Template.

  2. In the Archive this run template? dialog, click Archive.

Arcus unlinks the template from every plan and hides it from the tab. Runs already created from it keep their results. An archived template can’t start a run.

To restore it, click Filter by > Archived > Archived only, click the ellipsis icon (⋯) next to the template, and click Restore Run Template. Link it to its plans again afterwards, because restoring doesn’t relink them.

Test Capture is a Chrome extension that records what you see while you execute a test case and attaches the evidence to the case’s result in the run: a replay of the last 2 minutes, the console output, the network requests, and the browser details. You open the evidence later from the case’s Activity tab.

Test Capture works in Chrome only. Capture isn’t available on a closed run.

  1. Open a test run and click Capture in its header.

  2. In the Extension Not Installed dialog, click Download Extension. Chrome opens the Test Capture page on the Chrome Web Store.

  3. Click Add to Chrome and confirm.

  4. Sign in to the extension. On the Arcus page that opens, check the account under Logged in as and click Authorize Extension. Arcus shows Authorization successful!, and the tab closes on its own a few seconds later.

  1. Open the run and click Capture, or tick test cases and click Execute and Capture in the bar at the bottom of the page. The extension opens the run’s test steps, so you can keep them in view without switching tabs.

  2. Work through the steps in your application.

  3. When you spot a bug, attach the last 2 minutes of video and the technical logs from the extension.

Open the test case in the run and click its Activity tab. An entry recorded with Test Capture shows the status change, the comment, and Attachments with the comment with an Instant Replay link.

The Activity tab of a test case in a run, with two Failed status changes, their comments, and the Instant Replay attachment outlined.

Click Instant Replay. The Test Capture page opens in a new tab:

  • Replay: the recording, with playback controls, the speed, and the elapsed time.
  • Issue Details: how many Functional, Accessibility, and Visual issues the capture found.
  • Activity: the status change and the note recorded with the capture.
  • Summary: a written summary of what happened, “Based on replayed interactions and captured logs”, with each issue listed under its group. Expand an issue for its details.
  • Info: the URL, Browser, OS, Screen, Viewport, Timezone, and when the evidence was Captured.
  • Console: the browser console output.
  • Network: each request’s Name, Status, Method, Type, Size, and Duration. Filter by JS, Fetch/XHR, or Images, or by name.

The Test Capture page for a failed login, with the replay, Issue Details, and the Summary tab, and the Summary, Info, Console, and Network tabs and the share icon outlined.

To share the capture with someone without an Arcus account, click the share icon and, under Share Logs, turn on Anyone with the link can view. The link appears below the toggle. Click Copy to copy it.

A defect is an issue in your issue tracker that’s linked to a test case in a run. You create the issue from the case or link one that already exists, and the case lists it under Reported Defects.

Arcus logs defects in the trackers connected under Settings > Integrations: Jira, Linear, ClickUp, Azure DevOps, and OpenProject. The Report a defect sheet is the same for every tracker. Only the fields that say where the issue goes, and the tracker’s own fields, change with the tracker you pick.

To report a defect from an automated run, with the failed step and its screenshot filled in, see Report a defect from a run.

  1. Click the arrow next to the test case’s status and click Add a Defect. You can also open the test case and click the defect icon at the top of its panel, or click Add a Defect on its Update tab. The Report a defect against sheet opens, named after the test case, on the Create new tab.

  2. In Tracker, select the tracker. The list shows only connected trackers. The sheet starts on the tracker and destination you used last, even in another Arcus project.

  3. Select where the issue goes. The fields depend on the tracker, such as Project and Issue type for Jira, or Project and Work item type for Azure DevOps.

  4. Check the Title and the Description. Arcus writes both from the test case while the sheet loads, and marks the description “Written by Atto from this run. Edit anything.” The description has the case’s preconditions, steps to reproduce, expected result, and actual result. The steps to reproduce include the steps of any step group in the case. When you’ve commented on the case in this run, Arcus uses your comment for the actual result, so add a comment that says what you saw before you report the defect. Change anything that doesn’t match.

  5. (Optional) Set the Priority and Assigned to. Jira shows Assignee and Reporter instead.

  6. (Optional) Click Upload a file next to Attachments to attach a screenshot or a log.

  7. (Optional) Fill in the tracker’s other fields under More {tracker} fields, such as Area path and Tags for Azure DevOps, or Sprint and Due date for Jira.

  8. Click Create defect, or press ⌘+↵. To log another defect for the same case straight after, select Create more first. The sheet stays open.

    The Report a defect sheet on the Create new tab, with Azure DevOps as the Tracker, the Project, Work item type, and Title filled in, the Description written from the test case and the tester's comment, More Azure DevOps fields, and Create defect outlined

Arcus creates the issue in the tracker, shows “Created a … in {tracker}. It is linked to this case in this run.”, and lists it under the case’s Reported Defects.

If you click Create defect while a required field is empty, the footer shows how many still need a value, for example “2 fields need a value”, and the defect isn’t created until you fill them.

If the issue already exists in the tracker, link it instead of creating a second one.

  1. Open the Report a defect against sheet, as above, and click Link existing.

  2. In Tracker, select the tracker.

  3. (Optional) Narrow the list with the tracker’s filters, such as Project and Type for Azure DevOps or Project and Assignee for Jira, or enter an ID or a title in Search by id or title. The list is headed EVERYTHING IN and the tracker project’s name.

    The Link existing tab for Azure DevOps, with the tab, Type, Search by id or title, and Link defect outlined

  4. Tick one or more issues in the list.

  5. Click Link defect, or press ⌘+↵.

Each linked issue appears under the case’s Reported Defects.

When the test case already has open defects in the selected tracker from other runs, the Create new tab shows up to three of them above the fields, each with the run it came from. Click Link instead to open Link existing with that defect selected, then click Link defect. On Link existing, these defects are listed under Likely matches · this case in other runs, and a defect already linked to the case in this run reads Already linked to this case and can’t be selected again.

Open the test case in the run, click its Activity tab, then click Reported Defects. Until a defect is logged, the tab reads “No Defects reported yet.” Each defect shows its tracker ID and status, its title, where it’s filed, who reported it and when, and the actual result. Click Show steps and expected result for the rest, or the open icon to view it in the tracker. You can’t unlink a defect here.

Was this page helpful?