AI agents

Testsigma’s AI agents take on parts of the testing workflow that would otherwise be manual: writing test cases, building test data, working out why a step failed, and filing the bug.

Note

AI agents need Generative AI features turned on under Settings > Preferences.

Agent	What it does	Where you use it
Generator	Turns requirements, files, and prompts into test cases, then automates them against the live application	QI Home
Test Data Generator	Builds a test data profile for a data-driven test case	Test Case Settings
Analyzer	Explains why a step failed and suggests fixes	Run Results
Bug Reporter	Files the failure as a bug, with the analysis attached	The Analyzer overlay
Healer	Repairs an element locator mid-run, and records what it changed	Run Results

Note

The Coverage Planner and Optimizer agents are announced but not yet available.

Set up the AI agents

The agents read requirements from the tools your team already uses, and runs on a large language model. Both are configured before you generate anything: the integrations under Settings > Integrations, and the model under Settings > Gen AI Keys.

Integrations

Every integration follows the same shape. Go to Settings > Integrations, turn on the widget’s toggle, enter the credentials in the dialog, and click Save & Enable. What differs is which credentials each one needs, and where you get them.

Integration	Credentials	What it gives the agents
Jira	Account URL, user name, API key	User stories, epics, and issues as input, plus bug reporting back to Jira
Jira Server or Data Center	Server URL, and admin access to the instance	The same, for a self-hosted Jira
Figma	Team ID, API key	Design frames as input
Xray	Jira account URL, client ID, client secret	Xray tests, epics, and stories as input
qTest	Host URL, bearer token	qTest modules and test cases as input
GitHub	Personal access token, resource owner, and a webhook in your repository	Test case generation when a pull request is raised

Note

Xray needs the Jira integration active as well, since it reads through Jira.

Note

For Jira Server or Data Center, the Server URL takes the protocol and the domain or IP address, such as https://jira.yourcompany.com.

Get the Figma credentials

Team ID. Open Figma in a browser, select the team from the dropdown in the left navigation bar, and read the URL. In https://www.figma.com/files/team/{TEAM_ID}/your-team-name, the {TEAM_ID} segment is what you need.

Note

Without access to the team page, ask your Figma admin for the Team ID.

Personal access token. Click your profile icon, select Settings, go to Security > Personal access tokens, and click Generate new token. Name it, choose an expiration period, click Generate token, and click Copy this token.

Note

The token is shown once. Store it before closing the dialog.

Tip

A Figma file is organized as Project, then Design File, then Pages, then Sections, then Frames. The Generator selects at the frame level, so knowing the hierarchy makes picking the right frames faster.

Note

Take the qTest bearer token from the Download qTest Resources page in qTest, and copy it without the word Bearer.

Set up GitHub

GitHub takes more than a toggle, because Testsigma has to receive pull request events from your repository.

1. Go to Settings > Integrations in Testsigma and turn on the Github toggle. The dialog shows a Webhook URL and a Webhook Secret. Keep it open.

2. In GitHub, open your organization, go to Settings, and click Webhooks under Code, planning, and automation.

3. Click Add webhook, and fill it in:

   • Payload URL: the Webhook URL from Testsigma
   • Content type: application/json
   • Secret: the Webhook Secret from Testsigma
   • Select Let me select individual events, then select Pull requests

4. Click Add webhook.

5. Back in GitHub, click your profile picture, go to Settings > Developer settings > Personal access tokens > Fine-grained tokens, and click Generate new token.

6. Name the token, select your organization as the Resource owner, choose an expiration, select Public repositories or All repositories under repository access, and set the repository and organization permissions.

7. Click Generate token, confirm, and copy it.

8. Enter the Personal Access Token and the Resource Owner in the Testsigma dialog, and click Save & Enable.

Note

The personal access token is shown once. Copy it before leaving the page.

Bring your own LLM keys

By default the agents run on Testsigma’s models. BYOK points them at your own LLM account instead, which keeps prompts and data within your provider and puts the model choice and its cost under your control.

Four providers are supported: Azure OpenAI, Open AI, Gemini AI, and Vertex AI.

To add a key:

1. Go to Settings > Gen AI Keys and click Create New Key.

2. Enter a Key Name and an optional Description.

3. Select an AI Provider and enter the details it asks for.

4. Click Validate API Key.

5. Click Create.

The key appears in the Keys section.

Adding a key changes nothing on its own. Each Testsigma feature is pointed at a key and a model separately, so different features can run on different models.

In Feature Model Configuration, select the Key and the Model for each Feature. Those features then use the mapped model.

Generator

The Generator agent writes test cases from your requirements, then automates them against your application.

Add an input source

Go to QI Home, click Generate with AI, and select a source in the Generate Test Cases section. Add as many as you need before prompting.

Jira

1. Click Jira Requirements.
2. Select a project from the Jira Project dropdown in Add Jira Tickets.
3. Select Epic or Story under Issue Type. Selecting Epic lets you choose the stories under it; selecting Story lets you choose stories directly.
4. Click Save.

Note

For Salesforce test cases, Prefer API steps is selected by default in the prompt box Settings. Clear it to generate UI-based steps instead.

Figma

1. Click Figma Designs.
2. Select a Team, Project, Figma design file, Section, and Page in the Figma Designs dialog.
3. Click + Select Frames, select the frames you want, and click Save.

Tip

Clear Selection removes everything you have selected.

Note

A page with no sections lets you select frames directly. You can select up to 10 frames.

qTest

1. Click QTest.
2. Select a Project and a Module in the Qtest dialog.
3. Select the test cases to use as input.
4. Click Save.

Confluence

1. Click Confluence.
2. Select the Confluence Space holding your requirements.
3. Select the Pages to use as input.
4. Click Save.

Video

1. Click Video Recording.
2. Click Add Files in the Video Recording dialog.
3. Click Browse, select the video, and wait for the upload.
4. Click Save, then Save again.

Files

1. Click Files.
2. Click Add Files in the Files Upload dialog.
3. Click Browse and select the files holding the test information. Images and PDFs are supported.
4. Click Add Files, then Save.

Salesforce Flows

For a Salesforce application, the Flows and Workflows in your connected Salesforce instance are available as input alongside the sources above.

Note

The instance needs active Flows or Workflows, and has to be connected to Testsigma.

Live Recorder

The Live Recorder captures your own walkthrough of the application as context.

1. Click Live Recorder.
2. Click Start Recording in the Live Recorder dialog. A Chrome session opens.
3. Enter the URL to test and click Start Recording.
4. Select Entire screen or Window when asked what to share.
5. Perform the actions you want the agent to learn from.
6. Click Stop, review the recording, and click Continue.
7. Review it once more and click Continue. Testsigma reopens with the recording in the Live Recorder dialog.
8. Click Save.

Generate the test cases

1. Enter a prompt describing the test cases you want.

2. Leave Read existing test case library selected so the Generator builds around test cases you already have, or clear it to ignore them.

3. Click Generate with AI.

Note

Once Read existing test case library is on in a session, it cannot be turned off again. It can be turned on partway through a session that started without it.

The generated test cases carry manual steps. See Automate the generated test cases to turn them into automated ones.

What each application type needs

Every application type takes the same input sources: Jira, Figma, qTest, Confluence, video recordings, files, and the Live Recorder. Two have something extra:

• Salesforce adds the Flows and Workflows of a connected Salesforce instance as an input source. Its steps default to API rather than UI
• REST and SOAP APIs take a schema file instead of a requirement, through the separate flow below

Whichever you use, the integration for that input source has to be configured first, along with a project and an application of that type. See Set up the AI agents above.

Tip

On Android and iOS, Figma frames are usually the best starting point, since the design is the specification. On desktop, the generated test cases align with the application’s UI elements.

Generate API test cases from a schema

API generation does not run from QI Home. It reads a schema file instead of a requirement.

Turn on Generate test cases from Swagger schema under Settings > Preferences > Generative AI features first. REST schemas are .json; SOAP schemas are .wsdl or .xml.

1. Go to Create Tests > Test Cases.

2. Click QI in Test Case Explorer and select Generate Test Cases from API Schema.

   

3. Click Select file to import in Add API Schema and choose your schema file.

4. Clear any test cases you do not want. All are selected by default.

5. Review the steps in Test Steps, and the endpoint, body, and status verifications in Verification Details.

6. Click Save Test Cases.

Note

The Generator groups REST endpoints into test cases by their Swagger tags, so tag every endpoint before importing.

Automate the generated test cases

The generated test cases carry manual steps. Turn them into automated steps in a Testsigma Copilot session, in one of two ways:

• Agentic Learning explores the application to find the steps you are missing, and verifies each one live
• Run with Copilot executes the steps the Generator wrote, so you can debug them before saving

Save over an existing test case

The Generator checks the library while it generates. Where a test case already exists, it updates it rather than creating a second copy, and marks it with an Update tag.

1. Expand a subfolder and select the updated test case.

2. Click See What’s New to compare the previous steps against the newly generated ones, and Hide Difference to close the comparison.

3. Click Generate Automated Steps.

4. Click Save to Library. The Overwrite Test Case dialog opens.

5. Choose what happens to the existing test case:

   • Overwrite: replaces it with the new version.
   • Save as New: keeps both, saving the new version as a copy.
   • Link to original test case: lets you review the existing test case before saving.

Note

Read existing test case library is selected by default, which is what lets the Generator find the existing test case. Once it is on in a session it cannot be turned off again, though it can be turned on partway through.

Test Data Generator

The Test Data Generator builds a test data profile for a data-driven test case, instead of you entering the data set by set.

1. Go to Create Tests > Test Cases, open the test case, and go to Test Case Settings in the utility panel.

2. Click Test Data Profile, then Generate TDP with AI.

3. Check the fields in the Test Data Generation dialog and click Generate.

4. Click Add more rows for more data, or enter a prompt to change what is generated. A prompt asking for an Indian context, for instance, reshapes the whole data set.

5. Click Create and Replace when the data looks right.

Analyzer

The Analyzer explains a failed step: the error type, the root cause, the visual evidence captured during execution, and a set of suggestions. It needs a run containing a failed step, from a test plan or a dry run.

1. Go to Run Results and select the run with the failure.

2. Open the test case and select the failed step. Its ANALYZER VERDICT section already holds a short analysis of the failure. See Debug.

3. Click Analyze with Agent in the action bar at the bottom of the page. The Analyzer with QI panel opens and reads Analyzing step failure… while it runs.

The analysis returns 4 things:

• Error Type: the category of failure, such as ELEMENT_NOT_FOUND
• Root Cause: why the step failed, based on the error message and the captured evidence
• Visual Evidence: the screenshots At the time of Recording and At the time of Execution. Only the execution screenshot appears where no recording screenshot exists
• Suggestions: a numbered list of fixes you can apply to the step

Tip

To ask a follow-up question about the failure, enter it in Debug with QI and click Analyze.

Note

Responses are generated by AI, so review the analysis before acting on it. The feedback icons and Give Feedback report how useful it was.

Apply a fix

1. Select a suggestion in the Suggestions list. Apply Fix stays disabled until you select one.

2. Click Apply Fix. The agent returns an Add/Update Step card.

3. Click Add/Update Step. The updated step appears in the step list as an extra row below the original steps, marked with the Analyzer icon.

4. Rerun the test to confirm the fix. For a test plan run, click the run number in the breadcrumb to return to Run Results, then click Rerun. For a dry run, click Re-Run in the page header.

The current result still shows the original step as failed, because the change applies from the next run.

Bug Reporter

The Bug Reporter files a failure as a bug from the Analyzer panel, carrying the error type, root cause, suggested fixes, and screenshots into the ticket. That removes the step where someone reproduces the failure manually to describe it.

This needs a bug tracking tool integrated with Testsigma, and a step the Analyzer has already reviewed.

1. Click Report Bug in the Analyzer with QI panel. The QA Agent panel opens.

2. Select your bug tracking tool from the dropdown at the top right.

3. Take either route:

   • Create New: review the prefilled details and click Report Bug
   • Link To Issue: search for the existing issue and click Link To Ticket

Note

The fields shown depend on the tool.

If the tool returns no projects, check that its configured credentials have access to the projects you expect. For Jira the panel reads For given JIRA Credentials, the projects list is empty. Please make sure it has the right access to required projects on JIRA.

Healer

The Healer repairs an element locator during a run, instead of failing the step. Where the UI changed and the stored locator no longer matches, it identifies the element again and carries on. The run records what it changed, so you decide afterwards whether the new locator is the one the test should keep.

This needs Auto Healing turned on under Settings > Preferences, and the element itself left opted in. See Account preferences and Elements.

Find a healed step

When a saved locator fails during execution, Testsigma searches for the element and continues the run with a new locator. The step’s status reads Healed, and the new locator is kept as a pending heal until you approve it.

On the Test Case Results page, select a step that shows the auto-heal icon. A banner above the step list shows how many steps were healed, with thumbs up and thumbs down icons to rate the heal.

Approve the heal

The Healed this step card below the step header shows the Locator change:

• Previous · recorded: the saved locator that failed, struck through
• Current · applied: the locator Testsigma used to continue the run

The count at the bottom of the card shows how many linked artifacts use the element.

1. Click Approve as Primary. The Autoheal Details panel opens, naming the element and the locator type used for the heal, such as Autohealed using CSS Selector, with the failed locator struck through above the new one.

2. Click Update. The Update Element dialog opens.

3. Review the test cases under Linked to Test Cases. The open icon next to a test case opens it in a new tab.

4. Click Update to update the element in all linked test cases.

The Healed this step card then shows Approved.

A pending heal is not the element’s primary locator. Testsigma tries it before searching again on later runs, but the primary locator changes only when you click Approve as Primary.

Read the locator trace

Click Locators tried on the Healed this step card to see how Testsigma found the element. The panel header shows the step number, the number of stages, and how the element was found. The stages run in this order:

Stage	What it does
Primary	Tries the primary locator saved on the element. Every step starts here
Last Working	Tries the locator that found the element in the previous run. It is kept on the element as a pending heal, and is not part of the backup pool
Backup Pool	Makes a single pass over the live screen to see which stored backup locators still resolve. No action runs on the page at this stage
Relearn Screen	Reads the screen as the app rendered it and writes new locators for the element. The result is kept as a pending heal and tried first on the next run

Each stage shows its outcome, such as Did not match, Skipped, or Located. Each locator in a stage shows its result, such as NOT FOUND, AGREED, or NEW, its source and type, such as smart-recorder · csspath, its role, such as PRIMARY, LAST WORKING, ANCHOR, or CANDIDATE, and its quality, Good or Fragile.

When a heal fails

When auto-healing cannot find the element, the step fails. Select it and read the ERROR and ANALYZER VERDICT sections to see why the element could not be resolved. See Debug.

Fix the locator yourself

Two routes, where you would rather not accept the healed locator:

• Update element corrects the locator directly
• Relearn step re-captures the step, so a fresh locator is recorded

Frequently asked questions

Why do my Figma pages fail to load?

Figma’s API rate limits have been reached, so it stops returning file and page data to external tools, and Testsigma shows No pages. Lower-tier seats hit those limits sooner.

Check that the API key belongs to a Figma account with a Dev or Full seat, which carries higher limits. Generate a new key from such an account if the current one does not. If the limit was already exceeded, wait a few minutes before trying again.