Regression Testing vs Retesting-Differences and Examples

Regression Testing vs Retesting-Differences and Examples

| December 28, 2019

During the multiple phases of software development life cycle(SDLC), multiple kinds of testing are performed depending on the stage of development.

Each stage of the the SDLC has different requirements and different objectives for the testing to fulfil. The testing phase starts with unit testing followed by integration testing, system testing, system integration testing, acceptance testing, and regression testing

During each phase of testing, if something does not work as per the requirements, the defects detected by the testers are raised and assigned to the developers to fix. Once defects are fixed, they are retested by testers to assure it works as desired. This is called retesting.

But isn’t it same as regression testing? They are not. Let us discuss regression testing vs retesting in detail in the article.

Regression Testing

Regression testing is a type of black-box testing. It ensures that any modification or addition to the existing code base has not adversely impacted the previously developed and tested features.

Regression testing confirms if the application is working as desired and expected after the change. Smoke and Sanity testing are often considered types of regression testing.

Both functional and non-functional tests are executed while performing regression testing. Regression testing is never performed on a new module.

A quick explanation of the different types of testing are discussed here.

What is the Objective of Regression Testing?

The objective of regression testing is to check that the new code changes do not negatively impact the existing developed and tested functionalities of the application.

When is Regression Testing performed?

Regression testing is carried out in any of the below circumstances:

  1. When a change request (CR) is raised by the client, which leads to change in the code-base.
  2. When there is a change in the testing environment.
  3. When back-end code is migrated to a different platform.
  4. Critical bug(s) is found during the testing phase and is fixed by the developer.
  5. A new feature is added to an existing application.
  6. The major concerns about performance issues and crashes are fixed by the developers.
  7. Patch fixes are added.
  8. The UI of the application was changed for better user experience.

Example of Regression Testing in Agile Framework:

The below diagram illustrates the role of regression testing in the final delivery of the product in an Agile framework:

Sprint is a set period of time during which specific work has to be completed and made ready to be shipped.

Sprints_Testsigma

In Sprint X, Module 1 is planned to be completed.

During this phase, the developer develops Module 1. The tester performs unit testing on the module, raises any defects found and retests the fixed defects.

After the issues are fixed and tested, it is deployed to a production-like environment.

In Sprint Y, Module 2 is planned to be completed and both modules are merged.

The development of module 2 is completed and both the modules are integrated. Later, different types of testing are performed to ensure developed product matches defined specifications.

  1. Unit testing- Module 2 is working as desired.
  2. Integration testing- performed to check data transfer flow between modules.
  3. Regression testing- performed to check the integration of both modules has not led to code break.

Finally, the product is deployed to a production-like environment.

In Sprint Z, the final product is delivered.

Development of Module 3 is done, the change requested is incorporated and all three modules are integrated.

During this phase, various types of testing are performed to guarantee the developed product works as desired.

  1. Unit testing- Module 3 is working as expected.
  2. Integration testing- performed to ensure the transfer of data between all three modules works fine.
  3. System testing- checks end to end flows works as expected.
  4. Acceptance testing- to validate the system is developed as per business requirement.
  5. Regression testing- performed to check the integration of all modules and modifications or enhancements in Module 1 has not led to code break. 

Finally, the product is deployed to production and is ready to be used by the customers.

For more details on the Agile framework, please refer below link:
https://www.scrum.org/resources/blog/typical-sprint-play-play

Regression in itself is a wide topic and has multiple sub-types. For an indepth study on it, refer the guide here: Regression Testing

Regression testing is also an ideal candidate for automation. If your team is spending a lot of time executing the same regression test cases multiple times, then this could be the time when you may have to start automating them. This article can help you know for sure: 5 signs you need to implement test automation

What is Retesting? 

Retesting is also known as confirmation testing. Retesting is to ensure that the defects raised during the SDLC (software develolpment life cycle) are fixed and work according to the specifications. During retesting, the failed test cases are re-executed and passed. 

What is the Objective of Retesting?

The objective of retesting is to check if the identified defects are fixed. To verify the fix, the test cases linked to the defect are re-executed and passed.

When is Retesting Performed?

  1. Whenever a defect in the software is fixed, retesting needs to be carried out. 
  2. Retesting has to be carried out prior to regression testing.
  3. The tester only performs retesting when a defect raised by them changes its status’ to ‘awaiting retest’. 

Example of Retesting

Below given diagram presents the process of retesting :

Step 1:  Module 1 has been developed to produce output as A+B+2.

Step 2: The module is tested to check if the functionality works as expected. The output produced is A+B instead of A+B+2. The associated test case failed and a defect is raised.

Step 3: The developer fixes the defect by making necessary changes in code. Then defect is reassigned to the tester to retest and verify.

Step 4: The defect is retested and output is matched with the expected result. The linked failed test case is re-run and passed. Finally, the defect is closed.

Testing Techniques used in Regression Testing

1. Prioritize test cases

The tester should be wise enough to trace out which area would be potentially impacted due to changes in code or additional features. While creating test suites, test cases should be tagged on the basis of priority (high, medium, low). 

Based on the time-frame and the business risk involved, testers can take a call on how much regression is required. Whether running the high priority test cases to ensure end-to-end coverage would be sufficient to find any code break or is it required to run the entire test suite to ensure there is no failure. High priority test cases in the regression package should include newly added features/functionalities and customer-facing aspects.

2.  Re-execute all

This technique is usually applied when the entire code-base is migrated to a different language or there is a major update in the back-end system. As the name implies, in this practice you are required to re-run the entire test suite. As this activity is time-consuming and requires substantial resources, this is taken up when the entire test suite is automated. 

3. Regression test selection

When the changes and their associated impact is not major, it is advisable to make a test suite that covers relevant areas impacted due to change instead of re-executing the entire existing test suite. You should perform regression testing in those selected areas only.

The Testing Technique used in Retesting

There is no specific technique used in retesting when defects get fixed, they are retested and corresponding cases are passed.

Why should the Regression Test Suite be Automated? 

Regression testing is performed on existing functionalities that have been tested before. After every release, the test cases for features added in that release are added to the regression test suite for the next release. Thus, number of test cases included in the regression test suite increases by a big amount with each passing release phase. These regression test cases are good automation candidates usually because by the time the release happens, the application becomes stable and testers are well versed with functionalities.

When the regression test suite is automated, it gives testers ample amount of time to perform ad-hoc and exploratory testing. This prevents defect leakage for scenarios that might be missed or are not a member of the regression test suite. 

The automation package should be designed in such a way that scripts are simple and easy to maintain.

There are many challenges faced while automating the regression test suite such as data challenges, frequent changes in the application, etc.

In-Sprint Test Automation Challenges for Regression Testing discusses some challenges in implementing In-Sprint Test Automation to perform Regression Testing.

Testing Technique used in Retesting 

There is no specific technique involved in retesting, you need to verify the fix by retesting the defect and re-execute the linked failed cases to it. Automation of test cases is not required. 

Test Strategy used in Regression Testing

The regression activity should be well planned and proper test strategy should be followed.

The regression test plan must include below pointers:

  1. Purpose of testing
  2. Test strategy 
  3. Features to be tested
  4. Resource Requirement
    1. Hardware Requirement 
    2. Software Requirement
  5. Test schedule 
  6. Change request 
  7. Entry / Exit criteria 
    1. Entry criteria for when to start  testing
    2. Exit criteria for when to stop testing
  8. Tools

Let’s talk about all of them in detail:

  1. Purpose: Describes how testing would be performed to accomplish the desired results.
  2. Test strategy: Consists of the approach, techniques that would be used while performing testing. 
  3. i. What will be completion criteria, 

    ii. Who will write test scripts, 

    iii. who will perform testing, 

    iv. Which tool can be used, 

    v. What will be the course of action, if product risk occurs like resource crunch, delay in production, etc.

  4. Features to be tested: The features which will be members of regression are listed here.
  5. Resource Requirement:
    1. Hardware Requirements: Any hardware requirements which will be required during testing are identified, such as laptops, desktops, smartphones, etc.
    2. Software requirement: Which operating system, browsers will be required.
  6. Test schedule: Number of resources required to complete the testing activity within the estimated time.
  7. Change Request: What changes or new features are added to the existing application, for which regression needs to be performed.
  8. Entry / Exit criteria:
    1. Entry criteria: What all basic needs should be fulfilled to start testing such as environment, test data, etc.
    2. Exit criteria:It is always dependent on the risk involved. When to stop testing, how much testing is enough, all such pointers are described here.
  9. Tools: The section lists tools that can be used to make the job easier. As regression testing is re-execution of passed cases so mostly it is automated with the help of tools.

Regression Testing vs Retesting Differences

As we see, Regression Testing and Retesting are not the same.

Top 5 Best Regression Test Automation Tools

Tools are used to make the job easier and simpler. Some of the widely used regression test automation tool available in the market are:

1. Testsigma

It is the finest automated regression testing tool available in the market to perform automated regression testing. It allows the user to write test scripts in normal English language which is as easy as writing test steps for manual testing.

To ease maintenance, Testsigma has built in AI/ML that identifies affected and relevant tests and suggests changes accordingly. It also self-heals the failures that happen because of minor changes in website code.

Key feature: The test cases for Web, Mobile and API – all can be automated in one place and executed in the cloud on the pre-integrated Testlab. Testsigma also has built-in support for cross browser testing. The results for the test runs can be accessed via customizable reports too. The tool provides options to prioritize test cases, build multiple test suites, schedule and run automated regression tests for each build automatically.

2. TestComplete 

It enables you to have quick, easy creation and maintenance of regression tests across the web, mobile and desktop applications.

Key Features: Supports database, object-driven, data-driven, keyword-driven testing.

3. TimeShiftX

TimeShiftX is a time shift regression testing tool. It allows applications to travel in the past or future to help you perform temporary or date simulating testing. 

Key Features: Requires no Environment Reboots & Reloads, Supports all platforms & operating systems.

4. Regression Tester 

Regression Tester is a regression testing tool by Info-Pack.com. It allows you to perform remote testing of web-based applications. 

Key Features: Easy creation of tests list, automatically runs test, fully-customized report generation.

5. Katalon 

It is an open-source tool. It has a dual interchangeable interface for creating test cases,  a manual view for the less technical users and a script view for experienced testers.

Key feature: Supports all platforms(Web, Native & Desktop) and browsers, script execution on multiple devices.

Cover pic: https://chamaileon.io/resources/animated-emails/