Incremental Testing in Software Testing

You know about different types of testing that follow a hierarchy: unit testing, integration testing, regression testing, acceptance testing, and more. But do you know all these testing types can be executed using different approaches? For instance, a popular approach to Unit testing is TDD (test-driven development). TDD involves creating the unit test code before building the application. Similarly, Incremental testing is a common and most used approach for Integration testing. This blog takes the discussion further on Incremental testing and talks about its benefits, different methodologies, and much more.

What is Incremental testing?

Incremental testing is an Integration testing approach that uses stubs and drivers to check different modules.

Integration testing comes after Unit testing, where developers connect individual modules to verify the interaction between them. Incremental testing is one method used to integrate the modules one by one using stubs and drivers. Using several stubs and drivers to test the modules one after another helps to uncover bugs in a specific module. A simple process in Incremental testing is testing a module, connecting it with another module, verifying their integration, and then adding another module to continue the same steps.

Testers or developers can implement this technique to perform Integration testing on the application modules.

Why is Incremental Testing Important?

Incremental testing has multiple advantages, making it a common go-to approach for Integration testing. Here are some of the reasons why Incremental testing is important:

• It helps discover issues in particular modules by executing unit testing before moving on to the next testing phase.
• This approach helps to identify bugs early in the Integration testing phase, before all the modules are connected, which saves time and resources.
• It ensures that all the modules are successfully connected and functioning properly after integration.
• It is more flexible, easy, and cost-effective from a business viewpoint.

When Should You Use Incremental Testing?

Incremental testing relies on connecting modules one by one rather than integrating all of them at once. This approach starts after unit testing brings out issues in individual modules.

The right time or the situation to use Incremental testing is when one of the modules is still under development. Stubs and drivers can be used as replacements for such modules if needed. Suppose you have a requirement to verify modules A, B, and C together, but only modules B and C are up and ready. In this case, you will use drivers to replicate module A, integrate that with B and C, and continue your Integration testing.

We will elaborate more on stubs and drivers and their functions in the next few sections of this blog. For now, this is a simple example for you to understand when Incremental testing is the right approach to use.

When Should You Automate Incremental Testing?

Incremental Integration testing starts after unit testing, where the developers test the modules separately. Thereafter, they integrate them and begin integration testing by following an incremental approach. You can execute these tests however you like, but we have a list of scenarios where you can automate Incremental testing:

• When you need to use the same test cases for testing the modules multiple times.
• Your product supports automation testing and is stable for you to run the test scripts.
• The test cases provide better results when you automate them and require less manual effort.
• When test automation will provide greater ROI on running the tests.

These are some common situations where automation testing is the right thing. But let’s move forward with a particular scenario where you should automate incremental testing.

Incremental Testing Example

You need to verify the connection between the pages of an e-commerce website: the product listing page, the product description page, and the shopping cart. Let’s consider them three different modules, which will undergo separate testing before integrating.

• The product listing page will consist of all the products available to buy on the website.
• The product description page will integrate with the product listing page and contain information about the specific product, customer reviews, and the add-to-cart option.
• The shopping cart will integrate with the product description page that will reflect your shopping items. You can only add the products to the cart from the description page.

Automating the test cases in this scenario will save time and effort and help you reuse the same tests for other listing and description pages. For instance, you can use the same test cases for the electronics listing page and apparel page. The only difference here would be item type; the functionality of the page/module will stay the same.

When automating, make sure that you use the right tool for automation. The tool should support automation of your workflows and should be easy for your team to work on. Testsigma is a tool for test automation that lets you automate for web, mobile, desktop and APIs in minutes. Testsigma is also available as an open-source and free version.

Start automating your tests for web, mobile, desktop, and APIs with Testsigma open source.

Incremental Testing Methodologies

There are three Incremental testing methods for Integration testing. Testers make use of stubs/drivers depending on which methodology they are using. Let’s discuss them in detail below.

Top-Down Methodology

The top-down approach refers to performing the testing process from top-to-bottom. In this case, the integration starts from the top-level modules and goes down to connect lower-level modules to the top-level one by one. If some lower-level modules are unavailable for testing or are still in development, stubs are used as their substitute.

Stubs are dummy programs that replace a lower-level module in a top-down hierarchy. They are referred to as ‘called programs’ as higher-level modules call them to check the interface and integration.ExampleLet’s consider multiple modules for integration.

• Module Log: Log-in page for users to log into the application/website
• Module FP: Forgot password option present on the log-in page
• Module PL: The product list page
• Module PD: The product description page
• Module Cart: Shopping cart on PD page
• Module Pay: Payment option on PD page

All these modules have different functions and integrate with one another differently.

Suppose you have already checked the Log module and now need to integrate it with the FP module. However, the latter is still under-development. In that case, the FP module will be replaced by a stub. The hierarchy of integration will go down with Module Log, PL, and PD integrating with one another. If any lower-level modules are missing, they will be replaced by stubs.

Why Should You Use Top-down Methodology?

With the top-down approach, you can identify higher-level defects quickly, as you will first test the top modules before moving downward. And you will also find it easy to discover interface errors in a sequential manner as you go testing and integrating the modules in an incremental manner. It will further highlight the issues in the main control points where the different levels will integrate and pass information.

Disadvantages of Top-Down Methodology

One huge disadvantage of the top-down method is that stubs do not accurately mimic the replaced modules. While the stubs help test the connection and information flow upon integration, they are unreliable in checking the functionality of the modules they are duplicating.

And if there are modules with major/important functionalities coming in the bottom level, then replacing them with stubs to continue with testing will only identify connection flaws. It will not be able to highlight functionality issues.

Bottom-Up Methodology

Opposite to the top-down approach in Incremental testing is the bottom-up method. It moves in an upward manner, where the lower-level modules are tested first and integrated with higher-level modules. This approach is right when your bottom-level sub-systems are ready for testing. In this case, stubs are not required because the modules at the lower level are already up and running. However, if higher-level sub-systems are not ready for testing and integration, they are replaced by drivers that are referred to as ‘calling programs.

Let’s take the same example as the top-down approach for bottom-up. In this case, you will first test modules Pay and Cart before integrating them with PD. Thereafter you will integrate and test PD with PL and keep going the same way until you integrate and test module Log, the highest-level sub-system in this application. And if any of the top-level modules are missing, you can replace them with drivers. Here, we will use drivers for PD and PL.

This is the integration and testing sequence:

1. Unit test module Cart and Pay
2. Integrate and test PD-Cart-Pay
3. Integrate and test PL-PD-Cart-Pay
4. Unit test module FP
5. Integrate and test Log-FP-PL-PD-Cart-Pay

Why Should You Use the Bottom-Up Methodology?

Often critical application functionalities are present in the lower-level modules. And with the bottom-up approach, you can test and integrate these essential operations early on and fix the issues if any are found. Creating test conditions, such as transactions and functions, is also easier in the bottom-up method.

Disadvantages of Bottom-Up Methodology

It requires more time as writing drivers is comparatively more difficult than stubs. You would need to articulate the top-level modules with a proper interface structure so that they connect with the lower-level modules. And if the number of modules is greater, the complete integration and testing process takes more time and becomes complex.

Sandwich Testing Methodology

This approach is a mix of top-down and bottom-up methodologies. It is a hybrid model where both stubs and drivers are used to replace the missing modules depending on their levels.

In the Sandwich testing method, you identify a middle layer and move upward or downward. Suppose there are three layers of modules, 1, 2, and 3. Layer 2 is the middle layer. From there, if you move upward to integrate and test the modules, you follow the bottom-up approach. Similarly, if you move down to layer 3 to integrate and test the modules, you follow the top-down approach. If any top-layer modules are missing, you use stubs; if any bottom-layer modules are missing, you use drivers as substitutes.

For example, consider the below image with three layers and seven modules in total.

This is what the test cases would look like under Sandwich testing:

• Test the modules in the first layer and bottom layer separately as part of unit testing: Modules A, E, F, and G (unit testing is done before integration testing)
• Then, integrate and test modules A, B, C, and D
• Integrate and test B, E, and F
• You integrate and test C and G
• Integrate and test A, B, C, D, E, and F

Why Should You Choose Sandwich Testing Methodology?

You should choose Sandwich testing when you have multiple layers with several modules. To avoid making the integration and testing process long and complex, use this approach to run top-down and bottom-up testing methodologies in parallel.

Disadvantages of Sandwich Testing Methodology

It requires a higher budget as both top-down and bottom-up methodologies are used here. And this testing is not accurate for modules that are hugely dependent on each other and influence the functioning of the application to a high degree.

Conclusion

Incremental integration testing follows a hierarchy to integrate the sub-systems and ensure that the overall integration process is not complex and difficult to manage.

Incremental testing undertakes three types of methods: top-down, bottom-up, and sandwich. They offer varying benefits and are preferred by testers for different integrations. We have discussed the examples above for your better understanding.

Frequently Asked Questions (FAQs)

What is Incremental testing in DevOps?

Incremental testing in DevOps provides the most efficient use of development and testing resources. It includes identifying the different modules of the application and automatically testing and pushing the changes using DevOps tools to the production environment for the users.

Which is the benefit of Incremental testing?

Incremental testing is easier to debug if there are a smaller number of modules. It allows to discover bugs early in the SDLC, supports parallel testing, and offers the space to scale the tests with the testing requirements. That is why choosing a test automation tool that consists of all these features is the right option.

What is an example of Incremental testing?

An example of Incremental testing is integrating and testing the log-in page or module before integrating it with the forgot password and homepage modules. There are three sub-systems in this case, which connect with each other. Incremental testing would include

• checking the log-in module separately as part of unit testing (should be done before incremental integration testing),
• integrating it with the forgot password first
• Then, integrating to the homepage module, and
• executing the final Integration test.