Assertion Testing – What it is and How to Perform?

Assertions are your trusty companions for documenting, debugging, and testing your code’s integrity as it takes shape. They are the vigilant gatekeepers that help you craft efficient, robust, and reliable test cases.

Assertion testing: your key to bug-free, reliable software.

What is Assertion-Based Testing?

Assertion-based testing is a robust methodology in software quality assurance where developers or testers use specific statements, known as assertions, to validate the expected behavior of a program or system. These assertions are logical expressions that define the conditions that must hold proper at various points within the code. Assertion-based testing is primarily employed to uncover defects or bugs during the development and testing.

By incorporating assertions into the testing process, teams can systematically verify that their software functions correctly under various scenarios. When an assertion fails, it serves as an early warning system, pinpointing the precise location and nature of a defect, making debugging more efficient, and facilitating the creation of reliable and resilient software. 

Assertion-based testing is an essential practice in ensuring software quality and reliability.

Benefits of Assertions

Assertions are a valuable tool in software development for several reasons:

Detecting Subtle Errors: Assertions excel at uncovering subtle, hard-to-spot errors that might otherwise go unnoticed during development and testing. They act as a safety net, catching issues that might slip through manual code reviews or traditional testing methods.

Early Error Detection: Assertions allow for the early detection of errors. They check the validity of certain conditions as the program runs, pinpointing issues as soon as they occur. This early detection reduces the time and effort required for debugging, ultimately leading to more efficient development.

Guaranteed True Statements: Assertions make assertions (hence the name) about the state of the code that are guaranteed to be true if the code functions correctly. This provides a precise and reliable way to document and validate the expected behavior of the software.

Limitations of Assertions:

While assertions offer numerous benefits, they also have some limitations and potential drawbacks:

Error Reporting Failures: Assertions, like any code, can contain errors. If an assertion is faulty, it may fail to report an existing bug, leading to a false sense of security.

False Positives: Conversely, assertions can also report errors that don’t exist, potentially wasting time and causing unnecessary concern among developers.

Side Effects: Assertions may have unintended side effects on the program’s execution, especially if they involve complex operations. This can lead to unexpected behavior and make debugging more challenging.

Execution Overhead: Assertions take time, especially when they involve complex conditions or calculations. In addition, they consume memory resources. While this overhead is generally acceptable during development and testing, it may not be acceptable for production systems where performance is critical.

In summary, assertions are a powerful tool for identifying and addressing software defects, but their effectiveness depends on their proper implementation and careful consideration of their potential limitations.

Types of Assertions in Software Testing

There are two types of assertions:

Hard Asserts

• Definition: Hard asserts immediately terminate the test when an assertion fails. This means the test stops executing further steps, and the failure is reported.
• Usage: Suitable when you want to ensure that subsequent test steps are not executed if a critical condition fails. This helps in identifying issues early and avoiding potential cascading errors.
• Example: Using TestNG for a hard assert.

import org.testng.Assert;
import org.testng.annotations.Test;
public class HardAssertExample {
    @Test
    public void testHardAssert() {
        System.out.println("Starting test");
        // Step 1: Check a condition
        int actualValue = 5;
        int expectedValue = 10;
        Assert.assertEquals(actualValue, expectedValue, "Values do not match!");
        // Step 2: This step will not execute if the assertion fails
        System.out.println("This line will not be executed if the assertion fails.");
    }
}

Explanation: In this example, if actualValue is not equal to expectedValue, the test fails immediately, and the subsequent System.out.println statement is not executed.

Soft Asserts

• Definition: Soft assertions allow the test to continue even if one or more assertions fail. They collect failures and report them after all test steps have been executed.
• Usage: Useful when you want to validate multiple conditions in a single test without stopping the test execution. This can provide a comprehensive view of multiple issues in a single run.
• Example: Using TestNG for a soft assert.

import org.testng.asserts.SoftAssert;
import org.testng.annotations.Test;
public class SoftAssertExample {
    @Test
    public void testSoftAssert() {
        System.out.println("Starting test");
        SoftAssert softAssert = new SoftAssert();
        // Step 1: Check a condition
        int actualValue = 5;
        int expectedValue = 10;
        softAssert.assertEquals(actualValue, expectedValue, "Values do not match!");
        // Step 2: Check another condition
        String actualTitle = "Home Page";
        String expectedTitle = "Login Page";
        softAssert.assertEquals(actualTitle, expectedTitle, "Titles do not match!");
        // Report all soft assertion failures
        softAssert.assertAll();
        System.out.println("This line will be executed regardless of assertion failures.");
    }
}

Explanation: In this example, both assertions are checked. Even if the first assertion fails, the second assertion will still be evaluated. All assertion failures are reported together at the end of the test when softAssert.assertAll() is called.

To Summarize

• Hard Asserts: Halt the test execution immediately when an assertion fails. Use them when subsequent steps should not execute if a critical condition fails.
• Soft Asserts: Allow the test to continue even if assertions fail, collecting and reporting all failures at the end. Use them when you need to validate multiple conditions in a single test without stopping execution.

Here is a comparison between them:

Feature	Hard Assert	Soft Assert
Execution Flow	Stops test execution on failure	Continues test execution on failure
Failure Handling	Immediate failure report	Failure collected and reported later
Use Case	Critical conditions where further testing is meaningless	Multiple conditions or non-critical checks
Example (Java)	Assert.assertEquals(expected, actual)	SoftAssert softAssert = new SoftAssert(); softAssert.assertEquals(expected, actual);
Impact on Test Results	Directly impacts the test result (failures immediately affect the result)	All assertions are collected before determining the final result

Both types of assertions have their uses depending on the testing scenario and what you aim to achieve with your tests.

Advantages of Assertion Testing

Ease of Use: The assertions are easy to write and understand so that one can check whether the code behaves as expected without any difficulty.

Immediate Feedback: They act as an immediate feedback tool; this helps in the quick detection and rectification of issues.

Improves Code Quality: It helps in error-catching during development, which ultimately improves the quality of the code.

Self-Documenting: Assertions might be self-contained for your code by displaying what the code is supposed to do and helping to understand its expected behavior.

No extra overhead in production: A most common functionality of many assertion frameworks is that they can be turned off (or completely removed) in production mode, ensuring that assertions will not slow down running applications.

Disadvantages of Assertion Testing

Not a Replacement for Tests: They can catch some issues but will not cover all possible problems.

Can Be Ignored: Assertions may be skipped or disabled if not properly used, and error checks could be missed as a result.

May Hide Errors: Too much reliance on asserts might make developers overlook other potential issues or skip proper testing.

Performance Impact During Testing: Typically negligible, assertions might introduce some performance overhead during development and test execution.

Limited Error Handling: Programs that have hard assertion checking enabled will stop running as soon as a false assertion is encountered. This may not always be the most desirable behavior, especially in production systems.

How is Assertion Used in Functional Testing?

Assertions are crucial in functional testing for validating that an application behaves as expected. They act as checkpoints within test scripts to confirm the functionality of the application. 

Here’s how assertions are used in functional testing with some straightforward examples.

1. Checking Button Text

Test Case Example: Verify that the “Login” button has the correct text.

• Scenario: On the login page, ensure that the button labeled “Login” displays the correct text.
• How Assertion Is Used: In Java

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.Assert;

public class ButtonTextTest {
    public static void main(String[] args) {
        WebDriver driver = new ChromeDriver();
        driver.get("http://example.com/login");
        
        WebElement loginButton = driver.findElement(By.id("login_button"));
        String buttonText = loginButton.getText();
        String expectedText = "Login";
        
        Assert.assertEquals(buttonText, expectedText, "Expected button text to be '" + expectedText + "', but got '" + buttonText + "'.");
        
        driver.quit();
    }
}

• Explanation: The assertion checks if the buttonText retrieved from the web element matches the expectedText. If not, the test fails, indicating a discrepancy in the button’s label.

2. Validating Page Title

Test Case Example: Confirm that the page title is correct after navigating to the homepage.

• Scenario: After navigating to the homepage, verify that the page title is “Welcome Page.”
• How Assertion Is Used: In Java

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.Assert;
public class PageTitleTest {
    public static void main(String[] args) {
        WebDriver driver = new ChromeDriver();
        driver.get("http://example.com");
        String expectedTitle = "Welcome Page";
        String actualTitle = driver.getTitle();
         Assert.assertEquals(actualTitle, expectedTitle, "The page title is not as expected.");
         driver.quit();
    }
}

• Explanation: This assertion compares the actualTitle of the page with the expectedTitle. If they don’t match, the test fails, indicating that the page title does not meet expectations.

How to Perform Assertion Testing in Test Automation

Assertion testing in test automation is a critical practice to validate the correctness of your software application. It involves using specific checkpoints, or assertions, to verify whether the application behaves as expected. Here’s a step-by-step guide on how to perform assertion testing, along with examples of different types of assertions, while also exploring how a tool like Testsigma can streamline this process:

Step 1: Test Scenario Setup

Begin by defining the test scenario you want to automate. This includes specifying the user actions, input data, and expected outcomes. Ensure that you have a clear understanding of the application’s functionality you intend to test.

Step 2: Select an Automation Tool

Choose an automation testing tool that supports assertion testing. Testsigma, for instance, is a powerful automation testing platform that simplifies the process. It offers a user-friendly interface and supports various types of assertions.

Automate your tests for web, mobile, desktop and APIs, with powerful assertions, with Testsigma

Step 3: Record or Write Test Scripts

Record or write test scripts that replicate the user actions within the application. These actions should include interacting with different elements such as buttons, forms, and links.

Step 4: Implement Assertions

Integrate assertions into your test scripts to validate the application’s expected behavior. Here are examples of different types of assertions:

   a. Text Assertions: Verify that specific text appears on the webpage. For instance, you can assert that a login success message is displayed after a successful login.

   b. Element Existence Assertions: Check whether an element exists on the page. This can be useful to ensure that a critical element like a submit button is present on a form.

   c. Data Assertions: Validate data integrity by comparing data retrieved from the application with expected values. For instance, confirm that a user’s name on the profile page matches the expected name.

   d. Navigation Assertions: Ensure that users are directed to the correct page after performing certain actions. You can assert that clicking on a “Home” link takes the user to the homepage.

   e. Error Message Assertions: Confirm that error messages are displayed when they should be. For instance, assert that an error message appears when an incorrect password is entered during login.

Step 5: Run the Test

Execute the test script using your chosen automation tool. It will simulate user interactions and perform assertions to validate the application’s behavior.

Step 6: Analyze Test Results

Review the test results to identify any failures or unexpected behavior. Assertions that fail indicate potential defects in the application. Detailed reports should be generated to facilitate debugging and analysis.

Step 7: Debug and Refine

If any assertions fail, debug the test script to pinpoint the issue. Modify the test script or update assertions as necessary. Ensure that the test script is robust and reliable.

Step 8: Re-run and Schedule Tests

After addressing any issues, re-run the test script to confirm that the application’s behavior is as expected. Automation tools like Testsigma allow you to schedule tests for regular execution, ensuring continuous validation of your application.

Testsigma: Simplifying Assertion Testing

Testsigma plays a pivotal role in simplifying the assertion testing process. It offers a user-friendly platform that streamlines the creation and execution of test scripts. Notably, Testsigma supports many assertion types, making it a versatile tool for quality assurance. Additionally, it boasts features such as data-driven testing, cross-browser compatibility testing, and parallel test execution.

With Testsigma as your testing companion, you can efficiently conduct assertion testing, identify defects at an early stage of development, and consistently uphold the quality and reliability of your software applications. Its intuitive interface and robust capabilities empower testing teams to deliver more dependable and resilient software products.

How to Do Assertion Testing with Testsigma ?

To work with Testsigma effectively, utilize the following assertion statements to validate various types of test cases and scenarios:

Text Assertion:

Use this assertion to verify text content on web pages, ensuring that the expected text matches the actual content.

Element Presence Assertion:

Validate the presence or absence of specific elements on a page, such as buttons, forms, or images, to ensure proper rendering.

Element State Assertion:

Check the state of UI elements, like buttons being enabled or disabled, to validate the expected behavior.

Element Value Assertion:

Confirm that the values of input fields or dropdown selections match the expected data, ensuring data accuracy.

Element Attribute Assertion:

Validate the attributes of elements, like checking if a link has the correct URL or a checkbox is checked.

Element Count Assertion:

Use this assertion to verify the count of specific elements on a page, ensuring that, for example, a list contains the expected number of items.

Element Order Assertion:

Check the order of elements, like validating the sequence of steps in a process or items in a list.

Element Size and Position Assertion:

Validate the size and position of elements on a page, ensuring proper layout and alignment.

Page Title Assertion:

Confirm that the page title matches the expected title, verifying that users are on the correct page.

Page URL Assertion:

Ensure the current page’s URL matches the expected URL, validating navigation and redirection.

Page State Assertion:

Check the overall state of a page, such as whether it’s loaded, in a specific view, or displaying a particular message.

Custom JavaScript Assertion:

Create custom JavaScript assertions to validate complex scenarios or calculations within the application.

API Response Assertion:

Use this assertion to validate API responses, ensuring the returned data conforms to expected formats and values.

Database Assertion:

Verify data integrity by asserting values retrieved from a database against expected values.

Performance Metrics Assertion:

Check performance metrics like page load time, response times, or resource utilization to ensure optimal application performance.

Leverage these assertion statements within Testsigma to thoroughly validate your test cases and test scenarios, ensuring the reliability and accuracy of your automated tests.

Summary

In this comprehensive guide, we’ve explored the world of assertion testing in test automation. Assertion testing is a critical practice for validating the correctness of software applications. We discussed its benefits, limitations, and the step-by-step process of performing assertion testing. Additionally, we highlighted the role of Testsigma, an automation testing tool, in simplifying the assertion testing process.

Frequently Asked Questions

What is an example of assertion testing?

An example of assertion testing is verifying whether a user registration process on a website is functioning correctly. Here, assertions could be used to confirm that after users submit their registration details, they are redirected to a welcome page, and their information is correctly displayed in their profile.

How many assertions would you place in automated tests?

The number of assertions in automated tests can vary significantly depending on the complexity of the application and the specific test scenario. In general, it’s a good practice to include multiple assertions in a test suite to validate different aspects of the application’s behavior thoroughly. However, the key is to balance ensuring comprehensive coverage and avoiding overly complex test scripts. Each assertion should serve a clear and meaningful purpose in verifying the application’s functionality.