JUnit vs TestNG: Comparing Two Popular Testing Frameworks

JUnit vs TestNG: Comparing Two Popular Testing Frameworks

Testing is the most critical part of the Software Development Life Cycle(SDLC). Testing is performed at different levels such as Unit Testing, Integration Testing, and Functional Testing. Testing can be performed either manually or through automation. The tools or libraries which are used for test automation vary with the framework that you are using. For example, if you are using a JavaScript framework, you might have to use Jasmine, Jest, etc. The JUnit and TestNG are the most popular Java testing frameworks, so let’s explore them.

JUnit vs TestNG: Key Differences

The differences between JUnit vs TestNG are mainly in their features and functionality. JUnit is a popular Java testing framework that provides annotations and assertions for testing code.

FeaturesJUnitTestNG
Open SourceJUnit is Open Source Unit Testing FrameworkTestNG is Open source Testing framework
Set upSetting up JUnit is easySetting up TestNG is also easy
User friendlyJUnit doesn’t support everything out of the box. For example, parameterization needs JUnit Jupitor integrationTestNG supports all required features out of the box
Listener’s Implementation through annotationsJunit doesn’t support listeners implementations through annotations, it supports via Listeners APITestNG supports listeners implementations through annotations
ReportingNo in-built HTML reporting. The report can be generated through MavenTestNG has its own in-built HTML reports. However, it can also be integrated with Maven to generate extensive reports
Parallel Test ExecutionJUnit doesn’t directly support parallel executionTestNG supports parallel execution
Dependency TestJUnit doesn’t support the Dependency testTestNG supports dependency tests
Data-Driven TestingThough JUnit supports Data driven testing, more effort and technical expertise are required to setupTestNG provides special annotations for data-driven testing
Test GroupingSupports Grouping TestsSupports Grouping test cases
Tests Priority/Order of ExecutionOrder of execution can be achieved in JUnitTestNG supports setting up test cases priority which helps to maintain the order
Assumptions SupportJUnit supports assumptionsTestNG doesn’t support assumptions

Let’s understand the JUnit and TestNG in detail.

What is JUnit?

JUnit is the most popular unit testing framework for Java. The JUnit was built primarily focused on Unit testing. With some tweaks and other tools integration, JUnit can be used for functional and integration testing as well.

JUnit empowers Test Driven Development (TDD), where tests are written before writing actual code. 

Features of JUnit

  • JUnit is Open source
  • JUnit can be easily used with popular IDE’s such as Eclipse and IntelliJ
  • JUnit tests can be integrated into CI/CD pipeline
  • JUnit can be used with Unit tests and Selenium tests
  • JUnit provides robust assertions to test expected outcome
  • It provides a powerful debugging feature

Test Set up of JUnit

Test setup in JUnit refers to preparing the test environment before running the test cases. These setup and teardown methods ensure that each test case runs in an isolated environment and test results are not affected by external factors. 

Installing JUnit Step by Step  

  • Install Java 8 or above
  • The JUnit 5 requires Java 8 or above
  • Navigate to the Java download page and install Java

JUnit latest version is called JUnit Jupiter,  which has a bundle of new features and competes with TestNG.

Create a Simple Maven Project

From your IDE create a Simple Maven project, ensure POM.xml is created

Install JUnit5 to your Project

Navigate to POM.xml file and add the JUnit5 Dependencies as below

<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>5.9.1</version>
<scope>test</scope>
</dependency>

Once you add the above dependency, it will install all required jars

  • junit-jupiter-api.jar
  • junit-jupiter-engine.jar
  • junit-platform-commons.jar
  • junit-platform-engine.jar

We have completed the Basic setup for JUnit 5

Simple JUnit Test Example

import static org.junit.jupiter.api.Assertions.assertTrue;
import org.junit.jupiter.api.*;
public class AppTest {
@Test
public void testApp() {
int a = 10;
assertTrue(a > 5);
}
}

In the above code, we created a Simple JUnit test with the name test app. We are using @Test annotation, which will compile that this is the JUnit test.

Executing JUnit Tests

Once you have created the above test, Run the test with JUnit.

If you are executing using Eclipse, Right-click on the Project, Click on Run with JUnit Test

Note: Ensure In the Run configuration JUnit is selected for Test Runner

JUnit Annotations

Below are some most commonly used JUnit annotations:

@Test:  This annotation is the most used annotation that denotes the test method

@BeforeAll: Denotes the method should execute before  all 

@AfterAll: Denotes the method should execute after all

@BeforeEach: Denotes method should execute before each test

@AfterEach: Denotes method should execute after each test

@Tag: This annotation is used for declaring filtering tests, or grouping tests

@Disabled: Used to ignore or disable the test

@TestMethodOrder: Used for declaring the test method order

@TestClassOrder: Used for declaring the test class order

Declaring the Test Suites in JUnit

The JUnit provides the option to declare the test suites with @Suites annotations. This helps in aggregating the tests. You can choose the set of classes, tags, or packages to aggregate.

Example:

@IncludeTags("staging")
@Suite
@SuiteDisplayName("A demo Test Suite for Staging")
public class JUnit5TestSuiteDemo {
}

Grouping Tests 

JUnit latest version supports grouping tests, The JUnit 5 (Jupiter) you can group the tests in multiple ways 

1. Using Test suites 

Use the @Suite annotation (as explained above) to group or aggregate the test together.

2. Using Tag annotation

The Tag annotation helps to group the test. For example, you might have to group the tests as Sanity, Smoke, Regression, SomeFeature, etc. you can use the @tag annotation for this.

Example Using the Tag Annotation in JUnit

public class AppTest {
@Test
@Tag("Smoke")
public void Test1() {
System.out.println("Example 1");
}
@Test 
@Tag("Regression")
public void Test2() {
System.out.println("Example 2");
}
@Test
@Tag("Smoke")
public void Test3() {
System.out.println("Example 3");
}
}

Note: To run selected tags, use the rung configurations in your IDE and choose the tags.

Disabling or Ignoring the tests

JUnit 4 allows to ignore the tests of the @Ignore tag, however, the JUnit5 version uses the @Disabled tag for the same. Using the @Disabled tag you can skip or ignore the particular tests.

Example: Ignoring or Disabling the Tests

public class AppTest {
@Test 
@Tag("Smoke")
public void Test1() {
System.out.println("Example 1");
}
@Disabled //This test will not execute
@Test
@Tag("Regression")
public void Test2() {
System.out.println("Example 2");
}
@Test
@Tag("Smoke")
public void Test3() {
System.out.println("Example 3");
}
}

Parameterization of Tests

The parameterization is mainly used in data-driven testing. Using the @parameterizedTest annotation, you can pass the different types of values. This helps to validate the output against multiple inputs

Example:

public class AppTest {
@ParameterizedTest
@ValueSource(ints = {0, 2, 3, 5, 6}) // six numbers
void exampleOfPrameterizedTest(int digits) {
System.out.println(digits);
}
}

The Annotation @ ParameterizedTest is available in the package junit-jupiter-params, so you need to add this dependency to your POM.xml file.

<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-params</artifactId>
<version>5.9.1</version>
<scope>test</scope>
</dependency>

By using some of the above concepts lets create a Sample JUnit Test

package com.junit_demo.junit_demo;
import static org.junit.jupiter.api.Assertions.assertTrue;
import java.util.Random;
import org.junit.jupiter.api.*;
import org.junit.jupiter.api.extension.ExtendWith;
import org.junit.jupiter.api.extension.ExtensionContext;
import org.junit.jupiter.api.extension.TestExecutionExceptionHandler;
import org.junit.platform.suite.api.IncludeTags;
import org.junit.platform.suite.api.Suite;
import org.junit.platform.suite.api.SuiteDisplayName;
@Disabled
public class AppTest2 {
static int a, b, randomNum, sum;
@BeforeAll
public static void setup() {
a=10;
b=20;
}
@BeforeEach
public void setupForEachTest() {
randomNum = new Random().nextInt((100-0));
System.out.println(randomNum);
}
@Test
@Tag("Smoke")
public void testAddition() {
int sum = a+b+randomNum;
assertTrue(sum<131);
}
@AfterEach
public void afterEachTest() {
System.out.println("After each executed");
}
@AfterAll
public static void teardown() {
a=0;
b=0;
}
}

What is TestNG?

TestNG is a testing framework, which can be used for both Unit and End to End testing. Unlike JUnit, we don’t have to install multiple jars for multiple features. TestNG comes with all the features in one jar.

Features of TestNG

  • Easy to install and easy to use
  • In Built HTML reporter
  • Easy-to-debug tests
  • Ordering and grouping of tests
  • Dependency test
  • Easy to configure data-driven testing
  • XML based test configuration
  • Easy to maintain tests.

Installing TestNG for Java Project

  1. Install the Java
  2. Add the entries into POM.xml file 
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>7.6.1</version>
<scope>test</scope>
</dependency>

Once you add the above dependency the testNG automatically adds the relevant jars to your project.

Understanding TestNG

TestNG offers multiple features such as annotations, test configuration, data-driven testing, parallel test execution, and HTML test reports. This Framework also enables flexible test configuration and allows users to define dependencies between test methods and groups. Moreover, this also supports parameterized tests, allowing users to run the same test with different input parameters. Understanding TestNG is essential for Java developers who want to create practical and comprehensive automated tests for their projects. 

TestNG annotations

Just like JUnit, the TestNG uses annotations. Below are some most commonly used annotations.

@BeforeSuite: This method runs before all the methods in the suites

@AfterSuite: This method runs after all the methods in the suites

@BeforeTest: This annotated method runs before all the test methods in the class

@AfterTest: This method runs after the test methods in the class

@BeforeGroups: The list of groups that this configuration method will run before. This method is guaranteed to run shortly before the first test method that belongs to any of these groups is invoked.

@AfterGroups: The list of groups that this configuration method will run after. 

@BeforeClass: The annotated method will be run before the first test method in the current class is invoked.

@AfterClass: The annotated method will be run after all the test methods in the current class have been run.

@BeforeMethod: The annotated method will be run before each test method.

@AfterMethod: The annotated method will be run after each test method.

What is a testng.xml file?

The TestNG uses XML files for configuration, that is testng.xml file. The testng.xml file is an import file, which can be used for defining test configurations such as suites, test name, class name, parallel execution, etc.

How to Generate testng.xml file?

To Generate the testng.xml file, right-click on the Project > Click on testng > click on convert to TestNG.

How to Configure suites in TestNG?

testng.xml is the file where we configure the test suites. To configure the test suites, open the testng.xml file and provide the suite name as below

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="ExampleSuite">
<test thread-count="5" name="ExampleTest">
<classes>
<class name="com.testngdemo.testngDemos.AppTest"/>
</classes>
</test> <!-- Test -->
</suite> <!-- Suite -->

Skipping or Ignoring the test in TestNG

The TestNG is very user-friendly. The @Test takes parameters called enabled, and you can pass a boolean value true or false to enable or disable tests. When you pass the enabled as false, the test will be ignored.

Example:

public class App 
{
@Test(enabled=false)
public void myTest()
{
System.out.println( "Hello World!" );
}
}

Grouping and Organizing the Test in TestNG 

The @Test annotation provides a groups parameter. Using the groups parameter, you can specify the group name.

Example:

public class App
{
@Test(groups= {"smoke"})
public void myTest()
{
System.out.println( "Smoke test" );
}
@Test(groups= {"regression"})
public void myTest2()
{
System.out.println( "Regression test" );
}
}

Executing TestNG group of tests

Once you include the groups to test, in order to execute specific groups, you need to specify the group name in the testng.xml file.

<test thread-count="5" name="ExampleTest">
<groups>
<run>
<include name="smoke" />
</run>
</groups>
<classes>
<class name="com.testngdemo.demoTest.AppTest" />
</classes>
</test> <!-- Test -->

Handling expected Exception in TestNG

The TestNG provides expectedExceptions option in @Test attribute, we can specify the exception class name to handle the known exception gracefully.

Example:

@Test(expectedExceptions = { IOException.class })
public void exceptionExample() throws Exception {
throw new IOException();
}

TestNG Parameterization

The parameterization feature of TestNG enables users to supply parameters to test methods and execute them numerous times with varied inputs. It is advantageous for data-driven testing, in which the same test case must be executed with varied input data sets. Data providers, which may get data from multiple sources such as Excel spreadsheets, CSV files, and databases, are used to facilitate parameterization in TestNG. TestNG can cover multiple scenarios with less effort thanks to parameterization, leading to more efficient and thorough testing.

@Test
@Parameters("somedata")
public void myTest3(String somedata)
{
System.out.println( "Smoke test" + somedata);
}

Testng.xml

<test thread-count="5" name="ExampleTest">
<parameter name = "somedata" value = "Example Parameter"/>
<classes>
<class name="com.testngdemo.demoTest.AppTest" />
</classes>
</test> <!-- Test -->

Parameterization using Data Provider

TestNG allows users to parameterize using data providers. A data provider is a method that delivers test data to a test method. Data providers can retrieve data from various sources such as Excel spreadsheets, CSV files, databases, etc. TestNG supports two types of data providers: Static and dynamic.

public class AppTest {
@DataProvider(name = "somedata")
public Object[][] exampleProvider() {
return new Object[][] { { "Value1" }, { "Value2" } };
}
@Test(dataProvider = "somedata")
public void myTest3(String somedata) {
System.out.println("Smoke test " + somedata);
}
}

Example TestNG test

package com.testngdemo.demoTest;
import org.testng.annotations.AfterClass;
import org.testng.annotations.AfterTest;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.Test;
public class AppTest {
int a, b, sum;
@BeforeClass
public void beforeClassDemo() {
System.out.println("Inside Before Class");
}
@BeforeTest
public void setUp() {
a=10;
b=20;
}
@Test
public void sumTest() {
System.out.println(a+b);
}
@AfterTest
public void teardown() {
System.out.println("After Test");
}
@AfterClass
public void afterClassDemo() {
System.out.println("Inside After Class");
}
}

Summary

The TestNG and JUnit are perfect tools for end-to-end and unit testing, which provide a lot of functionalities and customization. However, both require technical expertise and a good understanding of Java programming language. 

Recently, a lot of automation and testing frameworks are coming into the market. Testsigma provides Zero Code testing where you don’t need to write a single line of code. This way, any manual tester can jump into automation and start scripting. This is a win-win for both organizations and testers, where the organization doesn’t need to spend on upgrading the skills, and testers can easily switch to automation without knowing any programming languages. The effort gets further reduced as Testsigma provides integration with many CI/CD tools.

Frequently Asked Questions

JUnit vs TestNG, which is better?

This question has no definitive solution because both TestNG and JUnit have advantages and disadvantages. Other additions include flexible test design, data-driven test support, and improved reporting capabilities. On the other hand, JUnit is extensively used and has a large ecosystem of plugins and integrations.

What is the main difference between JUnit and TestNG?

The primary distinction between JUnit and TestNG is their features and capabilities. Other additions include flexible test design, data-driven test support, and improved reporting capabilities. JUnit is extensively used and supported by a large ecosystem of plugins and integrations.

Does TestNG use JUnit?

No, TestNG is a different testing framework that does not use JUnit.

JUnit vs TestNG? What are the advantages of TestNG?

Support for data-driven tests, flexible test setup, richer reporting, and the ability to create dependencies between test methods and groups are all advantages of TestNG over JUnit. Talking about parallel test execution with JUnit Vs TestNG, TestNG also supports parallel tests, which can drastically reduce total test execution time.

Suggested Reading

NUnit vs XUnit

Unit Test vs End to End Test

Unit Test VS Integration Test

Jest VS Mocha VS Jasmine


Test automation made easy

Start your smart continuous testing journey today with Testsigma.

SHARE THIS BLOG

RELATED POSTS


Power of POC in Testing: Your Exclusive Guide to Success
Power of POC in Testing: Your Exclusive Guide to Success
performance testing tools_banner image
Test objects in software testing | Types & How to Create it?
How To Write Calculator Test Cases With Sample Test Cases
How To Write Calculator Test Cases? With Sample Test Cases