📢 Are you a TestProject user looking to migrate to an open source alternative?

Learn more
JUnit vs TestNG: Comparing Two Popular Testing Frameworks

JUnit vs TestNG: Comparing Two Popular Testing Frameworks

Testing is the most critical part of the Software Development Life Cycle(SDLC). Testing is performed at different levels such as Unit Testing, Integration Testing, and Functional Testing. Testing can be performed either manually or through automation. The tools or libraries which are used for test automation vary with the framework that you are using. For example, if you are using a JavaScript framework, you might have to use Jasmine, Jest, etc. The JUnit and TestNG are the most popular Java testing frameworks, so let’s explore them.

JUnit vs TestNG: Key Differences


FeaturesJUnitTestNG
Open SourceJUnit is Open Source Unit Testing FrameworkTestNG is Open source Testing framework
Set upSetting up JUnit is easySetting up TestNG is also easy
User friendlyJUnit doesn’t support everything out of the box. For example, parameterization needs JUnit Jupitor integrationTestNG supports all required features out of the box
Listener’s Implementation through annotationsJunit doesn’t support listeners implementations through annotations, it supports via Listeners APITestNG supports listeners implementations through annotations
ReportingNo in-built HTML reporting. The report can be generated through MavenTestNG has its own in-built HTML reports. However, it can also be integrated with Maven to generate extensive reports
Parallel Test ExecutionJUnit doesn’t directly support parallel executionTestNG supports parallel execution
Dependency TestJUnit doesn’t support the Dependency testTestNG supports dependency tests
Data-Driven TestingThough JUnit supports Data driven testing, more effort and technical expertise are required to setupTestNG provides special annotations for data-driven testing
Test GroupingSupports Grouping TestsSupports Grouping test cases
Tests Priority/Order of ExecutionOrder of execution can be achieved in JUnitTestNG supports setting up test cases priority which helps to maintain the order
Assumptions SupportJUnit supports assumptionsTestNG doesn’t support assumptions

Let’s understand the JUnit and TestNG in detail.

What is JUnit?

JUnit is the most popular unit testing framework for Java. The JUnit was built primarily focused on Unit testing. With some tweaks and other tools integration, JUnit can be used for functional and integration testing as well.

JUnit empowers Test Driven Development (TDD), where tests are written before writing actual code. 

Features of JUnit

  • JUnit is Open source
  • JUnit can be easily used with popular IDE’s such as Eclipse and IntelliJ
  • JUnit tests can be integrated into CI/CD pipeline
  • JUnit can be used with Unit tests and Selenium tests
  • JUnit provides robust assertions to test expected outcome
  • It provides a powerful debugging feature

Test Set up of JUnit

Installing JUnit Step by Step  

  • Install Java 8 or above
  • The JUnit 5 requires Java 8 or above
  • Navigate to the Java download page and install Java

JUnit latest version is called JUnit Jupiter,  which has a bundle of new features and competes with TestNG.

Create a Simple Maven Project

From your IDE create a Simple Maven project, ensure POM.xml is created

Install JUnit5 to your Project

Navigate to POM.xml file and add the JUnit5 Dependencies as below

<dependency>

    <groupId>org.junit.jupiter</groupId>

    <artifactId>junit-jupiter-engine</artifactId>

    <version>5.9.1</version>

    <scope>test</scope>

</dependency>

Once you add the above dependency, it will install all required jars

  • junit-jupiter-api.jar
  • junit-jupiter-engine.jar
  • junit-platform-commons.jar
  • junit-platform-engine.jar

We have completed the Basic setup for JUnit 5

Simple JUnit Test Example

import static org.junit.jupiter.api.Assertions.assertTrue;

import org.junit.jupiter.api.*;

public class AppTest {

@Test

public void testApp() {

int a = 10;

assertTrue(a > 5);

}

}

In the above code, we created a Simple JUnit test with the name test app. We are using @Test annotation, which will compile that this is the JUnit test.

Executing JUnit Tests

Once you have created the above test, Run the test with JUnit.

If you are executing using Eclipse, Right-click on the Project, Click on Run with JUnit Test

Note: Ensure In the Run configuration JUnit is selected for Test Runner


JUnit Annotations

Below are some most commonly used JUnit annotations:

@Test:  This annotation is the most used annotation that denotes the test method

@BeforeAll: Denotes the method should execute before  all 

@AfterAll: Denotes the method should execute after all

@BeforeEach: Denotes method should execute before each test

@AfterEach: Denotes method should execute after each test

@Tag: This annotation is used for declaring filtering tests, or grouping tests

@Disabled: Used to ignore or disable the test

@TestMethodOrder: Used for declaring the test method order

@TestClassOrder: Used for declaring the test class order

Declaring the Test Suites in JUnit

The JUnit provides the option to declare the test suites with @Suites annotations. This helps in aggregating the tests. You can choose the set of classes, tags, or packages to aggregate.

Example:

@IncludeTags(“staging”)

 @Suite

 @SuiteDisplayName(“A demo Test Suite for Staging”)

 public class JUnit5TestSuiteDemo {

 }

Grouping Tests 

JUnit latest version supports grouping tests, The JUnit 5 (Jupiter) you can group the tests in multiple ways 

1. Using Test suites 

Use the @Suite annotation (as explained above) to group or aggregate the test together.

2. Using Tag annotation

The Tag annotation helps to group the test. For example, you might have to group the tests as Sanity, Smoke, Regression, SomeFeature, etc. you can use the @tag annotation for this.

Example Using the Tag Annotation in JUnit

public class AppTest {

@Test

@Tag(“Smoke”)

public void Test1() {

System.out.println(“Example 1”);

}

@Test

@Tag(“Regression”)

public void Test2() {

System.out.println(“Example 2”);

}

@Test

@Tag(“Smoke”)

public void Test3() {

System.out.println(“Example 3”);

}

}

Note: To run selected tags, use the rung configurations in your IDE and choose the tags.

Disabling or Ignoring the tests

JUnit 4 allows to ignore the tests of the @Ignore tag, however, the JUnit5 version uses the @Disabled tag for the same. Using the @Disabled tag you can skip or ignore the particular tests.

Example: Ignoring or Disabling the Tests

public class AppTest {

@Test

@Tag(“Smoke”)

public void Test1() {

System.out.println(“Example 1”);

}

@Disabled //This test will not execute

@Test

@Tag(“Regression”)

public void Test2() {

System.out.println(“Example 2”);

}

@Test

@Tag(“Smoke”)

public void Test3() {

System.out.println(“Example 3”);

}

}

Parameterization of Tests

The parameterization is mainly used in data-driven testing. Using the @parameterizedTest annotation, you can pass the different types of values. This helps to validate the output against multiple inputs

Example:

public class AppTest {

@ParameterizedTest

@ValueSource(ints = {0, 2, 3, 5, 6}) // six numbers

void exampleOfPrameterizedTest(int digits) {

    System.out.println(digits);

}

}

The Annotation @ ParameterizedTest is available in the package junit-jupiter-params, so you need to add this dependency to your POM.xml file.

<dependency>

    <groupId>org.junit.jupiter</groupId>

    <artifactId>junit-jupiter-params</artifactId>

    <version>5.9.1</version>

    <scope>test</scope>

</dependency>

By using some of the above concepts lets create a Sample JUnit Test

package com.junit_demo.junit_demo;

import static org.junit.jupiter.api.Assertions.assertTrue;

import java.util.Random;

import org.junit.jupiter.api.*;

import org.junit.jupiter.api.extension.ExtendWith;

import org.junit.jupiter.api.extension.ExtensionContext;

import org.junit.jupiter.api.extension.TestExecutionExceptionHandler;

import org.junit.platform.suite.api.IncludeTags;

import org.junit.platform.suite.api.Suite;

import org.junit.platform.suite.api.SuiteDisplayName;

@Disabled

public class AppTest2 {

static int a, b, randomNum, sum;

@BeforeAll

public static void setup() {

a=10;

b=20;

}

@BeforeEach

public void setupForEachTest() {

randomNum = new Random().nextInt((100-0));

System.out.println(randomNum);

}

@Test

@Tag(“Smoke”)

public void testAddition() {

int sum = a+b+randomNum;

assertTrue(sum<131);

}

@AfterEach

public void afterEachTest() {

System.out.println(“After each executed”);

}

@AfterAll

public static void teardown() {

a=0;

b=0;

}

}

TestNG

TestNG is a testing framework, which can be used for both Unit and End to End testing. Unlike JUnit, we don’t have to install multiple jars for multiple features. TestNG comes with all the features in one jar.

Features or TestNG

  • Easy to install and easy to use
  • In Built HTML reporter
  • Easy-to-debug tests
  • Ordering and grouping of tests
  • Dependency test
  • Easy to configure data-driven testing
  • XML based test configuration
  • Easy to maintain tests.

Installing TestNG for Java Project

  1. Install the Java
  2. Add the entries into POM.xml file 

<dependency>

    <groupId>org.testng</groupId>

    <artifactId>testng</artifactId>

    <version>7.6.1</version>

    <scope>test</scope>

</dependency>

Once you add the above dependency the testNG automatically adds the relevant jars to your project.

Understanding TestNG

TestNG annotations

Just like JUnit, the TestNG uses annotations. Below are some most commonly used annotations.

@BeforeSuite: This method runs before all the methods in the suites

@AfterSuite: This method runs after all the methods in the suites

@BeforeTest: This annotated method runs before all the test methods in the class

@AfterTest: This method runs after the test methods in the class

@BeforeGroups: The list of groups that this configuration method will run before. This method is guaranteed to run shortly before the first test method that belongs to any of these groups is invoked.

@AfterGroups: The list of groups that this configuration method will run after. 

@BeforeClass: The annotated method will be run before the first test method in the current class is invoked.

@AfterClass: The annotated method will be run after all the test methods in the current class have been run.

@BeforeMethod: The annotated method will be run before each test method.

@AfterMethod: The annotated method will be run after each test method.

What is a testng.xml file?

The TestNG uses XML files for configuration, that is testng.xml file. The testng.xml file is an import file, which can be used for defining test configurations such as suites, test name, class name, parallel execution, etc.

How to Generate testng.xml file?

To Generate the testng.xml file, right-click on the Project > Click on testng > click on convert to TestNG.

How to Configure suites in TestNG?

testng.xml is the file where we configure the test suites. To configure the test suites, open the testng.xml file and provide the suite name as below

<?xml version=”1.0″ encoding=”UTF-8″?>

<!DOCTYPE suite SYSTEM “https://testng.org/testng-1.0.dtd”>

<suite name=”ExampleSuite”>

  <test thread-count=”5″ name=”ExampleTest”>

    <classes>

      <class name=”com.testngdemo.testngDemos.AppTest”/>

    </classes>

  </test> <!– Test –>

</suite> <!– Suite –>

Skipping or Ignoring the test in TestNG

The TestNG is very user-friendly. The @Test takes parameters called enabled, and you can pass a boolean value true or false to enable or disable tests. When you pass the enabled as false, the test will be ignored.

Example:

public class App 

{

@Test(enabled=false)

    public void myTest()

    {

        System.out.println( “Hello World!” );

    }

}

Grouping and Organizing the Test in TestNG 

The @Test annotation provides a groups parameter. Using the groups parameter, you can specify the group name.

Example:

public class App 

{

@Test(groups= {“smoke”})

    public void myTest()

    {

        System.out.println( “Smoke test” );

    }

@Test(groups= {“regression”})

    public void myTest2()

    {

        System.out.println( “Regression test” );

    }

}

Executing TestNG group of tests

Once you include the groups to test, in order to execute specific groups, you need to specify the group name in the testng.xml file.

<test thread-count=”5″ name=”ExampleTest”>

<groups>

<run>

<include name=”smoke” />

</run>

</groups>

<classes>

<class name=”com.testngdemo.demoTest.AppTest” />

</classes>

</test> <!– Test –>

Handling expected Exception in TestNG

The TestNG provides expectedExceptions option in @Test attribute, we can specify the exception class name to handle the known exception gracefully.

Example:

@Test(expectedExceptions = { IOException.class })

public void exceptionExample() throws Exception {

throw new IOException();

}

TestNG Parameterization

The parameterization is mainly used for data driven testing.

@Test

@Parameters(“somedata”)

    public void myTest3(String somedata)

    {

        System.out.println( “Smoke test” + somedata);

    }

Testng.xml

<test thread-count=”5″ name=”ExampleTest”>

<parameter name = “somedata” value = “Example Parameter”/>

<classes>

<class name=”com.testngdemo.demoTest.AppTest” />

</classes>

</test> <!– Test –>

Parameterization using Data Provider

public class AppTest {

@DataProvider(name = “somedata”)

public Object[][] exampleProvider() {

return new Object[][] { { “Value1” }, { “Value2” } };

}

@Test(dataProvider = “somedata”)

public void myTest3(String somedata) {

System.out.println(“Smoke test ” + somedata);

}

}

Example TestNG test

package com.testngdemo.demoTest;

import org.testng.annotations.AfterClass;

import org.testng.annotations.AfterTest;

import org.testng.annotations.BeforeClass;

import org.testng.annotations.BeforeTest;

import org.testng.annotations.Test;

public class AppTest {

int a, b, sum;

@BeforeClass

public void beforeClassDemo() {

System.out.println(“Inside Before Class”);

}

@BeforeTest

public void setUp() {

a=10;

b=20;

}

@Test

public void sumTest() {

System.out.println(a+b);

}

@AfterTest

public void teardown() {

System.out.println(“After Test”);

}

@AfterClass

public void afterClassDemo() {

System.out.println(“Inside After Class”);

}

}

Summary

The TestNG and JUnit are perfect tools for end-to-end and unit testing, which provide a lot of functionalities and customization. However, both require technical expertise and a good understanding of Java programming language. 

Recently, a lot of automation and testing frameworks are coming into the market. Testsigma provides Zero Code testing where you don’t need to write a single line of code. This way, any manual tester can jump into automation and start scripting. This is a win-win for both organizations and testers, where the organization doesn’t need to spend on upgrading the skills, and testers can easily switch to automation without knowing any programming languages. The effort gets further reduced as Testsigma provides integration with many CI/CD tools.


Test automation made easy

Start your smart continuous testing journey today with Testsigma.

SHARE THIS BLOG

RELATED POSTS