Random Testing In Software Testing – An Overview

Imagine a kid playing around with a laptop, tapping keys randomly on the keyboard, and you watch and observe how the system responds to the random inputs. Sounds chaotic, right? But that’s the essence of random testing.

It is critical to ensure the quality and reliability of applications. While structured testing techniques like functional and regression testing are well-known, they add a fun and unconventional twist to the testing process.

In this blog, we will discuss what it is, its pros and cons, and the tools that will help you automate. 

What is Random Testing?

Random testing is a type of functional black box testing technique in which test inputs are generated randomly without following any specific test design or test case specification. It is also known as monkey testing. 

Here, random inputs are used to test the program. The test results are then compared to the expected outcomes to determine if the test passes or fails. 

Important Points About Random Testing

• It was first explored by Melvin Breuer in 1971. 
• In 1975, Pratima and Vishwani Agrawal evaluated this testing method to verify the software’s successful output.
• There is also a book for random testing that provides formulas for the number of tests to perform and the number of successful and failed results.

The Fun Fact About Random Testing

Random testing is where testers throw random inputs and check how the system processes the inputs. It is also known as Monkey testing, Fuzzy Testing, and Stochastic Testing. 

Here, the tester involved in the testing is considered a monkey. They do not require any prior understanding or knowledge about how the product works. Also, this testing is performed when the tester does not have much time to create and execute test cases.

It is based on the Infinite Monkey Theorem by Émile Borel. “The infinite monkey theorem states that a monkey hitting keys at random on a typewriter keyboard for an infinite amount of time will almost surely type any given text, such as the complete works of William Shakespeare.” 

How Random Testing Works?

The aim of random testing in software testing is to explore different scenarios in the software by using a random approach, which might uncover hidden defects and edge cases that traditional testing methods may not cover. Here is how random tests work,

Step 1: Determine the input domain

Step 2: Select the inputs randomly/independently from the input domain

Step 3: Use these inputs to test the system and create a random test set

Step 4: Analyze and compare the test result with the software specification

Step 5: If the test report fails, then take the required action.

The image below depicts how this testing works,

Random testing in Test Automation

In test automation, random testing refers to the technique where the automated testing tool generates random inputs and actions to test the application. 

Testsigma is a powerful test automation platform that allows you to automate web, mobile, and desktop apps and APIs in one place and has a function to generate random test data.

Testsigma generates random test data for test cases. To do this, all you have to do is invoke the ~|| format in the test data field.

For example, if you share the random test data as ~|20|, it will be later replaced by a 20-character-long alphanumeric string during test execution.

Testsigma offers a 21-day free trial, try Testsigma Cloud and start automating your tests 10x faster. You can also get a personalized pricing quote here.

Testsigma is both open source and Enterprise ready. The open-source version is also available for free. You can take advantage of the extended 24×7 customer support Testsigma offers its users across email, chat, phone, and community.

Automate your random tests using random test data generation from Testsigma

Advantages

Let’s discuss some of its advantages,

• Random testing is inexpensive and accessible to everyone.
• It doesn’t require in-depth knowledge of the program being tested.
• It works well even when specifications are not clear.
• Errors are easily identified, making it simple to trace and fix bugs.
• It can be automated, allowing more test cases to be executed.
• It is performed from a user’s point of view.
• No coding skills are required to perform this test.
• It does not have any bias. Unlike manual testing, this testing doesn’t overlook bugs due to misplaced trust in certain code sections.
• It provides diversity in testing inputs and helps identify defects caused by rare or uncommon input combinations.
• It can be effective in detecting common defects that occur with unexpected inputs or corner cases. It helps detect unexpected behaviors and potential vulnerabilities.

Disadvantages

• It can detect only basic bugs and might miss complex issues.
• Testers might not be able to recreate bugs. 
• It is not very effective at finding bugs when compared to other testing techniques.
• Some tests may become impractical and irrelevant over time.
• Analyzing all the tests takes up a lot of time.
• It is not possible to create new tests if their data is not available during testing.
• It offers only limited test coverage. There is no guarantee that all critical functionalities and edge cases will be tested.
• It can cause problems for continuous integration if different inputs are used in each test because the same tests may pass/fail randomly.
• Few believe white box testing is better than random testing.

Types of Random Testing

It can be categorized as follows,

1. With Respect to the Input

• Random input sequence generation 
• Random sequence of data inputs
• Random data selection from an existing database

2. Guided vs Unguided

• Undirected random test generation
• Directed random test generation

Let’s discuss this in detail below,

Random input sequence generation 

This type generates random sequential numbers, characters, or symbols that are not predetermined. It ensures unpredictable and varied input scenarios during testing. It is also called the Random Number Generator (RNG).

Random sequence of data inputs

Here, test data is randomly selected for testing purposes. All the data used in testing are chosen without any specific order or pattern. It is sometimes known as stochastic testing.

Random data selection from an existing database

Test data is randomly selected from an existing dataset to validate real-world scenarios. Only the existing data from the database will be used for testing, and no additional data can be added for testing.

Undirected random test generation

This type of testing involves generating test cases randomly without any heuristics or specific guidance to guide the test generation process.

Directed random test generation

On the other hand, directed random testing uses various techniques to steer the test generation process. For example – Feedback-directed random test generation and Adaptive random testing.

Tools Used for Random Testing

While there are not many specific tools exclusively available for random testing, there are a few testing tools that testers can use to specifically implement it. Take a look at the list below to know which all tools support random testing,

1. Testsigma

Testsigma is a robust cloud-based low-code test automation tool that lets you automate your web, mobile, and desktop apps and APIs from the same place. It supports over 3000+ devices/tablets and 1000+ browsers/OS combos. You can automate different types of testing like GUI, API, cross-browser, regression, portability testing, and more from one place.

Testsigma generates random test inputs for your automated test scripts. It also offers a free tool to generate random data for your test automation.

Automate your random tests using test data generation from Testsigma

2. QuickCheck

John Hughes created QuickCheck in 1999, a popular Haskell testing library. It is a popular tool developed originally for testing Haskell programs but is now available for other languages, including Scala, Java, and Erlang. 

• It helps formulate and test the properties of Haskell programs. 
• It generates random sequences of API calls based on a model and then checks if specific properties of the system are true after each run. 
• Ensures the correctness and reliability of the software by testing various scenarios automatically.

3. Randoop

Randoop is a popular unit test generator, especially for Java. It automatically creates unit tests for your classes in JUnit format. It uses feedback-directed random test generation to create unit tests. Testers use Randoop for two main purposes,

• To find bugs in the program 
• To create regression tests that help ensure that the program’s behavior remains consistent even if any changes are made to it in the future.

4. Simulant

Simulant is a Clojure library created by Stuart Halloway for simulation testing. All inputs, outputs, runtime, and other information are stored in a database, and it is thus possible to compare runs at different times and even to perform new validations on old data.

It uses a probabilistic model to simulate tests. All the data, like inputs, outputs, and runtime, are stored in a database. This allows testers to easily compare test runs at different times and validate old data with new tests.

5. AutoTest

AutoTest is developed by EiffelStudio that helps developers to create, manage, and run tests. It allows users to control things using two components,

• AutoTest Interface – Allows users to access, manage, execute, and monitor tests.
• Eiffel Test Wizard – Allows users to create new tests.

6. YETI

YETI, abbreviated as York Extensible Testing Infrastructure, is a language-agnostic tool. 

• It is a popular automated random testing tool that allows people to test programs written in different programming languages like Java, C, .NET, JML, CoFoJa, Kermeta, etc.
• It is one of the fastest tools that can perform more than 1 million method calls per minute on a fast code library.

7. GramTest

GramTest is a grammar-based random testing tool written using Java. 

It uses BNF notation (Backus–Naur Form or Backus Normal Form) to define input grammar. It lets you create test cases based on arbitrary user-defined grammar. 

Summary

Random testing is a valuable and intriguing testing technique that brings a fresh perspective to traditional testing techniques. It helps identify unexpected defects, vulnerabilities, and edge cases that other testing approaches might miss.

This will definitely not replace the structured testing methods. 

Instead, it serves as a complementary strategy to improve test coverage and increase the chances of discovering hidden bugs. Its unpredictable nature allows testers to simulate real-world scenarios and challenge the software in ways that may not have been expected during the other testing methods.

Frequently Asked Questions

Is random testing a black box?

Yes, random testing is a functional black-box testing technique where a program will be tested using random inputs. Here, the testers will have no idea about the product, and they will just test the product with independent inputs to analyze how the system responds. The result will be compared against the product specifications to verify whether the test output is a fail/pass.

What is an example of random testing?

Random testing is where random inputs will be used for testing the product. Let’s take a look at how it will be performed on an online calculator, where we will test its basic arithmetic functionality like addition, subtraction, multiplication, and division,

Step 1: Navigate to the online calculator application. 

Step 2: Enter random inputs for basic addition/subtraction/multiplication/division (Example – 2 and 5)

Step 3: Click on the = button to get results. 

Step 4: Analyze how the system responds and report any bugs or unexpected behavior. 

Step 5: Repeat the steps from 2 to 4 with different username and password combinations to get in-depth test results.