Data Driven vs Keyword Driven Frameworks for Test Automation

Data Driven vs Keyword Driven Frameworks for Test Automation

What is a Test Framework? 

A test framework is a set of guidelines which can be followed to create test cases and related processes. These guidelines can be about coding practices, storage and retrieval of test data and test results, interaction with external resources and many other things.

A test framework usually contains internal libraries and reusable code modules which provide a foundation for test automation and can be leveraged to build test automation systems for different types of applications.

Test Framework

Benefits of a Test Automation Framework

A test automation framework offers several benefits, some of which can be mentioned as follows:

  • Efficiency: A test automation framework provides a structure for the test automation tasks like test creation, test execution, organization of tests, test data creation or reporting.

    This helps reduce confusion, duplication of efforts and redundancy. Thus, using a test automation framework makes building a test automation system more efficient.

  • Easy Maintenance: As all test cases are created following the same structure, debugging and isolation of bugs becomes easier.
  • Reduced Manual Interactions: The main purpose of automating any process is to reduce manual labour and, implementing a test automation framework helps reduce manual labour that was involved in the execution of test cases.
  • Flexibility to Run Selective Test Cases: A test automation framework generally provides the flexibility to execute a single test case, a suite of test cases, or all test cases.

    Such flexibility helps during development and debugging sessions of test scripts and also for selective testing, for e.g. if a feature needs to be tested, only test cases related to that feature can be selected and executed.

Different Types of Test Automation Frameworks:

Test automation frameworks can be broadly classified as Linear Framework, Modular Framework, Library Architecture Framework, Data-Driven Framework, Keyword Driven Framework, Hybrid Framework, and Behaviour Driven Framework.

  • Linear Frameworks
  • In linear frameworks, test scripts are created in a record and playback manner which generates sequential test steps with hardcoded test data.

    Test scripts can be generated quickly with this framework, but they aren’t much reusable other than repeating the execution of the same test case with the same test data. This provides an easy way to learn a new tool or understand a test workflow since it doesn’t need coding expertise for a beginner, but not good for scaling purpose.

  • Modular Frameworks:
  • In modular frameworks, the application can be divided into different modules which can be tested independently.

    Test scripts can be reused to an extent, thus reducing the test script development time. Programming knowledge is required to work with these frameworks. Using object-oriented concepts, an abstraction layer can be developed which can help in easier maintenance of test scripts.

  • Library Architecture Frameworks:
  • .In library architecture frameworks, common tasks needed for test scripts are consolidated into a library of functions, which can then be called in different test scripts.

    Test scripts developed using this method are more maintainable and reusable. Technical expertise is needed to work with these frameworks.

  • Hybrid Frameworks:
  • In hybrid frameworks, more than one framework is used to achieve the desired objectives. The combination should be designed in such a way that weaknesses of one framework are compensated with the strengths of other frameworks.

  • Behaviour Driven Frameworks:
  • These frameworks allow the test cases to be written in such a format that they could be easily reviewed by non-technical stakeholders for a project for eg. Clients, Business Managers, Product Managers, Project Managers etc.

    The UI that could be reviewed is in a human-friendly language that could be easily understood. For eg. Cucumber, Specflow, Testsigma etc.

    What is Keyword and Data Driven Framework?

  • Data-Driven Framework:
  • In data-driven frameworks, test data is separated from test scripts and stored in an external resource such as text file, excel spreadsheet, CSV file or database table. These frameworks provide the flexibility of executing the same test script with multiple data sets.

    Data Driven Framework

  • Keyword Driven Framework :
  • In keyword driven frameworks, the test logic is divided into keywords and functions. A sequence of keywords is used to define the test scripts and these keywords are further defined as functions to implement the desired behaviour.

    Keyword Driven Framework

    An external table is generally used to maintain the mapping of keywords and functions, therefore this framework is also known as Table Driven Framework. For e.g. HP QTP and Selenium are widely used for keyword-driven testing.

    We shall now discuss the Data-driven framework and keyword-driven framework for test automation and also Data Driven vs Key Driven frameworks differences, in detail. 

    Need for Data-Driven Framework for Test Automation

    In non-data driven testing frameworks, test data is embedded in test scripts which makes it difficult to update the test data at any point of time. This becomes a festering problem in a product’s lifecycle; test data may need to be updated several times for various reasons.

    In data-driven testing frameworks, test data and test scripts are separated which makes it easier to maintain and update the test data at any point of time without affecting the test scripts. Similarly, any changes to the test scripts can be made without affecting the test data. 

    For example, if you need to test a video player, a very basic testing script would look something like this:

    For non-data driven framework for testing:

    Load A.mp4

    Click the Play button

    For data-driven framework for testing:

    Load media

    Click the Play button

    The Test Data file would look something like this:

    A.mp4

    B.mov

    C.avc

    Advantages of Data-Driven Frameworks in Test Automation

    Data-driven frameworks offer several benefits such as:

    • Less number of Test Scripts: As the same test scripts can be executed with multiple test data sets, the total count of test scripts required is greatly reduced.
    • Less Error-Prone: As test data and test scripts are different entities, each can be updated independently without interfering with others. This reduces the chances of introducing errors in the test automation system while updating either test data or test scripts.
    • More Flexibility: Since test data is stored externally, it is easier to perform positive and negative testing using the same test scripts just by updating the test data.
    • Easier Maintenance: Data-driven test automation systems are easier to maintain as compared to non-data driven test automation systems, where test data and test scripts are tightly coupled and cannot be updated independently.

    Data-Driven Frameworks Challenges in Test Automation

    Using a data-driven framework does come with its own challenges. A few of which are:

    • Programming Expertise: Testers need to have the good programming knowledge to write the test scripts.
    • Planning is Important: Test automation systems need to be well planned in terms of test scripts, test data, and their interactions.
    • Getting Started Takes Time: Getting started with a data-driven framework does take more time as compared to non-data driven frameworks where test scripts can also be made using record and playback manner. 

    Now, there are automation frameworks that offer codeless testing and easier integration with data files for data-driven testing. For e.g. Testsigma is one such tool.

    Here, no programming expertise is needed as the test cases can be created in Simple English and test data can be configured in the tool itself to enable data-driven testing.

    Testsigma is a tool that has made Data-driven testing very easy. Check here why you should choose Testsigma for your data-driven testing needs

    Need for Keyword Driven Framework for Test Automation

    With Testsigma, you don’t need to even setup keyword-driven testing because Testsigma lets you automate directly in simple English. Check out more details about Testsigma here:

    In non-keyword driven frameworks, writing and maintaining the test scripts becomes the prime responsibility of few automation experts in the team, whereas, the non-automation experts in the team who have a much better knowledge of the product are left out.

    This creates a bottleneck in the testing team’s productivity. Team members creating the test scripts are always under pressure to write more scripts to quickly build the test automation system.

    On the other hand, team members not creating the test scripts don’t have much faith in the quality of test scripts created as they know the product better than the team members creating the test scripts and try to depend on manual testing

    Keyword-driven frameworks help in eliminating these bottlenecks and create a balance where all team members with and without the programming knowledge can contribute to creating the test automation system of the product. 

    In keyword driven frameworks, test scripts are a sequence of keywords which are mapped to functions defining specific behaviours.

    Team members who are well-versed with the product can use these keywords to create the test scripts and team members who are automation experts can implement the functions corresponding to keywords to define the required behaviours. 

    Mapping of keywords and their corresponding functions can be maintained in an external resource such as a table.

    Anyone writing the test scripts only needs to refer to this external resource to see which keywords are available and anyone implementing the functions needs to update this external resource with the newly defined functions corresponding to the keywords. 

    Since writing the test scripts using keywords and implementing the functions mapped to keywords can be done independently, the team can work on both in parallel, which also helps in boosting the team’s productivity.

    For example, if you need to test a video player, a very basic testing script would look something like this:

    For a non-keyword driven framework:

    Load media

    Click Play button

    For a keyword-driven framework:

    Load

    Play

    Keywords and Functions Mapping would look something like this:

    Keyword Function Description
    LoadloadMedia()Loads the media in the player.
    PlayClick play()Clicks the play button in the player.

    Advantages of Keyword-Driven Frameworks in Test Automation

    Few of the benefits of keyword-driven frameworks are:

    • Application Abstraction: Test scripts can be designed with/without the application under test. This is very useful when test cases need to be developed before the application is ready for testing.
    • Enhanced Team Productivity: As keyword-driven frameworks allow everyone on the team to be part of the automation journey, it helps in boosting the team’s productivity and confidence.
    • More Suited for Agile: Keyword-driven frameworks are very helpful when products are developed using Agile methodologies.

      When there is a change in the product requirements, you only need to update the function’s implementation for accommodating the new behaviour rather than changing every test script.

    • Platform Independent Test Scripts: If the product behaviour is expected to be the same on different platforms, the same test scripts can be reused.

      Platform-specific changes would be needed only in the implementations of functions mapped to these keywords. Read all about Independent Testing

    • Code Reusability: Since multiple test scripts can be defined using the same set of keywords, the code is reused to a large extent.

    Keyword-Driven Frameworks Challenges in Test Automation

    Using keyword-driven frameworks has its own challenges, such as:

    • Keyword Management: Inventing and managing unique meaningful keywords for various types of actions becomes a daunting task for large projects. This sometimes poses problems for scaling.
    • Planning and Management are Crucial: If a keyword-driven framework is not planned ahead it could lead to the creation of keywords that are not useful in the long-run or may need too many edits every time a keyword is added.

      Thus, thinking about what keywords to implement and how becomes a crucial step to make the framework productive and efficient.

    • Programming Expertise is Required: To implement the functions mapped to keywords, you still need team members who have good knowledge of programming and test automation to make the whole system work together.

      Though, this problem can be easily solved if a tool that does not require coding expertise is used. For eg. Testsigma.

    What is the Difference Between Data Driven and Keyword Driven Frameworks?

    We can summarize the differences between Data-Driven vs Keyword Driven Frameworks as follows:

    • Need for Programming Expertise in the Team:

      Though both the frameworks need programming expertise in the team, keyword-driven frameworks have the ability to also leverage non-programming expertise in the team, such as manual testers who lack programming expertise but are well versed with the product.

      This allows everyone in the team to contribute in building the test automation system of the product they are working on. On the other hand, data-driven frameworks don’t allow such flexibility. To build a test automation system using a data-driven framework, you need programming experts in the team who can write test scripts using a programming language. Non-programming experts of the product don’t have many opportunities here to contribute in building the test automation system of the product they are working on.

    • Need of Product for Developing Test Scripts:

      It is easier to develop test scripts using the keyword-driven framework when the product development is not complete, only the implementation of keywords depends on actual product development.

      On the other hand, to develop test scripts using data-driven frameworks, the actual product is needed.

    • Planning:

      Keyword-driven frameworks need more extensive planning as compared to Data-driven frameworks. With data-driven frameworks, you only need to plan for what test data and test scripts are needed.

      With keyword-driven frameworks, you need to plan for keywords and their implementations along with test data and test scripts.

    • Management:

      If not properly planned, test automation systems using keyword-driven frameworks could be much harder to manage as compared to data-driven frameworks.
    • Maintenance:

      A well-planned keyword-driven test automation system is easier to maintain as it has well-defined layers of abstraction between test data, test scripts, keywords and their implementation.

      On the other hand, a data-driven test automation system has abstraction only between test data and test scripts.

    Now, understanding the Data driven frameworks vs Keyword driven frameworks difference, let’s know different factors that can be considered in selecting the right testing framework for your product. Let us explore them.

    When to use Data Driven Frameworks?

    • Same behaviour needs to be tested with different sets of test data: If your product needs to test the same behaviour with different sets of test data, a data-driven framework can be a good option.
    • Test data is expected to change often: If you know test data is expected to change, there is no point in keeping it embedded in your test scripts. A data-driven framework is your need here.

    When to use Keyword Driven frameworks?

    • Programming expertise in the testing team: If all the members in your testing team are equally good at programming, you can choose a non-keyword driven framework.

      If your testing team has members who are not good at programming but have good knowledge of the product to be tested, you can choose a keyword-driven framework so that everyone can contribute in creating automation for the product.

    • Product behaviour is expected to change: If you know there can be a change in the way some features of the product are expected to behave, a keyword-driven framework can be your option as you would only need to update the implementations of few functions at one place rather than changing multiple test scripts. 

    Data Driven vs Keyword Driven Frameworks Example

    Depending on your need you can also choose to have a hybrid framework, where you can combine data-driven and keyword-driven frameworks to take advantage of both.

    For example, if you need to test a video player, a very basic testing script would look something like this:
    Load

    Play 

    Test Data file would look something like this:

    A.mp4

    B.mov

    C.avc

    Keywords and Functions Mapping would look something like this:

    Keyword Function Description
    LoadloadMedia()Loads the media in the player.
    PlayClick play()Clicks the play button in the player.

    Managing and scaling such frameworks over time may bring its own challenges along with the benefits.

    Conclusion:

    Different types of testing frameworks may be suitable for different types of products and teams.

    Before finalizing any framework for building your test automation system for the product, it is important to make an analysis of what you need from a framework and what are your strengths as a team to work with the framework.

    This would help you in choosing the framework that’s right for your team and product. 10 Points to Help You Choose the Right Test Automation Tool

    Frequently Asked Questions

    Which is Data Driven and Keyword-Based Tool?

    Data-driven tools allow pulling data from any given source and filling it inside the testing application wherever and whenever data is needed. Data-driven tools speed up the software testing life cycle. Keyword-driven tools help set up the flow of action required during the software testing life cycle. Keyword driven tools help to access software elements using plain English keywords, which otherwise require the element location using locators.  

    What is meant by Data-Driven Testing?

    Data-driven testing is an approach where various types of test data inputs are collected in one place and extracted by iterating during the test case execution in sequence. In data-driven testing, stored input data can be in various formats like excel, CSV, etc.

    What is meant by Keyword Driven Testing?

    In software testing, targeting any element requires providing its location, which is done using locators like- XPath, element id, class name, tag name, etc. In keyword-driven testing, words written in plain English that this keywords are used instead of using locators to target any element of the software.

    What is the difference between Data Driven and Keyword Driven Testing?

    Data-driven testing is about iterating over a set of data and using it as user input instead of manually providing the data to test whether the test cases are passing or not.

    At the same time, keyword-driven testing is about using keywords instead of locators to target the element within the software getting tested.

    Is Gherkin Keyword Driven?

    Yes, it is a keyword-driven language. Gherkin is a language that defines business behavior without going into much detail about execution. Text in Gherkin behaves as documentation for the test automation.

    Suggested Reading

    What is meant by data-driven testing?

    What is difference between data driven vs keyword driven testing?

    How Data-Driven Testing Works?

    How to Create-A-Data-Driven-Automation-Framework?

    Data Driven Techniques

    What is Keyword Driven Test Automation

    Top Benefits of Data-Driven Test Automation


    Test automation made easy

    Start your smart continuous testing journey today with Testsigma.

    SHARE THIS BLOG

    RELATED POSTS


    Data driven techniques: How to measure when testing is enough
    Data Driven Techniques to Measure How Much Testing is Enough
    Scenarios when Data Driven Testing is useful
    Scenarios When Data-Driven Testing Is Useful
    Top benefits of data driven automation
    Top Benefits of Data-Driven Test Automation