Understanding Types of Frameworks in Test Automation

Automation testing involves frameworks that include test frameworks and guidelines to execute tests faster than manual testing. There are multiple automation frameworks available. Depending upon the application's complexity, the number of test cases, and the available resources, the QA team needs to decide the framework type.

The following post delves deep into the multiple automation frameworks. 

Linear Framework

The linear automation framework is the fundamental framework that is very commonly used. It is primarily used for testing small applications and is easy to understand. The test scripts are executed one after the other, following a particular sequence. Testers usually implement the framework to analyze a particular software feature or functionality. 

It is also referred to as a record and playback framework, as the testers record all steps like navigation and user inputs. These steps or tests are then played back to understand their efficiencies.

Moreover, the test scripts are executed individually, following a particular sequence. The method follows an incremental approach, where every new software interaction will be added to the test process.

So, the process is not very complex for inexperienced testers to handle and master. 

Benefits

• It does not require the tester to write custom testing code from scratch. 
• Extensive automation testing knowledge is optional.
• Test cases are easier to understand as they are arranged sequentially. 
• Fast generation of test scripts due to recording and play structure.
• Non-complex workflows reduce the need for rigorous test planning. 
   

Drawbacks

• Test scripts are not reusable due to hardcoded data geared toward a particular use case. 
• Test cases must be modified if the associated data is altered, which can increase complexity.
• The framework is not scalable, as modifications involve rebuilding test cases. It complicates the maintenance process.

Here's an example of linear-driven framework.

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;

public class TestModule {

  public static void performActions() {
    // Set up driver
    System.setProperty("webdriver.chrome.driver", "path/to/chromedriver");
    WebDriver driver = new ChromeDriver();

    // Go to login page
    driver.get("https://example.com/login");

    // Fill in login form
    driver.findElement(By.id("username")).sendKeys("testuser");
    driver.findElement(By.id("password")).sendKeys("testpass");
    driver.findElement(By.id("login-button")).click();

    // Go to search page
    driver.get("https://example.com/search");

    // Fill in search form
    driver.findElement(By.id("search-box")).sendKeys("test keyword");
    driver.findElement(By.id("search-button")).click();

    // Do something with the search results
    ...
  }
}

The TestModule is responsible for setting up the Selenium WebDriver and performing a specific sequence of actions on the website. This approach is simple and easy to use for straightforward test cases, but may become unwieldy for more complex test cases.

Modular-Based Testing Framework

The modular testing framework enables the tester to follow the “divide and conquer” approach. So, the tester can test each application section separately by dividing it into multiple modules. The process can be suitable for complex applications with many features to analyze. 

In some cases, testers need to break down or divide the entire software according to the client's requirements. A linear framework will not be advantageous in such situations, but a modular framework will be ideal. So, the application will be divided into separate modules, which will be tested individually. 

The tester will develop an individual test script for each module and combine the test with building a larger test representing the entire software. The process involves multiple steps such as test case generation, test execution report generation, and defect logging. 

Moreover, after the tester has written a function library, it is possible to store a test script. Particular adjustments or modifications can be made to this included script, so the entire app need not be modified. These features improve the convenience for both developers and testers.    

Benefits 

• Maintaining test scripts is easier and less complicated due to the modular structure. 
• The reusability of test scripts makes the framework cost-effective and flexible. Testers can conveniently edit individual sections of the application. 
• It is easy to include new functionalities and scale the software according to the project requirements. 
    

Drawbacks 

• Requires extensive programming knowledge and debugging expertise to identify failures in test steps. 
• Building up test cases takes little time for experienced programmers. 
• Application updates might cause problems as data is hardcoded into the test scripts.

Let's consider the 'MainTest' class is the main entry point that calls different modules (in this case, 'LoginModule' and 'SearchModule') to perform specific tasks. Each module is responsible for setting up the Selenium WebDriver and performing specific actions on the website.

// MainTest.java
import org.junit.Test;
import modules.LoginModule;
import modules.SearchModule;

public class MainTest {

  @Test
  public void test() {
    // Call LoginModule to login
    LoginModule.login("username", "password");

    // Call SearchModule to search for something
    SearchModule.search("keyword");

    // Do something with the results
    ...
  }
}

// LoginModule.java
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;

public class LoginModule {

  public static void login(String username, String password) {
    // Set up driver
    System.setProperty("webdriver.chrome.driver", "path/to/chromedriver");
    WebDriver driver = new ChromeDriver();

    // Go to login page
    driver.get("https://example.com/login");

    // Fill in login form
    driver.findElement(By.id("username")).sendKeys(username);
    driver.findElement(By.id("password")).sendKeys(password);
    driver.findElement(By.id("login-button")).click();
  }
}

// SearchModule.java
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;

public class SearchModule {

  public static void search(String keyword) {
    // Set up driver
    System.setProperty("webdriver.chrome.driver", "path/to/chromedriver");
    WebDriver driver = new ChromeDriver();

    // Go to search page
    driver.get("https://example.com/search");

    // Fill in search form
    driver.findElement(By.id("search-box")).sendKeys(keyword);
    driver.findElement(By.id("search-button")).click();

    // Do something with the search results
    ...
  }
}

Data-Driven Framework

A data-driven framework will be beneficial when professionals need to test a particular scenario or function having multiple data sets. Testing becomes simpler as the framework separates the internal test logic from the data set. 

This framework does not have data hard-coded into the test scripts, which is the primary difference between a modular and linear framework. So, when testers need to test a particular application feature multiple times, data decoupling becomes essential.      

It comprises the following steps - 

• Collecting information from multiple data sets and sources like .xls, .csv, and .xml on a single platform. 
• Designing test scripts to read the data and enter it into the respective application as input. 
• Comparing the test outputs with the expected outputs. 
• Repeat the process by using the information in the subsequent data row. 
   

For example, a data-driven framework can help test the functionalities of a website login page. In this process, the tester will create a test data file that will contain the inputs for the driver script. It will also contain the expected output and results in a separate table. Then, the input values and the operators will then be tested by the driver or test script. 

After comparing the expected results with the actual results received, the tester concludes the web page's functionality and efficiency.  QED42’s open-source product, Headway, is a data-driven framework built using Selenium, Testng and Java.

Benefits 

• Faster test execution by separating data and scripts. 
• Convenient testing of several data sets and better regression testing.
• The data does not get affected when modifications are performed on test scripts.
   

Drawbacks 

• Technical expertise and programming knowledge are required. Debugging might be difficult for inexperienced programmers. 
• Resource-intensive process and might involve complicated test data.   

Keyword Driven Automation Framework

The data and test script logic is separated in a keyword-driven framework. The data is stored externally, and every software function is included in a separate table. Moreover, specific keywords are assigned to particular actions used for GUI testing.

These keywords are arranged in a different table and represent GUI actions like click, log in or hyperlink. These are stored step-by-step and are associated with an object or area of the UI being tested. 

This is the major strength of the framework, as it makes the separate keyboards entities connected to functions. Testers write code to invoke a particular keyword-based action, and the corresponding test script will be executed.     

The key elements of any keyword-driven framework are – 

• An excel sheet to identify and store the keywords. 
• A function library comprising the functions which will be called during the test. 
• Data sheets for storing application test data. 
• An object repository for storing the objects associated with the keywords. 
• Single driver scripts or test scripts for every test case. 

Benefits 

• A highly flexible and reusable framework that does not require extensive programming knowledge. 
• Compatible with multiple automation tools and independent of languages or tools. 
   

Drawbacks 

• Difficult to configure and requires a managed environment.

Here's an example of a keyword-driven framework in Java using Selenium. In this example, the MainTest class sets up test data in a HashMap and calls the KeywordModule to perform actions based on the keywords in the test data.

// MainTest.java
import java.util.HashMap;
import org.junit.Test;
import modules.KeywordModule;

public class MainTest {

  @Test
  public void test() {
    // Set up test data
    HashMap testData = new HashMap();
    testData.put("username", "testuser");
    testData.put("password", "testpass");
    testData.put("keyword", "test keyword");

    // Call KeywordModule to perform actions based on keywords
    KeywordModule.performActions(testData);

    // Do something with the results
    ...
  }
}

// KeywordModule.java
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import java.util.HashMap;

public class KeywordModule {

  public static void performActions(HashMap testData) {
    // Set up driver
    System.setProperty("webdriver.chrome.driver", "path/to/chromedriver");
    WebDriver driver = new ChromeDriver();

    // Go to login page
    driver.get("https://example.com/login");

    // Fill in login form
    driver.findElement(By.id("username")).sendKeys(testData.get("username"));
    driver.findElement(By.id("password")).sendKeys(testData.get("password"));
    driver.findElement(By.id("login-button")).click();

    // Go to search page
    driver.get("https://example.com/search");

    // Fill in search form
    driver.findElement(By.id("search-box")).sendKeys(testData.get("keyword"));
    driver.findElement(By.id("search-button")).click();

    // Do something with the search results
    ...
  }
}

‍

Hybrid Test Automation Framework

A hybrid test automation framework is a combination of multiple testing frameworks. Depending on a particular framework is not the best option for most testing projects. A hybrid framework leverages the strengths of numerous frameworks to execute tests properly. 

It can be a combination of keyword-driven and data-driven frameworks. So, the test data and keywords can be externally stored. The test data can be stored in an excel file, and the keywords can be stored in a particular Java file.  

The hybrid framework enables testers to experiment and combine other frameworks to suit the testing requirements. The scalable approach supports different testing environments and improves application efficiency.

Benefits

• Combining the strengths of multiple frameworks, it takes less time to execute scripts. 
• Facilitates code reusability and maximum test coverage. 
• Better efficiency in testing with lower maintenance expenses.
   

Drawbacks 

• Sufficient programming proficiency and testing experience are required.

Summing Up 

Automation testing frameworks make the testing process smoother and allow testers to analyze the app thoroughly. By helping in bug detection and improving efficiency, these frameworks will boost the app deployment process. So, testers must select the automation framework according to their project or application requirements. 

Testing frameworks must be selected according to the technical experience and expertise of the testers. Factors such as scalability, flexibility, and speed also need to be considered. 

For instance, a linear framework does not need custom coding, but a hybrid framework will require coding expertise. Thus, careful consideration and interactions with senior professionals might be necessary for choosing a testing framework.

Headway is available for download on GitHub. Check out some of our automation strategy and testing work with global enterprises to know in detail how Headway minimizes risks and ensures impeccable quality at speed, helping our clients with faster go-to-market.