What Type of Test Cases should You Not Automate for Efficient Cross Browser Testing?
People use more than one browser to search for your queries online. If you are only focusing on one, then you are missing out. According to w3c, the most frequently used browsers are Chrome, Edge, Firefox, Safari & Opera. It is always advisable to run cross-browser test cases, at least on these. Alongside this, it is also advisable to know what type test cases you should not automate and which ones you should.
However, the process of analyzing and selecting the thousands of device, OS, and browser combinations requires time, expertise, and effort too.
Additionally, deciding which cross-browser test cases are ideal for automation requires proper analysis and expertise. Let us explore a few such cases in this article.
Find some helpful tips for cross browser testing here – Ultimate Guide to Cross Browser Testing
Table Of Contents
- 1 What Test Cases Can You Automate?
- 1.1 1. GUI test cases that test the appearance of elements on UI
- 1.2 2. Test cases that are dependent on respective browsers for proper functioning
- 1.3 3. End to End test cases that involve completion of workflow on a website/device
- 1.4 4. Performance test cases such as timely loading of the application
- 1.5 5. Functional & regression test cases
- 2 What Type of Test Cases Should You Not Automate?
- 3 How to Manage Automated Test Cases?
- 4 What Tool to Use for Automating Cross Browser Testing?
- 5 Conclusion
- 6 Frequently Asked Questions (FAQs)
- 7 Suggested Readings
What Test Cases Can You Automate?
Before we give you a description about what type of test cases you should not automate, lets’s discuss about the test cases you can automate.
1. GUI test cases that test the appearance of elements on UI
Features such as visibility, orientation, colour, text font are of utmost importance for the success of any software application. The compatibility of the code on different browsers, devices, OS combinations decides how the UI will be rendered on the front-end.
However, when done manually, these intrinsic details are difficult to test and report. Cross-browser automation testing is essential when there are minute UI details to be captured.
Examples of Cross-browser UI test cases –
- The layout of the page is rendered properly across devices.
- The text size, orientation, and colour are proper and visible.
- Images are visible, scaled as per page, and loaded timely.
Example – Transparent PNG images are not visible on old IE browsers.
- HTML and CSS rules are not causing any inconsistency in UI.
HTML5 and CSS3 are the latest technology and older browsers do not support them. Which may cause alignment and visibility issues for the UI elements. Therefore, test cases these rules and their impact should be automated for cross-browser testing.
- Elements like buttons, form elements, dropdowns, etc. are visible and clickable/accessible.
- The application content is visible on all browsers properly, as intended.
- Data tables have a layout as expected, are properly visible and scaled.
- On a slow network connection, the application should display the essential UI elements.
When to test?
The above points discuss the UI-related validations which we need to cover in cross-browser testing. As soon as one page is ready, we can start cross-browser testing to verify the above. Uncovered defects can be applied to all the pages which will be developed later, this will reduce redundancy in testing activities.
2. Test cases that are dependent on respective browsers for proper functioning
There are technologies that are browser-specific, and they work seamlessly on those particular browsers. But, we cannot restrict the users to use a specific browser – hence, cross-browser testing will be needed in this case to help us analyze how those technologies work on other browsers.
As mentioned above, in browser-specific scenarios, the other non-supporting browsers should degrade gracefully. This means that if an older or non-supporting browser doesn’t support a technology, then it should at least provide the basic functionality to users.
The application should not go completely blank or non-accessible by the user’s older version of the supported browser.
These test cases will be required to be run on many combinations, multiple times. We will understand the behaviour and fix the code, and rerun to see the results again. Test automation is required to perform this whole process efficiently and repeatedly.
Example of test cases to test on an older version of a supported browser:
Test cases that are dependent on specific browsers, and should be checked on other browsers as well-
Such scenarios require testing the application on IE to analyze and optimize the functionality on IE.
ii. Test cases to check quirk mode on older browsers:
All modern browsers today support ‘quirk mode’, which helps in providing backward compatibility for older browsers. However, this is not a standard like W3C, and the behaviour of the old browser is unknown.
Therefore, it is essential to test the application on older browsers to know their behaviour and fix code accordingly.
iii. Test cases to check Flash content on iOS
Flash player is still supported by Opera, Microsoft Internet Explorer, Microsoft Edge, Mozilla Firefox, Google Chrome. However, if an application has Flash content, then your iPhone is not going to support it.
Therefore, it is important to test the application behaviour on iOS, also make sure that essential content is not presented via Flash.
When to test?
The above issues are browser-specific, and a few of them will appear as soon as a few pages are ready. It is advisable to start testing when the smallest piece of code can be tested. The ones which require fully developed pages can be taken up for testing in later cycles of development.
3. End to End test cases that involve completion of workflow on a website/device
Complex applications have a complex workflow moving through multiple components and integrations.
There are multiple entities that are participating in a single process. So, their behaviour on different browsers(and their versions) can be different, which may lead to integration issues. This shows us that end-to-end test cases require cross-browser compatibility testing on multiple browsers.
Evidently, it is time-consuming and difficult to test it manually, when many devices, browsers, and platforms are also involved. Therefore, cross-browser test automation will be helpful in understanding the cross-browser as well as integration issues.
An e-commerce application, where there are below modules present which marks a complete workflow:
- Search the product in the application
- Add the product to the cart
- Check out and choose/add an address
- Payment is possible through multiple channels
- Integration of the payment channel and returning back to the application
- Creation of order
- Shipping of the order
- Integration with the order tracking system
- Delivery process
- Returns and refunds if applicable
Here, in this example, it is a complete end-to-end workflow, which is required to be tested as a whole. We cannot just go and test the ‘add to cart module and conclude that the application is cross-browser compatible.
Obviously, doing it end-to-end manually on thousands of devices is a painful and impossible task. Therefore, we require the help of cross-browser testing tools such as Testsigma to ease the effort and save time.
When to test?
Here, we are discussing the end-to-end flow and this requires the whole application to be ready for testing. Therefore, this can be carried out towards the end of the development cycle.
However, the cross-browser testing of the individual modules can be executed as and when they are ready.
4. Performance test cases such as timely loading of the application
An e-wallet application that takes time to load while selecting the payment option and making the payment will not be able to thrive in the market.
Performance is a very important marker of success for any application. Manually it is impossible to test the cross-browser performance of an application. Therefore, we require the help of automation tools to execute these performance-based cross-browser test cases.
When to test?
Mostly, it will be executed at the end of the development cycle. The reason being – we want to derive how much is the response/load time for the user. However, it can be started for a single page to be sure of the load time a single page takes.
If there are discrepancies then it can be rectified in the initial stages itself.
5. Functional & regression test cases
1. Functional tests: To identify the cross-browser compatibility issues which are tempering the application functionality.
2. Regression tests: They are essential to ensure that the new code changes are not causing any new defects.
A complex and large application will have a huge number of cross-browser functional and regression test cases. They will be run on hundreds and thousands of environment combinations.
Running these thousands of test cases manually is impossible – when automated cross-browser testing is used the time, effort, and cost are reduced drastically.
A complex and large project has 1000 functional test cases and 200 regression test cases for a cross-browser test suite. One cross-browser test case takes 10 minutes to execute.
Additionally, this needs to be run on 100 different environments(which is a small number, in reality, it is going to be way more than this).
Total time required to manually run test cases = 1200*100*10 = 1200000 minutes, which equals 20000 person-hours
When automated, these test cases can be run in parallel mode, simultaneously. Additionally, these test runs can be scheduled to run 24*7 without any manual intervention.
When to test?
Since these are functional and regression tests they will be executed at the end of the development cycle. If time allows, a few of the functional test cases can be executed in the middle of the SDLC to gain early insights.
What Type of Test Cases Should You Not Automate?
Not every test case can be automated. Even when your business is moving toward automating the testing process, you cannot simply forgo manual testing. Because, again, it is not possible and advisable to automate every test case. And you cannot as well. Here are the types of test cases you should not automate:
Ad-hoc testing is simply verifying the features of the application to check if they are working properly. It is informal/unstructured that does not follow any testing pattern to find defects that might have been missed in traditional testing. You cannot automate them because it is difficult to categorize the test cases.
User Acceptance Testing
Before users actually receive the product, a team of selected people uses the application in real-life conditions. It is to check if the product is stable enough for market release. And it is difficult to automate User Experience testing because software is unlikely to replicate exact user behavior and response.
Redundant, Low-Risk Test Cases
Just because you can automate a test case does not mean you should. Before automating, you must check if the process would need more time and resources. And if it does when compared to manual testing, then do not go for automation.
And at last, remember that some features simply cannot be automated. Security options, such as two-factor authentication and Captcha, are not possible or easy to automate.
How to Manage Automated Test Cases?
Managing Automated test cases can become tricky if the data is not handled properly or if there are no clear guidelines on application requirements. How will you know if you are running the right set of test cases and if the output is as desired?
In order to manage Automated test cases, you must take care of a few things:
- Decide the test cases you want to automate
- Select the right testing that aligns with your needs
- Assign testing tasks to your team
- Focus on creating good quality, reusable test date
Testsigma allows you to manage your test data and store it separately from the test case, so there is no contamination. Moreover, you can easily export and import test data to the platform to run your test cases.
What Tool to Use for Automating Cross Browser Testing?
Now, that we know what test cases to automate – it is essential to choose a tool that lets you automate your test cases efficiently. Here efficiency means that the test automation process when implemented, should result in ROI as soon as possible.
Testsigma is one such tool that is built with the focus on making test automation an efficient process for you from all angles:
- Test case creation is very easy: Test cases are automated in simple English and the process is very easy to learn and start.
- Test case maintenance efficiency improved substantially by supporting features like – self-heal, affected resources identification, reusable and groupable test steps, vidoes+screenshots+logs provided for failing test cases – making it easier to debug and fix.
- Can connect to the cloud device lab of your choice for test case execution – you don’t depend on one vendor all the time.
- Easily execute your test cases in Parallel and save on crucial time.
- Is completely cloud-based, can be accessed from anywhere, anytime
- Because test automation is so easy to create and understand – other team members can also be included in the test automation process for reviews.
- You can get started with test case creation in minutes.
Testsigma is giving a 21-day free trial – review all features before you decide
Cross browser testing is crucial for any application today. It validates the user experience, look and feel, efficiency, performance, and functionality of applications in different environments.
Identification of appropriate test cases for test automation is a daunting task and requires expertise. If the wrong test cases are automated, we will lose time in automation framework/test scripts development. Also, there will be no ROI on it. Therefore, it is advisable to make wise choices because losing time in SDLC is really expensive.
Frequently Asked Questions (FAQs)
Can you automate negative test cases?
Yes, you can automate negative test cases. But remember that you can only automate negative test cases for testing types that are discussed in this blog.
What type of test cases should you not automate in selenium?
Some of the test cases that you cannot automate in Selenium are HTTP requests, two-factor authentication, file downloads, and crash recovery.
Which types of tests can be automated?
Integration, Unit, regression, API, Performance, and Cross-browser testing are some of the usual types that you should automate using test automation tools to save time and manual labor.