Cross-Platform End-to-End Testing – Explained with Real-life scenarios

What is End-to-End testing?

The definition of end-to-end testing is as simple as “testing the application from one end to another”. A lot of the time people do generalize end-to-end testing as “testing done to test the application from beginning to end”. However, in an application, you may certainly define a beginning (generally the home page) but there can be a lot of endings.

For example, one eCommerce application’s end-user flow may look like this:

  • Add items to cart.
  • Tap the cart button and proceed towards the cart items.
  • Proceed to payment.
  • Cancel the payment.
  • Return.

Another flow may look like this:

  • Add items to cart.
  • Tap the cart button and proceed towards the cart items.
  • Proceed to payment.
  • Pay successfully.

These two scenarios represent two of the many ends that we might get while operating on the application. All such scenarios when tested thoroughly are termed under end-to-end testing. It is not a straightforward or “defined” process. In many cases, it resembles the exact behaviour you might see the end-user performing.

Apart from testing the end-to-end functionality, this type of testing also help us uncover a few bugs that would have been hard to find. For example, what impact does the disabling of network connection might have when the user is making the payment? Or what would happen if you start booking the same hotel on the same date twice in a row? These are a few possibilities and scenarios that can only be discovered through end-to-end testing.

What is Cross-Platform End-to-End Testing?

When done on a particular device, end-to-end testing has one small problem – you are just checking the test cases on a single platform. What if you are using Google Chrome on Windows and saw no bug but the actual bug was due to some API that couldn’t be called on a macOS? Or what if a user starts interacting on multiple devices with your application? Will the data remain consistent?

Similar to methods like cross-browser testing, cross-platform end-to-end testing caters to executing the end-to-end tests that need to be executed on multiple platforms. This is a critical part of end-to-end testing and is always recommended for each version of the application. 

Since the dawn of this century, device manufacturing has seen a surge like never before. In 2021, 60 mobile phones were released in the market and every four weeks a new Chrome, Firefox, and Opera version are released. Combine this with desktop releases and different mobile operating system versions. The combination is huge and the error probability is a bit high.

Another problem in cross-platform end-to-end testing is the user’s expectation that his account data is not local to the device but consistent across all the devices. For example, if we take the same example taken in the previous section with a cross-platform twist, it looks as follows:

On a mobile device:

  • Add items to cart.
  • Tap the cart button and proceed towards the cart items.
  • Exit the application.

On a desktop device:

  • View items in the cart.
  • Verify if all the items added through mobile devices are present.
  • Pay successfully.
  • Exit the application.

If such scenarios can be tested across various platforms on the same account, you have cross-platform end-to-end testing. Though, this has, till recently, been a tough scenario to automate – Testsigma has come up with a new feature to let you do just that. Testsigma now lets you automate your test scenarios on multiple platforms as one end-to-end test case.

Register for the webinar to demo Testsigma’s cross-platform end-to-end testing feature on 29th June 3pm IST.


When should cross-platform end-to-end testing be automated?

The following scenarios might help you understand the need for automation in cross-platform end-to-end testing:

When you have a cross-platform application

Have you ever wondered how your WhatsApp messages and media are restored locally when you log in through a new or another device? This is a cross-platform scenario and is an important feature for those who are looking to serve their users on various platforms. This is generally the case when account creation is required.

Synchronization among the devices is an expected feature nowadays. For example, a user receives an Amazon link from his friend on WhatsApp and adds it to the card on the phone and a few hours later checks it from the desktop. End-to-end testing adds a few more test cases when such a website is under release.

Testsigma now lets you automate your test scenarios on multiple platforms as one end-to-end test case.

Register for the webinar to demo Testsigma’s cross-platform end-to-end testing feature on 29th June 3pm IST.


When target devices are too many

Target devices are those devices that you intend to release your application. In 2022, you cannot release an application (native or web) for all the available devices. It would just mean a lot of sacrifices in terms of functionalities. So, we create a simple list of target devices and other specs we need to follow.

For example, if you are aiming to release a web application, you need to declare the minimum operating system version (android and iOS) you will be supporting. All the mobile browsers with their minimum versions and minimum resolution support. A similar pattern then needs to be followed for desktop as well. This will create a long target device list which would be a lot easier with automation in place.

When too many repetitive tests are there

A simple and general definition of repetitive test cases can be as:

  • Cases that need to repeat on various devices.
  • Cases that need to be repeated along with each version.

The second category, however, covers the first category as well. Repetitive cases (also termed regression test cases) ensure that with each releasing version and on each unique platform, we are releasing our application bug-free.

Automation is often called for when repetitive cases are in consideration. With the help of automation, these are put on the CI pipeline and run automatically to save time and costs.

When timelines are strict

Automation saves a lot of time from the period of pushing the application for testing till its release. When we are looking to test with automation, we execute cases faster and are able to quickly generate reports as well. One area that we discussed above is regression tests. Another important area is when we need to parallelize stuff. With advanced thread methods and leveraging a computer’s core power, we can move various mutually exclusive tasks together. Both of these things save time and help wrap things faster.

Real-life end-to-end testing scenarios

The final section before wrapping up this post is to introduce you to a few real-life scenarios that can help you identify more clearly the methodology of end-to-end testing. As mentioned in the previous section, there can be many ends in one application. Therefore, consider the following as one of the combinations.

Scenario 1: Online streaming website

The first scenario we take is of the streaming website (similar to Netflix). In this scenario, we would take the popular case when the user can watch a show or a movie from various devices. 

As a step-wise process, it would look as follows:

  • Login to the website.
  • Search for the show/movie.
  • Start streaming.
  • Verify if the streamed content is available as per copyrights.
  • Check for VPN based services.
  • Play the stream.
  • Exit at some random point.

Change the device:

  • Login with the same credentials.
  • Search for the same show/movie.
  • Start streaming.
  • Verify if the media is played from the same point where the user left on the other device.

Why automate an online streaming website: The above scenario is just one of many scenarios you can encounter in such an application. Automation here could also be necessary to check the load the app can take. Streaming applications can see a huge spike in traffic if any popular show is about to release. Also, streaming can be done on a very large range of devices such as mobile, tablets, TVs, casting devices, laptops, and desktops. Therefore cross-platform end-to-end automation could be important.

Scenario 2: Movie ticket booking

In a movie ticket booking website (or mobile app), we consider the scenario where a person is booking a movie ticket very close to the show timings from two different devices.

As a step-wise process, it would look as follows:

  • Open the website on the desktop and log in.
  • Use geolocation to detect the current city.
  • Give the user to type the city name manually.
  • Show all the available movies at that location.
  • Verify if movie names are correct.
  • Give an option to book tickets.
  • Share the show timings.
  • Verify if the show timings are correct.
  • Let the user select the timing.
  • Fetch the seating plan according to the theatre and show.
  • Verify the plan.
  • Let the user select a seat.
  • Exit.

On a mobile device: 

  • If the notification feature is there, check if the notification arrives to continue booking tickets on the mobile.
  • Open the application.
  • Check if the previously selected seat is still locked.
  • Proceed for payment. 
  • Exit.

This one scenario could be experienced by hundreds of people who book their tickets simultaneously. Hence, here too, we need to consider high loads. Apart from this, movie ticket booking is something that can be either done days before the show (hence you have time for desktop), urgently (maybe tablets) and on the go (mobile devices). Hence cross-platform end-to-end testing is necessary in movie ticket booking with automation due to a high number of cases.

Scenario 3: Chat application

The last scenario we consider is a video or chat application where the user is chatting under varied network conditions (or on the move).

As a stepwise process, it would look as follows:

On a mobile:

  • Open the application (or website).
  • Send and receive a couple of messages.
  • Verify if the message is sent successfully.
  • Change the bandwidth of the network.
  • Verify the time for each type of message (text and media).
  • Cut the network connection.
  • Send a couple of messages.
  • Turn on the network and verify if messages are successfully sent.
  • Exit application.

On a desktop:

  • Open the same application in the desktop mode.
  • Check if your messages are synchronized.
  • Send and receive a couple of messages.
  • Verify if the message is sent successfully.
  • Exit application.

Instant messaging applications have a huge load on them. At any point in time, there are many messages following from person to person or from multiple persons to a single person. The importance of such an application is huge in delivering information in an extremely short period of time. This should be equally efficient and consistent across various devices. Hence cross-platform end-to-end testing using automation becomes important to verify their perfect working on each device.

Conclusion

When we test a single function of an application, we just test a part of a big system. This does not mean that the big system will be as good as its single unit. For a book having all its pages printed clearly does not mean the book will hold them until bound properly. This calls for a need to test the application similar to what an end-user would be facing. Such a method we call end-to-end testing.

Today, end-to-end testing has become an important part of the testing process because of the internet accessibility on a large range of devices. However, the long list of target devices and browser versions has raised an issue of high manual labor.

As of today, if you are running on timelines, you might never be able to deliver if you choose to go manually. Therefore, we make use of automation in cross-platform end-to-end testing. But automation is only as good as the plan you make and the tool you use. In this post, we covered all these segments along with some real-life scenarios that will help you clear out the meaning of end-to-end testing and execute these tests confidently in your next project.


PC: Qustion mark vector created by freepik – www.freepik.com


Test automation made easy

Start your smart continuous testing journey today with Testsigma.

SHARE THIS BLOG

RELATED POSTS