7 Top Device Farms to test your iOS and Android applications

Device farms are used to simulate the testing process of iOS and Android applications on multiple device platforms. Subsequently, you can cut labor costs and time required by using some of the best device farms for testing. Hence, it helps you achieve greater test coverage and eases your DevOps mobile pipeline.

In recent times, there has been a gradual increase in the range of mobile devices. We see devices with different features, OS, screen sizes, etc. Therefore, it’s natural that we need to put more effort into testing our applications on each of these devices. We want our applications to work flawlessly on every possible device the user might use.

With all these new versions of Android and iOS devices available, manual testing will take a toll on your budget and labor hours. Hence, there was a need for a platform that would allow you to perform simulated manual testing on a broad range of devices all in one place.

What are device farms?

When the device fleet rose in earlier times, Nokia started an initiative to make the lives of testers somewhat easier. They would allow them to run their applications on rented devices in a remote-controlled mode. However, we could not scale it to the extent of mobile variations that we have now.

As a further development, now the OS itself could provide tools that would allow testers to simulate user actions on applications. Consequently, they can write scripts that can be executed on mobile phones running in developer modes. These scripts would easily imitate user actions on devices.

Subsequently, they built DevOps tools that automatically build and run these scripts on thousands of mobile devices. These devices would have the target applications on them and run in developer mode.

A fleet of such devices with different variations running on remote cloud labs is called a device farm.

Benefits of mobile app testing on device farms

If you are working at scale, testing your mobile applications on multiple devices becomes really expensive and time-consuming. On device farms, you have access to thousands of devices with different configurations.

You can leverage emulators and simulators to perform your testing. Cloud-based testing also allows you to capture performance issues. Here are some of the most important benefits of using device farms for mobile app testing.

• You have lots of devices with different OS platforms, screen orientations, display sizes, memory, network, etc. It reduces the overall maintenance and infrastructure cost.
• You can perform parallel testing which will save you tons of time.
• Cloud-based environments are usually secure and they can be accessed from anywhere.
• You can get detailed reports in the form of graphs, video recordings, screenshots, etc.
• With the help of all the reports, it becomes easier to spot bugs and fix them.
• You can easily integrate them with your CI/CD pipeline, making collaboration easier.

Best mobile device farms for testing

On a lower level, all the device farms use some kind of special script. This script is launched by the OS to simulate actions like gestures, button presses, touch, etc.

The main difference is the subsystems used by these device farms for executing these scripts. Some of the popular ones are – Espresso, Appium, Calabash, UI Automator, Robotium, XCTest, etc.

In this article, let’s discuss a mix of free and professional device farms and test labs.

1. AWS Device Farm

This is the most widely used and best mobile device farm for testing currently available in the market. It lets you test concurrently on multiple devices to speed up the execution of your test suites. It also generates logs and videos for extensive debugging.

Some of its core features are –

• A large number of devices with options to choose memory, CPU usage, location, carriers, etc.
• Videos and logs to reproduce the bugs faster.
• Fine-tune real-world environments by setting location, network, language, etc.
• Execute built-in test suites or choose frameworks like Appium, Espresso, etc.
• Several plugins and APIs to integrate with your CI/CD, like Jenkins, Android Studio, etc.
• Set up private device labs to avoid long waiting queues.

For script executors, you can use Appium (iOS + Android), Espresso and Robotium for Android, and UIAutomation and XCTest for iOS.

It offers a pay-as-you-go pricing model at $0.17 per device minute. You can also opt for a monthly flat rate starting at $250. If you want to use exclusively dedicated devices, it will cost you a minimum of $200 a month.

2. Sauce Labs

Sauce Labs allow you to expedite live and automated mobile app testing on public and private device clouds. You can use Android emulators and iOS simulators that are highly reliable, cost-effective, scalable, and optimized for your CI/CD pipeline.

Some of its cutting-edge features are –

• You can easily achieve beta, live, automated, and API testing using high volumes of real device tests with minimum error rates.
• It allows you to improve test coverage combined with high parallelization of tests.
• You will have access to features such as unlimited screenshots and videos, single-use VMs, secure local tunneling, REST API access, tons of debugging tools, etc.
• It lets you debug faster with the help of rich assets of real-time tests, user insights, error monitoring, and reporting.
• You can easily integrate it with your CI/CD pipeline and collaborate with teams and stakeholders with faster feedback loops.

Similar to AWS device farms, Sauce Labs also allows you access to the best script executors including Appium, Espresso, and XCUITest.

It has three pricing plans which vary based on the number of parallel tests you choose. For one parallel test, the live testing plan would cost you $39 a month. A virtual cloud plan would require you to pay $149/month, and for a real device cloud plan, they charge $199/month.

3. Firebase Test Lab

This has been developed by Google and is specifically designed for testing Android applications, but you can also test iOS apps. It uses real production devices for app testing running in Google data centers.

They have customizable locale settings and updated APIs. You can easily road-test your app by configuring the hardware and software configurations. Some of its key capabilities are –

• You can run all kinds of tests on a wide range of iOS and Android devices hosted on Google data centers.
• Firebase Test Lab is integrated with Android Studio, Firebase console, and gCloud CLI. You can even integrate it with your CI pipelines.
• You can use Espresso, Robotium, UI Automator 2.0, and Robo tools to execute scripts.

The steps to use the Firebase Test Lab are simple –

1. You have to choose a test run – Instrumentation or Robo test for Android, GameLoop, or XCTest for iOS.
2. Then, build and package your application and upload it to firebase.
3. Customize your test matrix by choosing the devices, locales, screen orientations, and OS versions.
4. Run the tests and view the reports on the firebase console.

It provides testing and cloud API quotas included in Blaze and Spark testing plans. The spark plan is free and can perform 15 test runs in a day. The blaze plan determines the cost based on the time taken for test runs.

It is free for up to 30 minutes of test runs on physical devices and 60 minutes on virtual devices in a day. Anything past this will cost $5 (physical devices) and $1 (virtual devices) per hour.

4. Samsung Remote Test Lab

Needless to say, Samsung owns a big chunk of the world’s smartphone and Android market. Hence, it’s worth taking note of the remote testing lab provided by Samsung.

This device farm contains all the flagship Samsung smartphone devices. Additionally, testers will have access to innovations even before they have rolled them out to market. This will allow you to test the operations of your applications on devices that have not yet even launched.

Not only this, you can even perform tests on Tizen-based devices like Galaxy smartwatches and Tizen mobiles.

There are more than 30 smartphone and tablet models to perform your tests on. It allows you to perform a manual recording of all the events and steps that you want to test in the same sequence. Besides, it enables you to perform remote debugging.

The Samsung device farms are completely free to use. However, the only constraint is that you can only test your application on Samsung smartphones.

5. Xamarin Test Cloud

The Xamarin device farms are a part of Microsoft’s Visual Studio App Center. It allows you to run your tests on over 2500 mobile devices. You can upload your application and tests on the Xamarin Cloud using Visual Studio. This will automatically install the application on the environment and prepare your test runs.

Next, you can decide on the devices you want to run your tests on. You can select multiple mobile devices in one go and execute your tests on all of them. Once you have decided on the devices, you can choose the system language and the test framework you want to use.

Xamarin offers frameworks such as Appium, Calabash, Espresso, and Xamarin.UITest to execute your test scripts. When the test runs are completed, it will send out a notification to all the registered users with detailed test reports.

You have 30-day free access to the visual studio app center, after which you have to pay as per usage. The package starts at $99 per month.

6. Kobiton

With Kobiton, you can access hundreds of real devices in the cloud. With respect to performance, these devices mimic actual devices in hand. It has a slew of features without any complicated setup procedures.

Some of its highlighting features are –

• Kobiton’s real-device testing cloud platform has lots of highly responsive natural gestures such as tap, swipe, scroll, etc.
• Whether you run manual or automated tests, it captures detailed, rich test logs in formats like screenshots and videos, along with metrics like memory and battery performance.
• You can easily replay scenarios to perform root cause analysis.
• You can perform parallel execution of all your test scripts. It allows you to integrate it into your CI/CD pipelines quickly.
• With the help of ADB debugging, you can easily get hold of cloud (or local) devices straight from your developer workstation. You can natively access mobile devices directly from your IDE.
• You can use VPN from your private Kobiton cloud to ensure flexibility but not at the cost of security.

Kobiton supports open standard frameworks, which means that your scripts are protected. Some of these are Appium, Espresso, Selenium, XCUI, etc.

It offers several pricing plans, which include a free startup plan (500 mins/month), Accelerate plan ($3960 per year with 3000 mins/month), and a Scale plan ($9000 per year with 7500 mins/month).

7. Testsigma

Testsigma is not just a device farm but a cloud-based, no-code test automation platform. It allows you to perform parallel automated tests on real devices hosted in the cloud. It helps you to create test cases using plain English, and there is no need to set up any infrastructure.

Here are the features of Testsigma that make it sit right on top of the deck.

• Testsigma allows you to build stable, automated test environments for mobile applications. You can easily run test scenarios remotely on over 2000+ iOS and Android devices.
• All mobile devices are available on-demand because they are hosted on the cloud. Subsequently, you can also perform parallel automated testing, which will help you to attain quick results.
• It becomes effortless to create detailed test cases for UI/UX testing. You just need to specify the UI identifiers that you want to test.
• You can access drill-down test reports in the form of video recordings and screenshots. Other features, such as bug sharing, CI/CD integration, etc., can make reporting of results faster and easier.

To perform mobile app testing on different devices using Testsigma, you need to create test steps in natural language. For example, you can add steps to tap on a button, enter a text in an input field, etc.

Once done, you can create a test suite and run them on different mobile device configurations.

Testsigma offers a free plan where you get 200 automated testing minutes/month and 100s of real mobile and tablet devices. For unlimited testing minutes, it costs around $349 per month for one parallel test run. You can try Testsigma cloud for free.

Conclusion

In this article, we have reviewed some of the best device farms you can use to test your iOS and Android mobile applications. There’s no doubt that, when used at scale, the benefits of using cloud-based device farms are immense.

It helps testers get out of the hassles of multi-device UI/UX testing, saves a lot of time, and cuts labor costs. If you are a single developer looking to test your app on a few devices, you can try out a free solution like Firebase Test Lab, Samsung Remote Test Lab, etc.

If you are working on an app with a potentially large target audience, it’s always better to test the application thoroughly on as many device configurations as possible. In that case, you can try out platforms like AWS Device Farm, Kobiton, etc.

But if you are looking for a completely hassle-free and automated solution, you should definitely give a test-automation platform like Testsigma a shot.

Stay Tuned!