Mobile phones have become part of our daily life. We use them for everything including capturing and editing photos, conference calling, and buying groceries. According to Datereportal.com, there are 5.5 billion mobile phone users, and that’s increasing by 1.8 percent each year.

With such enormous growth looming in the mobile space, there will be an increasing demand for app developers and testers who can deliver usable apps to the market. And your team will be responsible for testing devices of various sizes, operating systems, and specifications, before an application can be released to end-users.

It could be very costly and inefficient for your team to purchase and test every new device that comes to market. They will receive hundreds of requests for changes to the various mobile specifications on the devices they are testing. The testing processes they are using may not be repeatable, scalable or automated, and not able to be incorporated into the current workflows.

Cloud-based Mobile Testing

Cloud-based mobile testing is used by a development team and is applied when testing devices that hosted on cloud-based servers and can be accessed online anytime. Because most device cloud providers offer many types of real desktops and mobile devices, the team can see what the actual device looks like.

Cloud-based testing is cost-effective because it’s not necessary to purchase every device that needs to be tested. Currently, many large companies such as PayPal, Unity, IBM, Microsoft, etc. already use device cloud testing to ensure they deliver the best applications.

Here are a few providers that can provide you with cloud mobile testing experience:

• Saucelabs,
• Browserstack,
• Kobiton,
• AWS Device Farm,
• and many other providers.

Let us take a look at one cloud-based mobile testing provider. Here are some features that most providers such as Mobile App Live Testing and Mobile App Automated Testing have in their common.

Saucelabs

In 2008, San Francisco, Steven Hazel, Al Sargent, and Jason Huggins (Selenium creator) found the Saucelab platform for web and application testing based on cloud. This platform allows you to test various – more than 700 different browsers and devices. Users can create a new account on Saucelabs with one-month’s free testing.

However, free-trial account will get limited number of devices that available for testing. Saucelabs offers a live testing feature. With live testing feature, a user can choose between live testing on a cross browser, or mobile device.

Mobile App Live Testing

You can select mobile app live testing under Live menu.

First, you need to upload the Saucelabs application file. Your app will be opened on their device based on your selection.

Hover over your app on the list and click Choose Device.

Now we have a list of all devices in Saucelabs. Saucelabs has currently 159 devices available on the device list. However, we can ask to include other devices by contacting the Saucelabs team.

The device list contains information such as:

• Device Name
• Device Image
• OS Version
• Device Screen Size

Open the device’s detail and you can see the full specifications.

Let’s launch the application and see the live testing app on the device!

We can see on the screen that the application can run on the device. Users can tap, or swipe on the application. Features on the panel contains Stop, Restart, Session Info, Rotate Screen, Camera, Take Screenshots, ADB Shell, Logs and much more.

This live testing can be used to replicate a user’s behavior manually on the application. With so many options on the device, a development team can test an application on various live devices. The live testing on Saucelabs feels more like it’s running on emulators/mirroring in term of speed. On some providers, live testing is slower and like screenshots taken on every user’s action.

Mobile App Automated Testing

Before using the automated feature in Saucelabs, you need to set up the connection between your test and Saucelabs. If you are using Appium, you need to set up your hostname, path, user, key, region, and service under Options. Then, you need to set up the capabilities, set up for your Saucelabs device. Your app can be hosted in Saucelabs storage, AWS, or Github.

Finally, you can run your test and see the test result on the Saucelabs dashboard. Under the Automated menu, you can click on Test Result.

You will find the list of test result containing the device, version, and execution time.

Open the test result. On this page, you can see a video with the recorded test. You can also see a list of Commands, Logs, and Metadata from your test.

On the commands, you can click the command and timeframe of the test. Parameters and response results of the command are also displayed on this page.

This automated testing on a real device feature is useful to run your existing automated test in various devices that Saucelabs provided. Benefits of cloud-based mobile testing:

• Cost effective. No need to buy many devices to test your application.
• Access various devices in one place. You don’t need to store and maintenance many devices in your workplace.
• Online accessible. You can access all devices anywhere while you are online.
• Automated testing feature. Test your automated script in various devices. Some providers have script-less automated test feature.
• Real devices available. Most providers use a real device to test.
• Test in the same device with the customer. You can replicate real life customer issues with the same device specification.
• Integration. You can integrate your team dev ops into the cloud-based mobile testing.

Drawbacks of cloud-based mobile testing:

• Performance. Some providers have a slower live testing application.
• Pricing. The basic pricing for most providers only includes live testing, and for automated testing, you will pay an additional fee.
• Security. You need to be aware of the providers Privacy Policy and Terms of Service regarding the device usage. In addition, it’s important to know how the device is reset/cleaned every time a user accesses the device to make sure it is safe. You need to know how they record and manage test results if you need to input some credential and data. Some providers can give you a private device for additional cost.
• For a free version, some providers use shared public devices but you may need to wait until a device is available.

Conclusion

Before looking at the benefits and the drawbacks of cloud-based mobile testing, you need to assess your development team’s resources. You can use the provider’s free trial to match their product features with your needs. Be sure to compare their features, performance, and available devices. If you need to test apps with high/tight security protocol, it’s essential that you research several providers. How they manage their devices, and the security options they offer should be high on your priority list.

Live testing can be a solution to replicate the issues on the customer side. An automated test feature can be used to test your script on various real devices. However, be aware that based on a provider’s pricing package, automated testing might be more costly than just live testing.

In summary, cloud-based mobile testing can help you get the highest quality of testing by running your tests on various real devices. And it can provide a cost-effective solution to managing and accessing test devices.

Author:
Putri Dwitya Weniwandari – Senior Tester