Ein-des-ein blog

How to Do Mobile App Performance Testing?

How many downloaded apps do you think users delete after the first use? You may be shocked, but it’s nine out of ten. The causes are poor user experience, freezes and crashes, and low performance of essential functions. The list goes on for a long time. However, most of them can be avoided through comprehensive and rigorous mobile app performance testing before the release.

Some companies try to save money and refuse to test their mobile applications. However, they make a huge mistake, which leads to the fact that the final product is not functional, and time and money are wasted.
This article will tell you what kinds of testing are available and take a detailed look at one of the most important types —  mobile app performance testing. The ein-des-ein engineers, experienced in mobile app development, will share how it is performed, its purpose, tools, and approaches.

Mobile Application: What is it?

Today, a mobile app is a piece of software specifically designed for the functionality of mobile and portable devices such as smartphones, tablets, smartwatches, and other smart devices like TVs or even refrigerators. 

Their applications vary widely, from services, shopping, and entertainment to online medical assistants using machine learning and production line management for large factories. Users download mobile apps from mobile stores Appstore (iOS) and Google Play market (Android). Apps also can be elements of corporate ecosystems. All the applications are created for a particular operating system of the mobile gadget. The most popular operating systems are iOS, Android. Together they account for over 99% of the global market share.

Mobile Application Testing

Mobile application testing is a multistage process that allows checking an app by numerous parameters and under different points of view. The bottom line is to ensure that developed software meets all functional and non-functional requirements and provides end-users with an outstanding user experience. 

There are numerous options on how to test mobile applications. It includes both manual and automated approaches. 

Functional testing

It’s one of the basic types of testing. Its purpose is to check whether actual characteristics correspond to the functional requirements of the app. In simple words, to make sure that the application has all the functionality required to cover business needs.

Compatibility testing

This type has the highest importance when it comes to mobile app testing. The test’s purpose is to check the compatibility of a mobile app with all the devices specified in the development requirements. This is usually more of a concern for testing Android apps. The reason is an enormous variety of Android devices and versions of the operational system. Defining a list of mobile devices required for tests is not easy since testing with all the existing devices is costly and makes no sense.

Localization testing

These days, most applications are designed for global use, and you should always keep regional specifics in mind, such as languages, time zones, and culture. It’s essential to keep in mind that sometimes a design developed for the western market may not be suitable for an audience in eastern countries and vice versa. The most obvious scenario is the use of languages written from right to left.

Performance testing

Mobile performance testing is a set of testing types that aims to determine an application’s performance, stability, resource consumption, and other quality attributes under different usage scenarios and loads. Performance testing allows you to find possible vulnerabilities and weaknesses in the system to prevent their detrimental effect on the program’s performance in real life.

Security testing

This testing aims to help identify all possible vulnerabilities for hacking, authentication and authorization, data security, session management, and other security standards. 

Usability testing

Its task is to evaluate a mobile application based on the following three criteria for the end-user: efficiency, accuracy and completeness, and satisfaction. It is vital to conduct usability testing from the earliest stage of the development lifecycle. This type of testing requires active user participation. At the same time, The results can influence the application’s design, which is very difficult to change later in the project.

Others

In addition to the basic types of tests, there are many more tests that can be performed when testing a mobile application:

• Installation/Uninstallation testing
• Updates Testing
• Certification Testing
• Screen Orientation / Resolution
• Memory Leakage Testing
• Available Tools
• Touch Screens
• Soft & Hard Keys

Besides testing, many other factors impact the success of the app. Read our article 6 tips to keep your project on track and succeed to learn more.

Strategy for Performance Mobile Applications Testing

Performance testing is a voluminous multitasking process that consists of many steps. It checks the system’s performance in all possible conditions.

To ensure effective test results, a crystalline performance testing strategy is required. Fortunately, there is no need to reinvent the wheel every time. Its stages have long been developed and polished over mobile apps’ lifetime. They also have no difference from performance testing of any other types of software.

However, a detailed analysis of the application is performed each time. It includes all the information gathered about the system:

• Testing objectives
• Application requirements
• Test bench configuration
• Monitoring
• User-behavior scenarios
• Profile and load model
• Testing tools
• Planned test runs
• Type of results submission

Step 1: Identify test goals and business requirements to satisfy

The effectiveness of mobile performance testing is directly proportional to how clear its goals are. Before adding a target to the testing list, ensure that each test covers the functionality that meets the business goals. Suppose we are talking about a telemedicine application. In that case, the tester will first work with the main functions of the application that directly influence the quality of patient care — monitoring vital signs, communication with the treating physician, and prescribing medications.

Step 2: Define testing KPIs

Before the start of mobile application performance testing, be sure to define benchmarks for tested parameters. Otherwise, how can you tell if the application meets the specified requirements? Depending on the purpose of the application, the list of KPIs may vary. Here is an example of the most common metrics.

Performance Testing KPIs

Number of users

Determining the peak number of users of the application

Response time

The time it takes for the application to complete a user request

Requests per second

Measurement of the number of requests sent to the backend

Transactions per second

Measurement of the number of transactions sent to the server

Error rate

Percentage of errors from the total number of responses per unit of time

Processor load

Time the processor spends executing a worker thread

Memory load

The number of MB in physical memory that is free for other programs and processes.

Step 3: Prioritize testing scenarios

The biggest mistake of all novice testers is testing application features in a row regardless of their importance. Testing without pre-selecting cases will only result in wasted time with no benefit to the product. The optimal approach is to select a few scenarios which will test vital features of the app.

Only after all the main functionalities are tested from A to Z, it’s time to start testing the secondary features, without which the application will be able to fulfill the end users’ primary needs. 

Step 4. Simulate a real-life environment for application performance testing

One of the ultimate goals of application performance testing is to ensure that the user gets a great experience interacting with the app. You may assume that all tests should be conducted on real devices. However, this approach is not always time-efficient, and they are not always necessary. Experts often use system emulators to reduce the QA team’s efforts and the costs of testing. 

Such tools artificially recreate basic parameters of mobile platforms and allow testers to see and interact with the application’s interface from the user’s perspective. However, physical devices are usually used to test apps if they utilize native functions such as the camera, GPS, calls, and more.

Step 5. Define testing approach regarding to overall development methodology

The testing process differs depending on the methodology of the software development cycle. Let’s suppose your project is built on Agile. In that case, testing is not a separate phase. It is closely linked to all stages of development, such as requirements, design, coding, and test suite generation. Testing co-occurs through the development lifecycle. In Waterfall, testing is a separate phase of the development cycle, initiated after completion.

Each methodology has its pros and cons. However, it does not matter which one you use; application performance testing must follow the entire project’s methodology. Otherwise, there will be total confusion.

Differences Between Agile And Waterfall Testing

Step 6: Consider your carrier’s network bandwidth and signal latency when testing

One of the less obvious facts that need to be considered during the application testing is the difference in signal quality between the telecommunication companies, whose latency and bandwidth can vary. For example, far from all companies can cover the entire territory with 4G or at least 3G support.

From this, performance testers need to optimize the application. Even with poor internet quality, the user should gain a great user experience.

Mobile App Performance Testing Environment

Proper configuration of the testing environment is one of the most important parameters influencing the accuracy of the tests and the quality of the results. Depending on the test object, the settings of the performance testing environment can vary. However, it’s always recommended to perform the following actions to ensure the fairness of testing results.

Required Steps For Setting Testing Environment

Here are four main steps to set up a mobile performance testing environment:

Isolated testing environment

During testing, only the testy-user can access app resources. It doesn’t have to include any other activity that consumes system resources unless the test scenario requires it. Otherwise, you will compromise the accuracy of the testing. When the platform processes requests from several users at once, it increases the load on the server part, which affects the system performance and will cause biased test results.

Data simulation with test data generators

Performance tests are often related to database operations. It is crucial to focus on processes such as writing, reading, and deleting data during testing. Exactly these basic actions are the basis of all functions. Failure to perform them correctly can lead to a critical error and bring the application to a complete halt. Obviously, at the time of creating the application, QA experts have no actual data to test. This problem is solved by numerous data generation tools that artificially make analogies to the data that the system would handle in real-world use cases.

Isolating an app from the network

It’s an optional step in mobile app performance testing. Suppose the network has high bandwidth, which can fully cover the tester and other users’ actions. In that case, there is no need to isolate the application from the outside world. However, it’s worth noting that most networks will not provide such a low timeout, which will not harm the final result.

Excluding proxy servers from the network route

Another factor that can reduce server-to-client data flow speed and negatively impact application performance is a proxy server. Therefore, during performance testing, server-client data transfer is executed in a separate network without intermediaries.

Challenges That QA Esports face within Mobile App Performance Testing

In most cases, mobile app performance testing requires more of a testing team’s efforts than its desktop version. This is caused by the incredible diversity of mobile and portable devices users can use the app. The variety of operating systems, screen sizes, and aspect ratios, and radically different technical stuffing inside — all of this creates an additional headache for the testing team.

The Most Common Problems In Testing The Performance of Mobile Apps

Wide variety of device-specific features and native modules

All modern smartphones use GPS, camera, microphone, various sensors, and other functions, difficult to simulate with online emulators. If you need to test how the app uses the GPS module and tracks the location, the only way is to do it using a physical device.  And since the app needs to be tested on all possible devices, you have to spend more money to buy a broader database of devices, increasing the testing budget. 

Variety of user interfaces

Depending on the version of your operating system, page layouts, fonts, and placing control elements on the screen differ significantly. However, they are regulated with generally accepted standards and recommendations from manufacturers. One of the challenges is to make sure that your product meets the UI/UX requirements of the Play Market or App Store. Otherwise, your app won’t get published, and you will lose time and money and need urgent app updates.

Challenges of testing touchscreens

User behavior and the full range of possible interactions with an application through touches and gestures are challenging to simulate on emulation tools with a computer. Even using physical devices requires much time and expertise from testers to test all possible scenarios in a real-world environment. Most of the time, all these manipulations are performed manually since this aspect of testing applications is very difficult to automate. 

Tools for Mobile App Performance Testing

Conceptually, mobile app performance testing involves two main parts. The first one is testing the product performance at the business logic level, in other words, processing requests by the backend. The second is testing the speed and performance of the application on the mobile device, both physical or emulated, i.e., checking the application from the end user’s point of view. Currently, testers have a wide range of tools at their disposal to automate and speed up the process. Let’s take a look at the most popular ones.

The most popular tools for backend performance testing

JMeter

The undisputed champion among mobile performance testing tools, recognized by experts around the world. Jmeter is an open-source tool. At present, it has been enriched with a considerable number of plugins, which gives testers a vast space to utilize it. It also has a simple and intuitive interface. The tool covers a wide range of testing, including creating load generators and a variety of controllers. 

It also does a great job of volume and availability testing and visualizing the results and HTML reporting. 

The development team takes care of their user. Complete documentation describing all Jmeter functions is easy to find.

Gatling

It’s an open-source load testing tool written entirely in Scala. It has a simple and expressive DSL making it easy to write load testing scripts. In contrast to JMeter, it does not contain a graphical interface. It can handle a considerable amount of traffic on a single computer, eliminating the need for a complex distributed testing infrastructure. 

Gatling also allows you to embed code validation into your version control system and easily use it in conjunction with continuous integration tools to run load and performance tests as part of your CI build.

Client-side performance testing

After testing the server-side, it’s time to see what tools are used to test an app performance from the user side.

Sauce Labs

This service provides the world’s largest cloud-based platform for automated testing of web and mobile applications. The service avoids the time and expense of maintaining your testing infrastructure, freeing up development teams of all sizes to innovate and release better software quickly.

Founded by the original creator of Selenium, Sauce Labs helps companies accelerate software development cycles, improve app quality and confidently deploy to more than 700 browser/OS platforms, including Windows, Linux, iOS, Android, and Mac OS X.

Appium

Appium is a cross-platform tool, and it allows you to write tests for mobile platforms (iOS, Android, Windows) using APIs. It is one of the most widely used tools for regression testing applications on smartphones and tablets. It supports testing native, hybrid, and mobile web applications.

UI Automator

UI Automator is developed by Google Corporation and comes with the Android SDK. UI Automator is an analog of Apple’s UIAutomation tool for testing Android applications. It consists of  Java API libraries that contain methods for creating UI tests.

Robotium

Robotium is an open-source test environment for writing automated gray-box tests for Android applications. With Robotium support, test developers can write feature scripts, system tests, and acceptance tests covering several areas of Android activity. You can use this tool to test both apps where the source code is available and apps where only the APK file is available.

Selendroid

Selendroid is another test automation environment for testing native and hybrid Android applications. Their tests are written using the Selenium Webdriver client API.

Testdroid 

It’s one more cloud-based mobile app testing tool that allows developers to save app development costs, speeds product time to market, and reduces operational and unpredictable expenses. It is one of the reliable mobile game testing platforms for Android and iOS. It provides remote manual access to more than 300 real Android devices before an app launch.

Examples of Performance App Testing

Load testing 

Testing the application’s response time on different requests to make sure the application works as required under average user load.

Stress testing

Testing the performance of an application under loads is several times greater than user loads. With stress testing, we can get actual data on application performance boundaries and examine the program’s ability to handle exceptions, stability, and robustness. The difference between performance testing and stress testing is in the significantly increased load on the application.

Stability testing  

In the testing process, the application’s performance is evaluated during a long run over time, at an average load for the program.

Volume testing

Testing is performed based on an increasing amount of data stored and used in the application, not the load and runtime.

Wrapping It Up

When defining performance testing and its types, it often comes to a misunderstanding and confusion of these terms. To avoid this and to consolidate the knowledge gained, let us summarize. So, performance testing checks compliance to non-functional requirements such as application performance and operability under various loads and conditions.