.png){kind=icon}
What is KPI?
Key Performance Indicators (KPIs) are the quantifiable metrics used by app developers to measure the success of their apps and business. KPIs are used to measure various aspects for the optimal functioning of an app.
User Experience KPIs
User experience is perhaps the main factor concerning whether a mobile app succeeds or fails. KPIs that affect the experience of the end user can be considered as User Experience KPIs. Accordingly, using these mobile app KPIs to track, measure, and improve on it, is probably the most important activity you can engage in, as a mobile app developer, tester, or QA engineer.
Also read: Improving mobile user experience through SDK abstraction
The following are some of the important KPIs to be considered while testing apps in the gaming genre. These KPIs will change depending on which tier i.e. low-end, mid-end, or high-end the device belongs to. In addition to this, the KPIs related to time will depend on the network or Wi-Fi your device is connected to.
1. Play Store download and install time
It is the time that's measured from initiating download in Play Store to completion of installation.
The download time depends on the network as well as the mobile hardware.
Longer download time can be due to your network being slow or having low storage. Another obvious reason is the size of the file. Larger the file, the more time it will take to download and install. There are times when you might have to wait for up to 20 minutes depending on its architecture and the quality of your internet connection to install and run the application.
According to a report by the co-founder of Segment, Peter Reinhardt, larger apps tend to be less responsive, slow to load, and more likely to crash. As a result, such apps offer a poor user experience that triggers users to uninstall them.
As you can see from the study, and the user reviews on Google Playstore, an app with a shorter download time has a higher chance of being installed by the user.
2. First launch time
The first launch page of the apps is the page containing the main game contents. This screen varies with different apps. It may be the home screen, with the title of the game, the initial tutorial page of the core game, the main map of the game, etc.
To reach this page you may have to clear certain pages, like registration/login, age confirmation, terms agreement, android permissions, secondary download, splash, or movie screen.
The time measured from the launch of the app to the first launch page gives us the first launch time.
3. Cold start time
To achieve the cold start page of the app,
- Terminate the app completely
- Do not clear the cache. Relaunch the app and proceed to the page containing the main game content or page containing in-app advertisement or event pop up, which appears before the main menu page
- Preferably avoid reaching the tutorial screen, by clearing it during the first launch
The time from the launch of the app till this page gives the cold start time.
Both ‘First Launch Time’ and ‘Cold Start Time’ give the time taken to launch completely, with the difference that they are achieved differently.
App launch speed is very indicative of the overall quality of your app and tracking it will help assess the responsiveness of your app. Speed is an essential part of app performance. Users don’t like to wait and they especially don’t like slow apps. How fast your app launches can set very important first impressions with your users and go a long way to retain them. One report says that 21% of users now abandon an app after one use. And according to a different report, 80% would give a poorly performing app three chances or less before uninstalling it.
According to Google, your app’s startup time is not considered excessive if it falls under 5 seconds for a cold startup.
Based on an analysis of the top 100 apps, 39 apps start in under or around 2 seconds, and 73 in under 3 seconds. According to another study, 49% of users expect apps to start in 2 seconds or less. Ideally, you should be targeting an app launch time of 1.5-2 seconds. This, however, is talking about apps launching from scratch also known as a cold start.
Users expect apps to be responsive and fast to load. An app with a slow start time doesn’t meet this expectation and can be disappointing to users. This sort of poor experience may cause a user to rate your app poorly on the Play Store, or even abandon your app altogether.70% of mobile app users will abandon an app if it is taking too long to load. These reviews from game users make it obvious that the ‘First Launch Time’ and ‘Cold Start Time’ are important KPIs that give the overall performance of the game app.
Also check: How to Launch High Performing Mobile Gaming Apps with Confidence
4. Secondary download time
It is the time taken for the secondary download.
The App may be connecting to a server to download required data based on your device resolution (Mostly for games this is the case as they cannot package all the graphics for all the available devices in the app store).
The secondary download usually occurs after you launch the app, accepting the permissions and terms. You may or may not have to register/login before the secondary download begins.
This time depends on how much additional data needs to be downloaded for the app to give the best user experience. This also depends on the network or wifi you are connected to.
5. App size at launch
It is the app size after installing the app, before launch.
The application file (apk or ipa) which you download from the Play Store is a kind of archive with some compression used to bring down the size of the app file. This helps, as it lowers the data transfer during the download. It's a good practice to compress your data while sending it out, as it helps in saving the very precious bandwidth. The app takes more disk space in the internal memory, after installation than the size that is shown in the Play Store.
This is because the apk file which is nothing but a compressed file containing assets like images and compiled java codes and few more XML files needed by the app, gets uncompressed, which increases the size.
The app size doesn't change after launch.
6. User data size at launch
It is the user data size after installing the app, before launch.
7. User data post launch
The user data changes after you launch and interact with the app. Additional data includes the secondary download and other interactions. This KPI measures the user data after your cold launch.
User data is information collected from or about a user, including device information. These are usually the app’s access. In addition, apps may collect even personal and sensitive data. Personal and sensitive user data includes, but isn't limited to, personally identifiable information, financial and payment information, authentication information, phonebook, contacts, device location, SMS and call-related data, microphone, camera, and other sensitive device or usage data.
8. Total data post launch
Total data includes app size, user data, and cache.
As you start using the app the user data is collected based on the functionality. So the total data size keeps increasing as you keep using the app.
Games take up a lot of space on your phone and contain a whole bunch of data. It takes a great deal of processing power on your phone to make these applications work, which is why you may observe changes in your phone's performance.
When you download and install a game on your device, the data is stored within internal storage. When you run the game, it loads all the assets into RAM, which is Random Access Memory. Accessing RAM is much faster than accessing internal memory, which allows games to load scenes, animations, and other things faster. The game should know exactly where the assets are in RAM with a memory address. It can jump right to that point and pull the information down.
Check out: HeadSpin's Testing solution for Gaming
However, games are not the only thing that uses RAM. The operating system, system applications, and other installed apps are also using RAM. The more RAM that is used, the less there is for new operations to occupy temporarily. If RAM gets too full, then it could cause problems.
The app size user data at launch, user data post launch, and the total data post launch are KPIs related to storage and give information on how the storage of the user's phone gets affected, which in turn affects the performance of the phone. 25% of smartphone users delete apps because of a lack of storage space.
Mobile gaming is all about high-quality performance. Monitoring these KPIs while testing is critical to ensure that your app gives users the performance they expect.
FAQs
Q1. What is KPI in user experience?
A1. UX KPIs are metrics you collect to measure the success rate of UX activities. They help a company understand how close they are to meeting their UX goals and areas they need to improve.
Q2. Why should UX KPIs be measured?
A2. It is essential to measure KPIs as they help you understand the impact of your UX decisions and make changes using evidence. They reflect overall goals like retention, increase in user numbers, and revenue growth.
Q3. Why do some apps take so long to open?
A3. Some apps take longer to open due to an overloaded or overworked server. Sometimes, your app may rely heavily on other processes for its tasks. If these processes have issues with latency, this can cause apps to take longer to open.
Q4. What is a good app start time?
A4. For Androids, app startup times are excessive when:
- Cold startup - 5 seconds or more
- Warm startup - 2 seconds or more
- Hot startup - 1.5 seconds or more
-1280X720-Final-2.jpg)
