Helping BYJU's to Test in Multiple Locations and Debug Their Unity Application

Faster bug fixes, happy customers, awesome experience, higher reteintion, lower cost, and more revenue

Note:

Please note that this method is not available in older versions of Appium (1.6.0 and below).

For more on this process, visit http://appium.io/docs/en/advanced-concepts/image-elements/

When using AltUnityTester, please note that remote execution doesn’t work with versions before 1.4.0

For more on the process described below, visit https://gitlab.com/altom/altunity/altunitytester

In the example below, the tokens and IDs used in BYJU’s script have been masked.

byjus_learning_app_logo

BYJU’s – The Learning App helps students in India prepare for important national academic examinations (like the IIT-JEE, NEET, CAT, IAS) and standardized tests that are admissions requirements for graduate schools (such as the GRE and GMAT). 

BYJU’s app delivers rich, engaging educational content to students from many of India’s best teachers.

 

The Challenge

BYJU’s app is built with Unity, which makes automation tricky. Appium cannot detect elements inside of a Unity application like BYJU’s, making automation testing difficult. BYJU’s developers found that they had to conduct all tests manually, which slowed releases and created room for human error.

The Solution

HeadSpin’s Mobile Performance Platform allowed BYJU’s to implement precise automation tests across new builds of their app. These automations were set up two different ways.

Option #1: Find elements using images 

There are two ways for a developer to address the challenge of automating a Unity application. One option uses images to be identified by Appium. Appium can then click/perform automation using these images.

As with almost any approach to programming, using images to find elements has benefits and drawbacks. Leveraging images is not a one-size-fits-all approach that will work in all circumstances for Unity Automation. 

We recommend using  in cases of automating tests for game load or app load. 

Please note that this method is not available in older versions of Appium (1.6.0 and below). For more on this process, visit http://appium.io/docs/en/advance d-concepts/image-elements/ 

How to do it

First, take a reference image of a button. This can be done using Android Debug Bridge (adb) commands: 

`adb shell screencap /sdcard/reference.png`

 

Once you’ve captured the screenshot of the phone, pull it to your computer using the command:

`adb pull /sdcard/reference.png`

 

Use any image cropping tool to cut the image to the size of the intended place to be clicked on the application. 


Save the croppsed image. (For example, we’ll save the cropped image as “test1.png”)


In Python, you can then use the following method to find the element.

element = self.driver. find_element_by_image(‘test1.png’)


Many methods can now be used with the element. For example, element.click() would click on the intended place inside of the application.


Many other methods can be used with the element.

 

Pros
Cons
● Quick development of automation script
● UI changes (like logo or color changes) may affect the script
● Doesn’t require any change to code
● Image resolution is based on screenshots, which can lead to problems on devices with low resolution
● Can be used for both iOS and Android phones
● Screenshots taken on one device may not work on another (To some extent, these two issues can be controlled using tolerance.)
● Complex activities in the application (like clicking on a moving object) can create issues

Option #2: Automation using AltUnityTester 

When using images proves to be problematic, developers should consider leveraging an automation helper to address the challenge — as the main objective of any automation helper is to give the object information at run time. The easiest way to fix this problem is by embedding a server. By adding a package like AltUnityTester to Unity, Appium can find objects at run time.

When using AltUnityTester, please note that remote execution doesn’t work with versions before 1.4.0

For more on the process described below, visit https://gitlab.com/altom/altunity/ altunitytester 

How to do it

Add all the scenes to be used in the build and build the Unity application with them.


On launch, the application will internally run the server in the default port. AltUnityTester is designed to run at port 13000 by default on the device. For interacting with the device, port forwarding should be enabled. This can be done using the following command:
`adb forward tcp:13000 tcp:13000`


AltUnityTester always accepts the connections coming from `localhost`, so for running remotely we used `socat` for the port requests from another port number to 13000 as `localhost`. From the host machine, we run:
`socat TCP-LISTEN:14000,fork TCP:0.0.0.0:13000`


The test case is done using Python. So we would need the Python bindings to access the methods of AltUnityTester:
`pip install altunityrunner`


To get altunityrunner driver, invoke:
`from altunityrunner import AltrunUnityDriver` `altdriver = AltrunUnityDriver(driver,platform=”cloud”,TCP_IP=”<publicIP to remote host>”, TCP_FWD_PORT=14000, TCP_PORT=14000, requestEnd=’#’)`

 

Executing Automation Testing with HeadSpin

Using the above approaches, BYJU’S was able to set up and remotely execute Unity Automation tests and debug scripts. BYJU’s originally created Automation test and debug scripts to run locally. However, HeadSpin allowed developers to run the same set of scripts on real devices located in Delhi — from the comfort of BYJU’s Bangalor development office. 

The debug application were installed using Headspin install APIs. Port forwarding was done from the device to the Delhi host using an `adb forward` command. The debug server inside the application was running on 13000 on the remote device. Socat was used to provide forward requests coming from 14000 to 13000 as `localhost`.

NOTE: In the example below, the tokens and IDs used in BYJU’s script have been masked.

Sample Code

desired_caps = {}

desired_caps[‘platformName’] = ‘Android’

desired_caps[‘automationName’] = ‘UiAutomator2’

desired_caps[‘deviceName’] = ‘XXXXXXXX’

desired_caps[‘udid’] = ‘XXXXXXXXXX’

desired_caps[‘autoAcceptsAlerts’] = ‘true’

desired_caps[‘newCommandTimeout’] = 300

desired_caps[‘appPackage’] = “<XXXXXXXX>”

desired_caps[‘appActivity’] = “<XXXXXXXX>”

driver = webdriver.Remote(‘https://xxxxx.headspin.io:XXXX/xx/<XXXX>/wd/hub’, desired_caps)

altdriver = AltrunUnityDriver(driver,platform=”cloud”,TCP_IP=’xxxxx.headspin.io’, TCP_FWD_PORT=14000, TCP_PORT=14000, requestEnd=’#’)

altdriver.wait_for_current_scene_to_be(‘test’)

altdriver.wait_for_element(‘test2’).mobile_tap()

Conclusion

For an app like BYJU’s, testing can make or break the success of the app. Fast, efficient testing can help developers release higher quality products in a timely manner — which boosts user experience and fuels user adoption. 

HeadSpin allowed BYJU’s to turn Remote Unity Automation testing into reality. The move from manual to automated testing allowed BYJU’s to release a better quality, higher performing product. 

  • Time to market improved by 15X — going from 5 weeks to only a few days. 
  • Automated testing allowed developers to gain a better, more detailed understanding of app performance than manual testing ever could provide, eliminating bottlenecks and addressing specific user experience issues.
“HeadSpin enabled BYJU’s to turn Remote Unity Automation testing into reality. They were able to test mobile app performance for various locations seamlessly on a remote network, through the network of devices that HeadSpin provided. This saved BYJU's time and effort to release a better quality, higher performing product. Time to market improved by 15X — going from 5 weeks to only a few days.“
Mohammed. Irfan
Lead Quality Engineer, BYJU'S

Better experiences. Better results.

Accelerate development and innovation. Optimize performance and functionality. Ensure business success. Across all applications, devices, and networks. That’s Connected Intelligence.™