Automate Mobile Gestures in Appium: A Detailed Guide for Developers

Introduction

Touch actions represent the pinnacle of complexity and sophistication in implementing Android gestures. While some basic gestures, such as swipe, fling, and pinch, are commonly used in Android applications, it is beneficial to have shortcuts for these actions with configurable high-level options.

This blog delves into the evolution of touch gestures and demonstrates how a new plugin in Appium 2.0 can simplify the process of performing automated tests or Appium testing for these gestures.

Exploring Appium

Appium is an open-source project and ecosystem designed to facilitate UI automation across various platforms, including:

• Mobile: iOS, Android, Tizen
• Browser: Chrome, Firefox, Safari
• Desktop: macOS, Windows
• TV: Roku, tvOS, Android TV, Samsung, and more

Appium provides similar capabilities to Selenium but for mobile applications and mobile browsers. With the release of Appium 2.0, it aims to achieve the following primary goals:

1. Offer platform-specific automation capabilities under a cross-platform, standard API (compatible with W3C WebDriver protocol).
2. Enable easy access to this API from any programming language.
3. Provide tools for convenient community development of Appium extensions.

Understanding Touch Gestures

Touch gestures have revolutionized interaction, transforming the way we engage with devices. The effectiveness of mobile applications often hinges on the seamless integration of these gestures into the user experience.

In mobile testing, gestures are critical for simulating real user interactions. The W3C WebDriver Specification's Actions API, introduced in Appium 1.8, supports a range of mobile gestures, though it can be complex. Appium leverages this API to automate various gestures, including:

1. Long Press — Tap and hold the screen for a specified duration.

2. Swipe — A single swipe in any direction.

3. Scroll — Simulates scrolling action on the device.

4. Tap — A single tap on the screen.

5. Double Tap — Two quick taps on the screen.

6. Pinch — A two-finger gesture to zoom in or out.

7. Zoom — A two-finger gesture to magnify or reduce content size.

8. Drag and Drop — Move an object by dragging and dropping it at a new location.

Automating these gestures with Appium ensures consistent application performance across different devices and platforms. This blog will explore some of the most commonly used gestures and their importance in mobile testing.

Overview of Appium Architecture

Appium utilizes the WebDriver specification as its API, including the innovative Actions API for gesture actions. This design choice was inspired by Selenium, which established a reliable API for browser automation. Appium extends this foundation to support mobile applications (iOS and Android), ensuring a consistent platform automation approach.

Although user interactions differ between websites and native mobile apps, the WebDriver spec effectively maps to various platforms due to commonalities in software UIs. Appium aims to provide a uniform experience for developers and testers, regardless of the underlying technology.

In the following sections, we will detail how to use Appium for performing mobile gestures such as zoom, scroll, swipe, and drag and drop, utilizing the latest techniques and APIs:

• W3C Actions API
• W3C Mobile Gestures Actions (Android | iOS)
• UiScrollable Class (Deprecated)
• Appium Gestures Plugin
• TouchAction | MultiAction (Deprecated but still in use in many projects)

Check out: Top 14 Automated Mobile Testing Tools and Frameworks

Automating Gestures in Appium

1. Long Press: A long press is a gesture where a finger is pressed and held on the screen for a set amount of time. It is often used to trigger actions like opening a context menu.

WebElement element = driver.findElement(By.id("button"));
((JavascriptExecutor)driver).executeScript("mobile: longClickGesture",
	ImmutableMap.of("elementId", ((RemoteWebElement)element).getId(), "duration", 1000));

‍Components:‍

• elementId: The ID of the element to be clicked. If missing, both click offset coordinates must be provided.
• x, y: The offset coordinates.
• duration: Click duration in milliseconds (default 500). Cannot be negative.

‍2. Scroll: Used to simulate scrolling up or down on a mobile device, commonly for navigating through long lists.

Methods:

driver.findElement(AppiumBy.androidUIAutomator("new UiScrollable(new UiSelector()).scrollIntoView(text(\"your Text\"));"));	

boolean canScrollMore = (Boolean) ((JavascriptExecutor) driver).executeScript("mobile: scrollGesture",
	ImmutableMap.of("left", 100, "top", 100, "width", 200, "height", 200, "direction", "down", "percent", 3.0));	

Components:

• elementId: The ID of the element to be scrolled.
• left, top, width, height: Coordinates of the scroll bounding area.
• direction: Scrolling direction (up, down, left, right).
• percent: Size of the scroll as a percentage of the scrolling area.
• speed: Speed of the gesture in pixels per second (default 5000 * displayDensity).

‍3. Swipe: This function performs an Appium swipe gesture on a screen element, which is helpful for actions like switching views or pagination.

WebElement element = driver.findElement(By.id("button"));
((JavascriptExecutor)driver).executeScript("mobile: swipeGesture",
	ImmutableMap.of("elementId", ((RemoteWebElement)element).getId(), "direction", "left", "percent", 0.75));	

Components:

• direction: Swipe direction (up, down, left, right).
• velocity: Optional, supported from Appium server version 1.19 and Xcode SDK version 11.4+. Measured in pixels per second.
• element: Internal element identifier to swipe on. If not provided, the application element is used.

Read: A Step-by-Step Guide to Test Automation with Appium

How the Gestures Plugin Simplifies Appium Automation

The gestures plugin in Appium 2.0 streamlines the automation of gesture actions. Here's how it works:

MobileElement source = (MobileElement) new WebDriverWait(driver, 30)
	.until(elementToBeClickable(MobileBy.AccessibilityId("slider")));
driver.addCommand(HttpMethod.POST, String.format("/session/%s/plugin/actions/swipe", driver.getSessionId()), "swipe");
driver.execute("swipe", ImmutableMap.of("elementId", source.getId(), "percentage", 50));

Internally, the appium-gesture-plugin locates the specified element and calculates the target location based on the given percentage. It then creates a sequence of actions to perform the gesture on iOS and Android platforms.

Follow the installation instructions for the Appium gestures plugin for a working example of the swipe gesture.

The gestures plugin can handle simple actions like swipe, drag and drop, long press, and double-tap. For more custom actions, such as digital signatures, the Actions API can be used.

Additionally, Appium supports native gesture APIs provided by Android and iOS platforms through non-standard endpoints:

Map<String, Object> args = new HashMap<>();
args.put("direction", "up");
driver.executeScript("mobile: swipe", args);

Also read: A comprehensive guide for testing Android and iOS mobile apps with Appium

Appium Support for Mobile App Gestures

Appium provides several methods to support mobile app gestures:

1. TouchAction/MultiAction Class

The TouchAction class simulates various mobile gestures by chaining events such as tap, press, wait, longPress, and release, which can then be performed sequentially on a mobile device.

‍Examples:

// Long Press Gesture
TouchAction touchAction = new TouchAction(driver);
LongPressOptions longPressOptions = new LongPressOptions().withElement(ElementOption.element(element));
touchAction.longPress(longPressOptions).release().perform();
// Scroll/Swipe Gesture
touchAction.press(PointOption.point(startX, startY))
       	.waitAction(WaitOptions.waitOptions(Duration.ofMillis(1000)))
       	.moveTo(PointOption.point(endX, endY))
       	.release()
       	.perform();	

● Node.js, C#, Python: Refer to Appium documentation for specific implementations.

‍2. Appium Commands for iOS Gestures

Appium provides native commands for common iOS gestures like tap, double tap, swipe, and scroll.

‍Examples:

JavascriptExecutor js = (JavascriptExecutor) driver;
Map<String, Object> params = new HashMap<>();
params.put("element", ((RemoteWebElement) element).getId());
js.executeScript("mobile: doubleTap", params);	

params.put("direction", "down");
js.executeScript("mobile: scroll", params);	

3. Appium Commands for Android Gestures

Appium offers native commands for Android gestures such as long press, double click, swipe, and scroll.

‍Examples:

JavascriptExecutor js = (JavascriptExecutor) driver;
Map<String, Object> params = new HashMap<>();
params.put("x", x);
params.put("y", y);
params.put("duration", 1000);
js.executeScript("mobile: longClickGesture", params);	

params.put("elementId", AndroidElement.getId());
params.put("direction", "left");
js.executeScript("mobile: swipeGesture", params);

Enhancing Appium Gestures with HeadSpin Platform

The HeadSpin Platform elevates the capabilities of Appium Inspector for mobile gesture automation with the following advanced features:

• Real Device Cloud: HeadSpin's expansive cloud of real devices allows testers to execute Appium gestures on various devices and operating systems. This capability ensures that gestures such as swipe, pinch, and long press are tested in realistic environments, enhancing test accuracy and coverage.

• Detailed Gesture Performance Metrics: HeadSpin captures comprehensive metrics during the execution of Appium gestures, including real-time network conditions, touch response times, and device performance. These metrics help pinpoint issues related to gesture performance, enabling optimization of user interactions.

• Seamless Integration with Appium Inspector: HeadSpin integrates seamlessly with Appium Inspector, enabling testers to leverage the full potential of Appium's gesture automation capabilities. This integration provides a streamlined workflow for executing and analyzing gestures on real devices, enhancing the efficiency of the testing process.

• AI-Driven Gesture Analysis: HeadSpin employs AI-driven analysis to offer actionable insights into gesture performance. This includes identifying anomalies in gesture execution and providing recommendations for improving gesture accuracy and responsiveness, ensuring a smoother user experience.

By integrating with HeadSpin, Appium Inspector users can benefit from these advanced features to improve the effectiveness and precision of their gesture automation testing.

Closing Thoughts

Mobile gestures are crucial for effective mobile app testing with Appium. They enable developers and testers to simulate various user interactions, ensuring both the functionality and usability of mobile applications. Appium offers a comprehensive suite of methods and APIs to incorporate these gestures into automation scripts, facilitating robust and stable app testing.

Mastering mobile gestures in Appium enhances testing quality and effectiveness, meeting user expectations for a seamless mobile experience. Properly implemented gestures contribute to better user engagement and higher app ratings.

Integrating Appium Inspector with the HeadSpin Platform further amplifies testing capabilities. This combination provides access to real devices, detailed performance metrics, seamless integration, and AI-driven analysis, ensuring comprehensive test coverage and optimal app performance.

Connect Now

FAQs

Q1. What Are the Different Types of Wait Statements in Appium?

Ans: There are three different wait strategies in Appium:

1. Static Wait: Waits for a fixed duration by pausing execution for a hard-coded amount of time, e.g., using time.sleep(n).
2. Implicit Wait: Configures the WebDriver server to retry element searches up to a specified timeout if the element is not immediately found.
3. Explicit Wait: Waits for a specific condition or element to be met before continuing, providing more precise control than implicit waits.

Q2. How to Handle Notifications in Appium

Ans: To handle alerts and popups in Appium, follow these steps:

1. Programming Language: Java.
2. Mobile Automation Framework: Appium (Server Version 2.5.4, Appium Java Client - 9.2.x).
3. Test Runner: TestNG.
4. Build Tool: Maven.
5. Application Under Test: Android Proverbial app and iOS Proverbial app.

Q3. How Can Popups Be Managed in Appium?

Ans: To handle popups in an Android application with Appium, set the autoGrantPermissions desired capability to true. This setting automatically grants all permissions to the Android app once it is installed on the device, thus managing popups that require permission approval.