Automating Mac Apps with Appium

Appium isn't limited to automating mobile systems! As long as there is an open way to interact with a system, a driver can be written for it, and included in Appium. Using a project called Appium For Mac Appium can automate native macOs apps.

Setup

Appium comes bundled with a macOs driver, but the actual AppiumForMac binary is not included, so we need to install it ourselves first:

• Start by downloading the latest release from here.
• Unzip the AppiumForMac.zip file by double-clicking it in Finder.
• Move AppiumForMac.app file into your Applications folder.

AppiumForMac uses the system Accessibility API in order to automate apps. We need to give expanded permissions to AppiumForMac in order for it to work. Here are instructions from the README:

Open System Preferences > Security & Privacy. Click the Privacy tab. Click Accessibility in the left hand table. If needed, click the lock to make changes. If you do not see AppiumForMac.app in the list of apps, then drag it to the list from Finder. Check the checkbox next to AppiumForMac.app.

I'm using the latest version of macOS (10.14.2), if you are using an earlier version, specific instructions are included in the AppiumForMac Readme.

Run a Test

The example code for AppiumForMac already automates the calculator app, so let's do something different and automate the Activity Monitor instead.

In order to automate a macOs app, we only need to set the following desired capabilities:

{
  "platformName": "Mac",
  "deviceName": "Mac",
  "app": "Activity Monitor"
}

We specify Mac as the platform and set app to the name of the installed app we want to run. Once a test has been started, an app can also be launched using the GET command, e.g.:

driver.get("Calculator")

Absolute AXPath

AppiumForMac is a little tricky, since elements can only be found using a special kind of XPath selector called "absolute AXPath". All the AXPath selectors use Accessibility API identifiers and properties. I'm including the exact rules for AXPath selectors below, but don't be afraid if they do not make sense at first; in the next section I describe some tools for finding AXPath selectors.

Here are the rules for a valid Absolute AXPath selector:

• Uses OS X Accessibility properties, e.g. AXMenuItem or AXTitle. You cannot use any property of an element besides these.
• Must begin with /AXApplication.
• Must contain at least one other node following /AXApplication.
• Does not contain "//", or use a wildcard, or specify multiple paths using |.
• Uses predicates with a single integer as an index, or one or more string comparisons using = and !=.
• May use boolean operators and or or in between multiple comparisons, but may not include both and and or in a single statement. and and or must be surrounded by spaces.
• Does not use predicate strings containing braces [] or parentheses ().
• Uses single quotes, not double quotes for attribute strings.
• Does not contain spaces except inside quotes and surrounding the and and or operators.

Any XPath selector that follows the above rules will work as an absolute AXPath selector. Be warned: if your AXPath selector breaks the rules, you won't get a special error and instead will get an ElementNotFound exception. It can be difficult to identify whether your selectors are failing because the AXPath is invalid or the element simply is not on the screen.

The README contains the following examples as guidance:

Good examples:

"/AXApplication[@AXTitle='Calculator']/AXWindow[0]/AXGroup[1]/AXButton[@AXDescription='clear']"

"/AXApplication[@AXTitle='Calculator']/AXMenuBarItems/AXMenuBarItem[@AXTitle='View']/AXMenu/AXMenuItem[@AXTitle='Scientific']"

"/AXApplication/AXMenuBarItems/AXMenuBarItem[@AXTitle='View']/AXMenu/AXMenuItem[@AXTitle='Basic' and @AXMenuItemMarkChar!='']"

Bad examples:

"//AXButton[@AXDescription='clear']"
(does not begin with /AXApplication, and contains //)

"/AXApplication[@AXTitle='Calculator']/AXWindow[0]/AXButton[@AXDescription='clear']"
(not an absolute path: missing AXGroup)

"/AXApplication[@AXTitle="Calculator"]/AXWindow[0]"
(a predicate string uses double quotes)

"/AXApplication[@AXTitle='Calculator']"
(path does not contain at least two nodes)

"/AXApplication[@AXTitle='Calculator']/AXMenuBar/AXMenuBarItems/AXMenuBarItem[@AXTitle='(Window)']"
(a predicate string contains forbidden characters)

"/AXApplication[@AXTitle='Calculator']/AXWindow[0]/AXGroup[1]/AXButton[@AXDescription ='clear']"
(a predicate contain a space before the =)

"/AXApplication[@AXTitle='Calculator']/AXWindow[position()>3]/AXGroup[1]/AXButton[@AXDescription='clear']"
(a predicate is not a simple string or integer, and specifies more than one node)

"/AXApplication/AXMenuBarItems/AXMenuBarItem[@AXTitle='View']/AXMenu/AXMenuItem[@AXTitle='Basic' and@AXMenuItemMarkChar!='']"
(leading and trailing spaces required for the boolean operator)

"/AXApplication[@AXTitle="Calculator"]/AXWindow[0]/AXButton[@AXDescription='clear' and @AXEnabled='YES' or @AXDescription='clear all']"
(predicate uses multiple kinds of boolean operators; use one or more 'and', or, use one or more 'or', but not both)

Tools for working with AXPath Selectors

This special AXPath restriction is tricky to work with, but we have some tools at our disposal.

First of all, AppiumForMac provides a tool for generating the AXPath of any element on the screen. First, launch the AppiumForMac app manually using Finder or Launchpad. It won't display a window, but will appear in the dock. If you hold the fn key on your keyboard down for about three seconds, AppiumForMac will find the AXPath string to select whichever element your mouse pointer is currently hovering over. It stores the AXPath selector into your clipboard, so you can paste it into your test code. You'll know when it has worked because the AppiumForMac icon jumps out of the dock.

This behavior will work anywhere on your screen, because AppiumForMac can actually automate anything which is available to the Accessibility API.

(NB: Third-party keyboards may not work with this functionality.)

I found the AXPath strings generated by AppiumForMac to be pretty long. Make sure to organize your test so common parts of the string can be reused. I also removed many of the predicates since they were too-specific and not necessary.

Another tool which can help with AXPath strings is the Accessiblity Inspector. This tool will show the hierarchy of accessibility elements, allow you to click on an element to inspect it, and view properties on elements.

As a last resort, you can try to dump the entire view hierarchy by calling driver.getSource(). This works on simple apps, but hung indefinitely on the Activity Monitor app, most likely because the UI is constantly updating.

The Test

Here's an example test which starts the Activity Monitor, switches between tabs, and performs a search:

import io.appium.java_client.AppiumDriver;
import org.junit.After;
import org.junit.Assert;
import org.junit.Before;
import org.junit.Test;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.remote.DesiredCapabilities;

import java.io.IOException;
import java.net.URL;
import java.util.concurrent.TimeUnit;

public class Edition052_Automate_Mac {

    private AppiumDriver driver;

    @Before
    public void setUp() throws IOException {
        DesiredCapabilities caps = new DesiredCapabilities();

        caps.setCapability("platformName", "Mac");
        caps.setCapability("deviceName", "Mac");

        caps.setCapability("app", "Activity Monitor");
        caps.setCapability("newCommandTimeout", 300);
        driver = new AppiumDriver(new URL("http://localhost:4723/wd/hub"), caps);
        driver.manage().timeouts().implicitlyWait(5, TimeUnit.SECONDS);
    }

    @After
    public void tearDown() {
        try {
            driver.quit();
        } catch (Exception ign) {}
    }

    @Test
    public void testActivityMonitor() {
        String baseAXPath = "/AXApplication[@AXTitle='Activity Monitor']/AXWindow";
        String tabSelectorTemplate = baseAXPath + "/AXToolbar/AXGroup/AXRadioGroup/AXRadioButton[@AXTitle='%s']";
        driver.findElementByXPath(String.format(tabSelectorTemplate, "Memory")).click();
        driver.findElementByXPath(String.format(tabSelectorTemplate, "Energy")).click();
        driver.findElementByXPath(String.format(tabSelectorTemplate, "Disk")).click();
        driver.findElementByXPath(String.format(tabSelectorTemplate, "Network")).click();
        driver.findElementByXPath(String.format(tabSelectorTemplate, "CPU")).click();

        WebElement searchField = driver.findElementByXPath(baseAXPath + "/AXToolbar/AXGroup/AXTextField[@AXSubrole='AXSearchField']");
        searchField.sendKeys("Activity Monitor");

        WebElement firstRow = driver.findElementByXPath(baseAXPath + "/AXScrollArea/AXOutline/AXRow[0]/AXStaticText");

        Assert.assertEquals(" Activity Monitor", firstRow.getText());
    }
}

(As always, the full code sample is also up on GitHub)

Caveats

AppiumForMac is rough around the edges, probably because it does not currently have a lot of community use. Check the Github issues if you get stuck.

Nothing prevents AppiumForMac from being improved; contributions welcome!