Working with the iOS Home Screen (Springboard)

Sometimes you want to automate an iOS device, but don't want to automate any app in particular, or want to start from the homescreen as part of a multi-app flow, or simply want to automate a set of built-in apps the way a user would approach things. In this case, it's actually possible to start an Appium iOS session without a specific app.

To do this we make use of the concept of the iOS Springboard, which is essentially another word for the home screen. The Springboard is essentially an app, though it's one that can't be terminated. As an "app", it has its own bundle ID: com.apple.springboard! So, we can actually use this to start an Appium session without referring to any "real" app in particular:

  
capabilities.setCapability("app", "com.apple.springboard");

Check out: Tips for Speeding Up Your Remote Appium Sessions

On its own, however, this isn't going to work, because Appium will try to launch this app, and deep in the XCUITest code related to app launching is some logic that makes sure the app is terminated before launch. As I mentioned earlier, the Springboard can't be terminated, so trying to start an Appium session this way will lead to a hanging server. What we can do is include another capability, autoLaunch, and set it to false, which tells Appium not to bother with initializing and launching the app, but just to start a session and give you back control immediately:

  
capabilities.setCapability("autoLaunch", false);

Also check: Launch High Performing Mobile Gaming Apps with Confidence

At this point, starting an Appium session in this way will drop you at the Springboard. It won't necessarily drop you at any particular page of the Springboard, however. If you're an iOS user, you'll know that the "home screen" is really a whole set of screens, depending on how many apps you have and how you've organized them. One of the main things you would want to do from the home screen is find and interact with an icon for a given app. So how can we do this?

Finding App Icons

Ultimately, what would be nice to have code-wise is a handy custom expected condition. Let's imagine that this is our test method implementation, for example:

  
@Test
public void testSpringboard() {
    wait.until(AppIconPresent("Reminders")).click();
    pressHome();
    wait.until(AppIconPresent("Contacts")).click();
    pressHome();
}

Here we have created a custom expected condition called AppIconPresent, which takes the app icon text, and will attempt to find that icon, navigating through the different "pages" of the Springboard if the icon is not already present. This is actually conceptually a bit tricky, because of how the Springboard app is implemented. No matter how many pages you have in your Springboard, all pages show up within the current UI hierarchy. This means it is easy to find an icon for an app even if it's not on the currently-displayed page. However, if you try to tap that icon, it will not work (because the icon is not actually visible). So, we need some way of figuring out how to move to the correct page before tapping. Let's examine the implementation of AppIconPresent:

  
protected ExpectedCondition AppIconPresent(final String appName) {
    pressHome();
    curPage = 1;
    return new ExpectedCondition() {
        @Override
        public WebElement apply(WebDriver driver) {
            try {
                return driver.findElement(By.xpath(
                    "//*[@name='Home screen icons']" +
                    "//XCUIElementTypeIcon[" + curPage + "]" +
                    "/XCUIElementTypeIcon[@name='" + appName + "']"
                ));
            } catch (NoSuchElementException err) {
                swipeToNextScreen();
                curPage += 1;
                throw err;
            }
        }
    };
}

Read: Using a Tap-By-Location Helper for Working with Unresponsive Elements

The first thing we do is call our pressHome helper method (which is just another method implemented in the current class). It looks like this:

  
protected void pressHome() {
    driver.executeScript("mobile: pressButton", ImmutableMap.of("name", "home"));
}

(I just have it as a nice helper to avoid redefining the executeScript args all the time). What calling pressHome here does is ensure that we are always on the first page of the Springboard. Then, we set a class field to define what page we are on. We initialize it to 1, because after pressing the home button, we know we are on the first page. Then, in our actual condition check implementation, we try to find an icon that has the name we've been given to find.

Also read: How (And Why) To Create a Custom Expected Condition

Here's the tricky part: we don't want to just find any icon that has the correct name (because then we would find the icon even if it's not on the current page). We only want to find an icon on the current page (and then swipe to the next page if we can't find it). To do that, we take advantage of a fact about Springboard's UI hierarchy, which is that each page is actually coded up as an XCUIElementTypeIcon, which contains the actual app icons as children. So given that we know what page we're on, we can write an XPath query that restricts our search to the XML nodes corresponding to the current page!

See: Writing XPath Queries That Work

If we're unable to find an icon on the current page, we call another helper method, swipeToNextScreen. This is built off of the handy swipe helper developed in an earlier post, and has a simple implementation that just performs a swipe from an area near the right edge of the screen over to the left:

  
protected void swipeToNextScreen() {
    swipe(0.9, 0.5, 0.1, 0.5, Duration.ofMillis(750));
}

Once we've swiped to the next screen, we increment our page counter because we have now moved to the next screen! Of course, we're relying on the assumption that we will eventually find the app by the time we reach the last page, because we don't have any logic to detect whether our swipeToNextScreen was actually successful.

Also see: How to Find Elements in iOS (Not) By XPath

In general, AppIconPresent is a great example of a useful custom expected condition that has a side effect. We build it into an expected condition so we can use it flexibly with the WebDriverWait interface, and so we don't need to write any of the looping or retry logic ourselves. It's also a great example of being clever (and hopefully not too clever) with XPath!

So that's it for automating the iOS home screen or "Springboard" app. Let me know if you find any other interesting uses for automating this app beyond using it as a launcher or a quick way to get an Appium session on a device! And as always, you can find the full sample code for this edition up on GitHub.