Determing Element Locators

Determining Element Selectors

Hello everyone, and welcome back to the Appium Pro intro workshop. In this module, we're going to talk about determining selectors. How do we figure out which elements are available in our app for us to find, and how do we find them once we know they're there?

Well, there are two main ways to do this. The first is sort of the lower level way, and it's to use our driver instance and call the getPageSource method on it. And getPageSource will deliver us a big string of XML representing the user interface hierarchy at the point in time at which we asked for it.

We can then manually inspect that XML document. We can look at it, we can see attributes of the various elements, and we can use those attributes to figure out what selectors we can use. There is, though, a much better way, and that's to use Appium Desktop.

Appium Desktop is the graphical version of Appium that we installed back in the setup video, and it also comes with something that is more than just an Appium server wrapped up in some buttons and widgets. It comes with a graphical inspector tool that lets you get screenshots of your application and move your mouse around and see which elements exist in TheApp and inspect their attributes that Appium has access to.

Using driver.getPageSource()

So, we're going to take a look at how to do both of these things. So, first, we'll do things the hard way and try and find element selectors using the XML source output. So, as an example of this, we can load up the AndroidSourceTest.java file.

I'll go back to my project and open up AndroidSourceTest.java. And you can see that it extends our test base class. It's loading up the Android capabilities for the app. And finally, it has only one command in the actual test body, which is driver.getPageSource.

Then, we are printing the output of that command to the console so we can see it. So, we're not really doing anything in this test, we're just getting the state of the application at the point in time at which we ask for it. All right. Let's have a look at this. I'm going to go ahead and quit this, and we will run the test.

Before we run tests, it's always a good idea to make sure that we have the Appium server running. So, here's my server. It's running. And I'll make sure that my Android emulator is up, which it is. Okay. So, now, using our test runner, whether you're using IntelliJ or just Gradle from the command line like me, we can run the AndroidSourceTest.

And Appium should soon receive our command, and then we will see our app open just momentarily, and then shut back down once we've had a chance to retrieve the page source from it. There we go. So, we can see that we have a nice bit of XML that's output for us.

It's actually a little bit too hard to read in this format, so we'd probably want to bring it into a file that helps us to print it and format it and inspect it more easily. I've done that here. So, here's what the UI hierarchy of the application might look like if it was formatted a little bit more prettily.

So, the question is, what do we do with all of this stuff? Well, as we look through the various XML nodes that we have access to here, we see a lot of elements that aren't really relevant for a user of the application. We've got frame layouts and linear layouts and relative layouts and view groups and this is all kind of under the hood stuff that doesn't really map to elements that a user would interact with.

A user would interact with something like a button or a text field, but not a linear layout, whatever that is. That seems like it's more for developers. So, let's try and find an element that seems useful to a user. Okay. I see one right in the sort of middle of this big chunk. It's android.widget.TextView.

Text view sounds reasonable. So, we can look at some of its attributes. Well, it has a text attribute, and the text attribute that we see is called TheApp, so that sounds promising. That sounds like something that's being displayed in our application. There's a lot of other stuff going on here.

Let's see if there's any other elements that might be interesting. Down at the very bottom, we see two text views. The first one is called "Echo Box". We can see the text attribute is "Echo Box". It has a class, a package and a content description, although the content description is empty.

So, if we look up one level here, we see a view group, which is containing both of these texts views, and we see that it has a content description of "Echo Box". Now, on Android, content description is an alias for accessibility ID. So, the fact that I see Echo Box here in the content description attribute means I could find this element, the android.view.viewgroup element by the accessibility ID of Echo Box.

So, that is how we would basically use the XML output to find the elements that we want to interact with.

Using Appium Desktop

Okay. Now, let's talk about examining the UI hierarchy with Appium desktop, instead of using the page source command. And you can see that Appium Desktop looks quite a bit nicer. It actually has the whole XML output here for us to see as well. But on top of that, it has a nice graphical way of navigating through the application and will even suggest a locator strategies and selectors for us. So, let's have a little hands-on exercise here and get Appium Desktop going.

The New Session Window

So, here's that Appium Desktop. I'm actually not going to start an Appium Desktop server because I've got an Appium server running here on my terminal, and so I don't need to start another one for Appium Desktop. And instead, what I'm going to do is go up to the Appium menu and click new session window.

I could also hit control-N or command-N. And this will open up a window that lets me enter in a bunch of desired capabilities and start an Appium inspector session. So, we can see that the default tab here is that I have a selected custom server, and for remote host and remote port, I had put the host and port of my Appium server that I have running locally, which has local host and 4723.

And now, I need to enter in desired capabilities to start a session. So, what desired capabilities should I enter? Basically, I should enter the same ones that I use to start a session in code. So, I can actually go and look at what those are. Go back into my project file.

And this was all encapsulated in ProjectCapabilities.java. So, we're dealing with an Android test. So, I'm going to use these capabilities. Move this over. So, I'm going to use a platform name capability of Android, a deviceName capability of Android emulator and automationName capability of UIAutomator2.

Now, that's not the only capability that we're going to need, because we are also going to want the capabilities that refer specifically to my application. So, in this case, we're working with TheAppAndroid, and so we're going to need these two capabilities as well. Okay. The App capability.

And so, I need the absolute path to the application on my system. So, on Mac and Linux, I can find it by navigating into The App's directory in my project and typing pwd for present working directory. And then, I can copy this. That's the directory and the name of the file inside it that I care about is TheApp.apk.

Now, the other capability that I needed was app wait activity, so that capabilities app wait activity, and I can paste in the value here, com.reactnativenavigation.controllers.navigationactivity. That has to do with how I built this application using React Native. Okay. Now, I can hit start session.

The Inspector Window

And we can see that the Appium server is active because it's receiving the new session request from Appium Desktop. So, once Appium Desktop does its thing, we will get this view here with a screenshot of our application, and I can actually move my mouse around this application and work with the different elements.

I can refresh the screenshot and the source so that my single elements can always be in sync. And I can even tap on specific elements here and get information about them over in the selected element view. So, this is actually how we use Appium Desktop to discover element locators.

I've tapped on this list item here, and I see that at the top of the selected element view on the right, Appium Desktop actually suggests different locator strategies and selectors. So, it's saying that we could find this by accessibility ID, using a selector of login screen, or we could find it by XPath using this longer selector.

Now, it's a good idea to just use accessibility ID whenever it's available, so that's what we can do. We can actually do more than find elements with Appium Desktop, we can interact with them. So, if I wanted to tap this element, I could just click on tap here, and that will send a command to cause TheApp to receive the tap and move on to this next view here.

If I want, I can tap on this editable text field and even send a user name into it like so. Now, this is not my emulator. This is just a screenshot of it. And actually, if I want, I can still have my emulator up. I can interact with it like normal. And if I do that, then I can always come back to Appium Desktop and refresh the source to get back to the latest state.

There is a lot more we can do with Appium Desktop. We can explore the whole XML hierarchy. We can run a bunch of different specific Appium commands. We can even record our actions and generate code from them. But for now, what we care about is just figuring out how to find elements using Appium Desktop.

So again, to do that, we can click on the element in the screenshot that we care about finding, and then go over here to see what Appium desktop might recommend as the locator strategy and selector. Okay. That's everything we wanted to cover in this module. You now know, not only how to find elements in code, but how to know which selectors you should use to find them.

If there is an element that can be found, it will show up in the UI hierarchy. It will show up in the XML output, so you would see it in the page source, and you would also see it in Appium Desktop. Okay. Let's hang on until next time when we will actually begin doing things with all these elements that we've been working on finding.

‍