Appium aficionado Jonathan Lipps joined the HeadSpin team in June for a webinar about testing React Native apps with Appium, the pros and cons of using React Native for app development, making apps testable by Appium, and a comparison between Appium and other app-testing approaches like Jest and Detox.
Jonathan Lipps runs a consulting firm called Cloud Grey - they handle mobile automation strategy primarily for enterprise clients - and has been writing code for Appium since early 2013 when it was still new.
Below is a brief recap of the webinar. For a detailed account of the topics above, watch the webinar included at the end of this recap!
First and foremost, native apps are perceived as faster and better than hybrid or web apps. Writing native apps involves using native SDKs provided by the operating system vendors themselves, meaning they have access to all the lowest-level APIs available on a given device. The drawback here is that if you're developing an app for both iOS and Android users, you'll have to develop two separate code bases, doubling the work involved.
And, here's a visual breaking that down a bit.
So, what's unique about running Appium tests on apps that have been developed with React Native?
"Just like starting an Android test using a set of desire capabilities with Appium and passing in the path to your APK file, you [progress through the] same steps when dealing with React Native," said Jonathan Lipps. There aren't any specific capabilities or commands that need to be used. And, the same goes for iOS: just use your standard iOS platform capabilities and a path to your React Native application.
The following tables discuss Appium versus other app-testing approaches like Jest and Detox.
When it comes to React Native, people use a variety of tools when testing apps.
These resources provide a wealth of information on Appium and React Native.1. Wim Selles's AppiumConf 2018 talk on Appium + React Native
2. React Native Accessibility Docs
3. Jonathan Lipps's React Native test app
4. Jonathan Lipps's weekly newsletter Appium Pro
The following webinar (and transcript) provides a detailed account of how React Native works, the pros and cons of React Native for app development, making apps testable by Appium, and a comparison between Appium and other app-testing approaches like Jest and Detox.
0:05 Well, good morning everyone. This is Jonathan Lipps and I'm here with the HeadSpin team and we are going to have a little half hour webinar this morning talking about testing React Native apps with Appium. So if you're just joining us, don't worry, I'll be doing intro stuff for the next minute or so, so you can get in and get settled and run and make that last trip for coffee if that's what you need to do. Yeah, let's take it away. So this is me here. I work for a company - I shouldn't say I work for it - I basically am the entire company, I have a consulting firm called Cloud Grey. We do mobile automation strategy for primarily enterprise clients helping them get their mobile automation off the ground and meeting whatever challenges they have.
I've been working with Appium for a long time. I started writing the code for Appium back when it was a new thing in early 2013. So, it's been six and a half years of hacking on Appium. I know an awful lot about that, but had the chance in the course of preparing for this webinar to branch outside of Appium a little bit because when it comes to React Native, there are several other testing tools and methods that people tend to use, so I was able to have a look at those and at the end of the webinar will talk about how they compare with Appium and when you might want to use one or the other.
Let's dive into the first topic today, which is a brief discussion of what React Native is. I think we all have some idea whether or not we've used React Native that it has something to do with mobile applications and React - that's kind of right there in the name - but I think it's helpful to know a little more about how React Native apps works. so we're going to talk about that.
Just briefly, the motivation for React Native comes out of this fact - I suppose it's a fact, it's debatable - but there’s places out there where people have done research that claim, anyway, that from a user perspective, Native apps are perceived as faster and better than the alternatives of hybrid or web apps, and we've had high-profile cases of companies like Facebook taking their app from a hybrid version and doubling down on Native.
There are certainly examples of that and in a way it makes sense. This is because writing Native apps means using the Native software development kits provided by the operating system vendors, so they have access to all of the lowest level APIs that are available on a particular device. This does mean, however, that if you're trying to develop an app and get it into the hands of both iPhone and Android users, you have to write code for two different platforms, maintain two, totally separate code bases, which basically doubles the amount of people you need to hire to get the project off the ground, and doubles the amount of work you need to do. You might be able to share some design assets and maybe it's not exactly double in all respects, but in terms of the code, it's a lot of extra work.
From a development perspective, as developers, we tend to prefer writing as little code as possible, and for good reason, because code has bugs, so we want as little code as possible. And so when it comes to writing a mobile app, as developers, we would certainly prefer to have a single code base and would obviously prefer to use tools that we already know how to use. It's all good and fine to learn Swift or Kotlin and do iOS or Android development and that might be fun, but if we have a job to do and we're on a schedule and our skill set is in web development, we might prefer to work with web development tools. Indeed, that's the motivation for hybrid apps in the first place, and [that’s] why projects like Phonegap or Ionic or these types of things really focused on letting developers use web development technologies to produce mobile applications.
How does React Native do this awesome feet? Well, the React Native code base maintains a set of Native modules. This is code, which is written in Java for Android or Objective-C, or Swift for iOS, or the import libraries and frameworks from the Android and iOS development kits. These modules run whenever you launch an app, which has been developed with React Native.
5:44 These are just pure native bits of code: you didn't write these bits of code. It's part of the React Native project, but it gets bundled in whenever you use React Native to create your own app, so this is the stuff that actually gets launched whenever you launch a React Native app, and that's why the iOS and Android operating systems believe your app to be native, because it is, from the perspective of the operating system.
9:59 In reality, it's a little more complex because there's often platform specific configuration or customization that needs to happen to make your app work exactly the way you want it to on each of the two platforms, and that platform-specific configuration needs to be maintained in a platform-specific way.
So, for iOS, we have our info.plist file, which has a bunch of information about our iOS application.
For Android, we have our Androidmanifest.xml, which we might have to maintain little bits of information in.
This is actually part of the design of React Native: its developers don't claim that you can have a purely single code base running on both platforms, but that the core of it can be shared in this one place. And then, if you have any specific modifications, you can keep those small and easy to update. So, this is what it's like as a React Native developer.
What are the pros and cons of choosing this method for app development?
Well, there's a lot of positive aspects of it and I'm actually a fan of React Native personally. I've used it for my own app development and I like it quite a lot.
The good things about React Native are that you have mostly a single code base, as we talked about, and there’s mostly no need to learn iOS or Android app development. I've learned just enough to make my React Native apps and do the kind of stuff that I do with Appium, but I didn't have to do a deep dive into iOS or Android app development to write my React Native app.
Another nice thing about React Native is that it's extensible. If you want to do something to tie into the Native operating systems that is not yet available in React Native’s native module space, you can add your own extension and then just use it and share it with other people as well.
13:27 The cons of React Native development are, as you can tell on the list of pros, that there's a lot of “mosts”. So, it's not a perfect solution and you do need to get your hands dirty with a little bit of iOS and Android experience to be successful. As I said, we have this mostly Native app speed, mostly Native user experience, but every once in a while, you might run into a few performance issues or design issues as a result of trying to design a cross-platform application, things like that.
You're also reliant on the React Native team to update React Native whenever there are new versions of iOS and Android and to make sure that it keeps working with all those new versions, and that could be problematic.
14:15 In terms of the user experience itself - this is more of a conceptual issue just to do with writing a cross-platform app using any framework - but iOS and Android apps are different and iOS and Android users have different expectations about how apps should look and behave.
If you're writing one app, one UI for both platforms, you might run into problems where neither platform’s user base really thinks that you know what's going on, that you know how to design for them specifically.
So, you might wind up in a spot where you have a lot of conditionals: if it's Android do it like this, if it's iOS do it like that, whether it's a visual design thing or a functionality thing.
You might end up losing some of the benefits of a cross-platform code base by having a lot of these conditionals sprinkled in, but in practice, if you're willing to sacrifice a bit of that design or user experience, you can benefit a lot from the shared code base.
Finally, it's good for everyone to know that React Native is owned by Facebook. It used to have kind of a shady license actually, but as of sometime last year, they switched it to the MIT license, which is a great open-source license, so you can feel very comfortable using it. But, in terms of the future of React Native, it is a Facebook project, so who's to say what they'll do with the down the road. So, that's [a summary of] React Native.
Now, let's talk about Appium. If you haven't heard of that before, we're not really going to talk about it in this webinar here, but it's an open-source mobile automation library that people use for testing native, hybrid and mobile web applications.
16:05 If you are relatively new to mobile automation and you want to learn about Appium, I actually just released a course on starting Appium from scratch yesterday, in conjunction with LinkedIn. Here’s the URL - you can check it out. That's about two hours long and takes you through Appium set-up and all the basics. This will teach you how to write tests for native mobile apps with Appium in general.
Let's talk specifically about React Native and Appium and what is interesting or unique to running Appium tests on apps that have been developed with React Native.
The way that Appium works as it just uses the standard automation tools that have been provided by Apple and Google. So, from the perspective of those tools, everything just looks like a normal native application. All the elements are Native elements and you can typically automate a React Native application without too much trouble just as it is. So, just like you would start an Android test using a set of desire capabilities like this with Appium and you pass in the path to your APK file - all of this is exactly the same when you're dealing with React Native.
There's no special capabilities you need to use, no special commands you need to use, and same goes for iOS. If you build your React Native debug application, you can certainly load it into an Appium session using capabilities something like this: your standard iOS platform capabilities and a path to your React Native application.
18:14 Okay, so what is different?
Well, the one thing that you need to worry about is making sure that your React Native app is appropriately testable. There's just a few simple things you can do to make sure that this is the case.
Here's an example of a React Native component. It's an input field and it has a bunch of attributes. You can see that this input field has a certain style, certain placeholder text, and function, which is executed anytime the text in the field is changed. So, it calls a setState on the component. This is just a very basic React pattern.
The interesting attribute here is the one called testId. We're basically telling React Native that we're interested in this component when we're running UI tests of this application, and that we want React Native to make sure to put this testId somewhere on the Native component, which implements this React component, so that when we go to do UI testing, we can find the Native component appropriately. This works pretty well.
If we take a look at an iOS version of React Native - here's an application which I developed using React Native on the left. You can see a screenshot of the application and I'm using Appium Desktop’s inspector here to inspect the Native application. I've just loaded it up again without any React Native specific capabilities just like any other application that I'm loading up with Appium Desktop.
We can see that, just by including the testId in my React Native component, I can find the iOS component, which maps to that React component, because it has the appropriate name here. In other words, messageinput. I could very happily automate a test of this view using Appium by finding this element by its accessibility ID of messageinput and sending keystrokes to it and going on. So, that's very straightforward.
Now, let's say I loaded up the Android version of this application. Here's the Android version. I'm on a different view, but it's the same idea. We can see that there is sort of an interesting issue happening with the Android version of this application. The issue is that I have this whole list of buttons I can tap to go to different views of my application, but from the perspective of Appium, from the perspective of the Android operating system, I don't see all these elements. All I see is one frame layout which has no children. So the children aren't available here.
This is a problem because now I can't find the element that I set a testID on. The reason for this is that, for React Native for Android, when you assign a testID to a component, it puts that ID in something called a view tag of an Android element. Unfortunately, the standard Android automation technology that Appium uses doesn't know anything about view tags and can't access them. React Native does this because many Android developers use something called Espresso to test their applications and Espresso can access data inside view tags. From the React Native perspective, you're supposed to use Espresso to test your applications, and that's why they did it that way.
21:51 So what can we do to make this application testable by Appium?
Well, what we need to do is make sure that, in addition to setting the testID for the component we want to test, we should also set the accessibility label, at least on Android. Now, the accessibility label is a label that shows up to the operating system for accessibility purposes so that the operating system can make sure to surface that information to someone who's using the app in some kind of accessibility mode.
Of course, you want to be careful here, since you are setting accessibility labels, to set it to something that an actual human user would appreciate reading or hearing spoken to them. So, when we do this, if we reload the Android application, we’ll see that, voila! All of these elements which I've assigned test properties to can now be accessed in the hierarchy. Again, I could just write up a normal Appium script and test this application now without further ado.
23:29 There's another strategy for dealing with this issue, and that's making sure to set the accessible attributes appropriately. As a rule of thumb, if React Native believes that an element has an accessibility property, it won't show you the child elements. I guess, [this is done so as to] not confuse the operating system to make sure that only one element in a hierarchy at a time is able to have accessibility attributes on it.
Let's take a look at this component. Here is a component which has a scroll view and a text header and a list inside of it. This is actually the component that defines the view we just saw here. This is the list view that we're seeing now in code.
24:17 So what I've done is I've added test properties, but what I've also done is made sure that the scroll view, which is the parent element, has its accessible attribute set to false. That allows everything underneath it to actually show up in the accessibility hierarchy. React Native doesn't mask it in this case. These are two strategies which you can mix and match to make sure that all the elements that you want to interact with are actually findable by Appium, and that's actually it. That's really the only thing that I've come across in testing React Native applications that you need to worry about. Everything else is just standard Appium scripting.
Here's a little video of a very simple test script I wrote for my React Native application, which is obviously cross-platform, just logging in to the secret area of the application. I wrote the same test code using Appium to automate both platforms of this React Native application. That's Appium.
Let's close by briefly discussing Appium versus other approaches, because in the React Native world, people use a set of different tools for testing their apps.
But, this is a great way to test many internal aspects of your application not having to rely on Appium or some other tool to do UI tests of your React Native app, which can be a lot slower than these kinds of tests. And, because this is a unit testing model, all this testing happens via the command line: there were no simulators or devices that are loaded - this is all just in memory testing.
27:30 For end-to-end testing, there's a tool called Detox, which is produced by a company called Wix, and it's built on a couple other gray box testing technologies from Google. There's Espresso for Android, which is Android’s first-class test automation framework. Then, there is a framework called Earl Grey, which is kind of an Espresso clone for iOS that Google developed.
I said Detox is gray box, which differentiates it from Appium, which is a black box testing model. Gray box tests live inside the app that you're trying to test, which means that your app will undergo modification. So, if you're not using React Native, you'll have to build Detox into your app explicitly. If you are using React Native, Detox will do that for you using their command line tool.
28:26 Either way, the app under test is modified to support Detox having access to the internals of the application. Why does it need access to the internals of your app? Because it wants to use Earl Grey and Espresso to make sure that your app is in a state where it can be interacted with. One common problem with UI tests is that you try and interact with your app at a point in time when your app isn't actually interactable: the view might be loading, an element might not be available, there might be a spinner, there might be a network request.
There's a lot of issues in writing UI tests for web and mobile applications that have to do with making sure that your app is in the correct state. Detox and Earl Grey and Espresso try and get around this by living inside of your app under test so that they actually know, from the app’s perspective, whether or not anything is happening, and they can defer the execution of any test steps until a point at which your app has stopped doing something. This makes it easier to write tests scripts, because you don't have to worry much about, waiting for application states.
Detox claims, and it’s probably true, that it leads to greater speed and stability of your tests. Actually, Detox has special support for React Native and it hooks into multiple layers of React Native to make sure that it waits for appropriate moments and the React Native flow for app interaction as well. So, of all of the UI testing tools that are out there, Detox is the one that's designed most specifically to work with React Native.
30:05 Let's look at Appium versus Detox. We didn't do a big overview of Appium, but I’m assuming that you know a little bit about it. Let's talk about when you might use Appium versus when you might use Detox.
Based on the attributes that I was just sharing with you, I would argue that you should use Appium. If you have a situation where maybe a QA team is writing the test and not the developers themselves (because, of course, it might be difficult to modify the app if you're just receiving it from developers and responsible for testing it).
31:10 Of course, if you need to test, let's say, a hybrid app or a web app, Appium has a really great story with its web driver-based protocol of automating those other app modes, but Detox is less concerned about that. But, you might use Detox if you are React Native developer who is writing the tests yourself because there's a really nice integration there; or, if the idol synchronization features of Detox are really your highest priorities and you just have no time for learning how to wait for app states appropriately with Appium.
I personally have written a lot about how to do that appropriately with Appium, and I don't think it's too challenging. I think you can do it successfully, but there is certainly something to be said for a framework that lives inside your app and actually knows with a high degree of certainty when your app is idle and can receive new test commands. And then, Detox has a really great integration story with the React Native workflow. So, if that's a need that you have, Detox might come out on top for you.
That's basically what we're going to talk about today. Some further resources for you to take a look at:
- There was a great talk at last year's Appium conference on Appium and React Native by Wim Selles. You can take a look at that.
- I had a lot of fun reading the React Native Accessibility Docs - that's how I learned some of the stuff that I shared with you today.
- [Jonathan Lipps's] React Native test app. If you want to play around with it and look at the code and see how we React Native app is developed, the code is up there on GitHub - it's all open source
- [Jonathan Lipps's] newsletter called Appium Pro, which is a weekly newsletter that comes out with a new Appium tip or tutorial every week. Most of the code samples involve automation of the particular app that we already saw an example of - it's written in React Native. So, if you read the code for the code samples for Appium Pro tests, you'll see examples of automating React Native applications. And again, it’s basically the same as automating any other kind of application with Appium.
Q: Should I always use testID, or can I use class instead? I tried to automate without testID using class, but was too slow.
A: Appium tries to find elements using a locator strategy, and Appium has a number of different locator strategies. The most commonly recommended strategy is to find an element by its accessibility ID. And as we saw, when you set a test ID in your React Native application, it comes out as an accessibility ID on iOS, and then if you make sure to set the accessibility label as well that also comes out as an accessibility ID on Android.
That's by far the easiest way to find elements using Appium with React Native is to set both testID and accessibility label. Appium has another locator strategy calls class name, but that doesn't find an element by an attribute that you set as class on a React Native component. What it does is it finds an element by its Native UI object class, so there are Native UI objects called Android.widget.edittext, or on iOS UI elements text field.
If you use Appium’s class name locate a strategy, you'll be finding elements by that criteria. So, unfortunately, if you set the class property or attribute in your React Native component, as far as I'm aware, that doesn't make it into the Native component in any way which is discoverable by Appium. So, I would say that yes, you should use testID and accessibilityLabel when trying to make sure that your React Native app is testable with Appium.
Q: Can React Native apps be tested in WebView context, like hybrid apps?
Q: Does WebDriver weight class work for the React Native app?
A: Yes, it does. That would be my recommended strategy: using Appium with React Native. Make sure, just like with any other application you're testing, you use WebDriver weights appropriately to wait for your application to be in the correct state, whether that's waiting for an element to be present or not present, or a view to be loaded, etc.
Q: How many UI load-synchronization-related delays are there if we compare Appium versus Detox or Appium versus Espresso?
A: Detox uses espresso, so it's the same comparison. Using Detox or Espresso, the test will run faster and it will run without needing to use any kind of weight on your test code itself. That's the benefit of the gray box testing model: because Espresso, and therefore Detox, runs in memory with your application, they're able to ask your application whether it’s in the midst of downloading or animating something, or moving to a new view, for example. If so, it ensures that no more test apps are executed until the view is idle, which tends to make tests more stable and fast.
With Appium, there are a few different options.
- You can use Appium’s default Android driver, which works from the outside of your application and therefore doesn't have any access to knowledge about whether your app is working on something or not. In this case, you have to code that knowledge into your test script using WebDriver weights
- There is an Espresso driver for Appium, which offers the benefit of the Appium testing model, and all the benefits of Espresso (especially idle synchronization).
Using the gray box testing model, you can also interact with code inside of your application or call methods inside your app directly without going through the UI.
Q: Which would you recommend: WebDriverIO or WebDriver?