Appium Overview

Overview of Appium

‍

Hello everyone and welcome back to the Appium Pro intro workshop. In this first unit, we are going to give an overview of Appium, the open source mobile automation tool. So let's dive on in.

What is Appium? Well, Appium, as I just mentioned, is open source, meaning you can go and find all of its code on GitHub. You can also participate in the development of Appium itself. It has an open development methodology. It is cross platform, meaning it runs on iOS and Android. It also runs on other platforms that aren't mobile platforms, for example, Windows and Mac, but in this workshop we're focusing on Appium for mobile automation. So it is cross-platform across iOS and Android, and it is an automation tool. I call it an automation tool, not really a framework or a test framework per se, because it really has just one main purpose, which is to let you control your mobile devices and applications remotely from scripts that you write.

Now, people usually use Appium for testing, but this isn't the only purpose that you could use Appium for. What are some other things you could use Appium for that aren't testing? Well, you could write a script to teach Appium to play your favorite mobile game. There's a lot of games out there that involve repetitive behaviors. You could teach Appium to tap buttons for you and play the game while you're taking a nap so that when you wake up, your character has leveled up. That's not really testing. You're not working for the game company trying to improve quality, but it is automation. So when you think of Appium, think about automation. And the reason most of us are thinking about automation is for testing or quality assurance, but it's important to keep these concepts distinct in our mind. Automation and testing are not the same thing, although they can be used together, and that is no doubt why you're watching this video.

Appium as an automation tool and more

So in a little bit more detail, what Appium is, is a program, a tool, that wraps other tools that already exist that do useful things for us. Specifically, these other tools or frameworks provide automation of mobile apps and mobile devices. So there are some automation tools for Android and for iOS. For Android there's a tool called UiAutomator2, which is released by Google. And there's a tool called Espresso, which is also released by Google. Apple has something called XCUITest that they released as part of their iOS and Mac software development kits that you can use from within Xcode.

So these are sort of preexisting automation tools for the platforms that we care about, and all Appium does is kind of wrap those existing frameworks up in a different package. And I'll share in a bit why we want to do this, why we want a different package, but let's first just say what that package is. It is called the WebDriver API. And it's also called the Selenium API. Selenium is the historical name of the project that pioneered web browser automation, and pioneered the WebDriver API, but a lot of people just refer to it as Selenium. And you can already see that Selenium sounds kind of similar to Appium, and in fact these two projects are quite closely related.

So continuing to discuss Appium itself though, Appium allows us to author test scripts or automation scripts in any programming language. Java, Python, C#, JavaScript, PHP, Ruby, Haskell, whatever we care about, we can probably find an implementation of the Appium protocol in that programming language. Because Appium is really just an automation tool, we can import it and use it inside of any test framework. And we can integrate it with any CI system. It's not really tied specifically to one particular test framework, so if you're coming from the world of Java, you might be familiar with test frameworks like JUnit or TestNG. Appium isn't specifically related to any of these frameworks. It's just a tool that you can mix and match in with any of your other tools or frameworks that you like using.

More than all these things, or at least besides all these things, Appium is really a massive community of users that have figured out how to be successful with Appium in automating their mobile apps. And this has meant that they've been able to build up a huge knowledge base of how to use Appium, of how to solve particular problems on particular platforms and so on. And so this amazing community is really what has given Appium the longevity that it has. So it's an important point about Appium, and it isn't really a technical point, it's more of a social point, but a very important point, that there's a lot of people that use Appium and a lot of people that are asking questions on Stack Overflow and answering questions on Stack Overflow and so on. So you can get help with Appium from the community, which is really useful when you're getting started, and when you're struggling with specific issues.

So that's kind of what Appium is. I think it's also important to talk about what its main features are for mobile automation, mobile testing. Because that's probably, again, why you're in this workshop. To learn about testing your mobile apps. So what can Appium offer you on that level?

Appium's features for mobile automation

Well, as we mentioned before, Appium is cross-platform. It supports both iOS and Android. It supports all the major types of applications that exist. Native applications, which are built with the native SDK's released by Apple and Google. Web applications, which are just websites or web apps that you access using a mobile browser like mobile Safari or mobile Chrome. And hybrid apps. And we'll talk more about hybrid apps in the full workshop to be released later this year. But basically hybrid apps are apps that have at least one sort of native component, and also have something called a webview, which can display web content that's hosted either locally within the application or that might be hosted on the internet. So hybrid apps are a popular kind of app in addition to native apps.

Appium supports real devices, so if you have real Android or iOS devices laying around, you can plug them into your computer and have Appium run tests on them. Appium also supports running tests on emulators and simulators. So we always make the distinction between emulators and simulators because with Android, we use emulators for our virtual device, and for iOS we use simulators for our virtual device. The distinction isn't super important right now, but you might see these two words around. Emulators and simulators. Basically they're just referring to the fact that we have a virtual device that we can play with and test apps with and build apps for that run on our typical desktop computer or laptop computer, and we can just use them as if they were real devices, but they're simulated or emulated devices.

The kind of test model that Appium goes most successfully with is what we would call a black box testing model, where we are intending to automate the user interface. So we're not trying to automate internals of the application, although that is something that in certain special modes you can do with Appium. The idea is that what we're trying to accomplish with Appium is something very different from a unit test. We're not trying to exercise or test application internals. We're trying to make stuff happen in the user interface the same way that a user would and we want to do the same things a user would do. And then we want to check that the app responded in the ways that we hope it would respond to a real user. And that's called black box testing because we're sort of treating the application like a black box. In other words, something that we can't see inside. It's opaque to us. Just like your user treats your application; it's just a bunch of surfaces of controls and inputs and then outputs that are displayed. The user can't get inside your application to muck around with the internal code. And for the most part, Appium can't either. So Appium is kind of like a robot that represents your user and does the kinds of things that your user might be doing.

So what do users do? Well, they find UI elements. They find the things that you've put on the screen, form controls or text fields or whatever, and they interact with them. They might tap, they might swipe, they might use a keyboard to enter text. They might use their eyes and their brain to read text that's displayed on the screen. So those are the types of things that Appium can do too using the Appium API, which we're going to learn a little bit about in this workshop.

On top of that standard interaction that kind of mirrors what users typically do with your applications, Appium provides a huge suite of extra functionality. We provide features that help you manage applications and application state on the device, installing apps or removing apps, doing different things with the device itself, like changing the screen orientation or retrieving details about the device, like the manufacturer or the version, things like that. A lot of this stuff just ends up being useful either generally or in specific testing requirements. So Appium makes as many of these additional features available as we can, even though sometimes they don't fall into the category of what a user might intentionally do with their fingers or their eyes. They might look like a little more helper utility methods that just make your life a little bit easier as you're trying to set up your test or navigate through your test steps.

All of these features are provided via one API. We already talked about this. It's called the WebDriver API. And this WebDriver API is accessible via a simple client library available in any programming language. So there's a client library for Appium written in Java and Python and all the other languages that I mentioned a few moments ago.

Important Appium links

So just to share some of the important Appium links, I'll put this up on screen. You can always pause it and write down any views that seem important for you. We've got the main Appium website, the Appium documentation. All the Appium source code lives in a variety of repose under github.com/appium. So the main Appium repository is appium/appium under GitHub, and that's also where we try and track all issues, future requests and bugs. So that's where you go if you think you've run into a problem with Appium itself. We also have a discussion forum where users help one another. And then I write a weekly article on Appium where I try and talk about an interesting tip or tutorial or something like that. That's called Appium Pro, and I highly recommend checking out some of the existing articles and subscribing so that you get the new ones in your inbox every week.

Appium's history

Okay. Let's talk a little bit about the Appium project's history. I always think it's interesting to learn about the history, especially of an open source project, so that we can see how it's evolved over time and get an idea of where it might be going. So the glimmerings of Appium came to be a long, long time ago, in internet years, back in 2012. So 2012 was I believe the second Selenium Conference, and it happened to be in London. And there was this gentleman named Dan. Dan was working at a dating app company in San Francisco and he was the QA manager there and had the task of ensuring quality for the iOS version of the application, and found manual testing to be quite tedious, and his team found it to be tedious. And so being a fun-minded and creative engineering type, Dan was trying to find ways to automate iOS applications.

And what he discovered was that there was a tool provided by Apple called Instruments. And Instruments was this basically graphical application that you would launch and you could paste a bunch of JavaScript into a text box inside of this Instruments application in a special region of Instruments called UIAutomation. And what would happen is that Instruments could launch your iOS app on a simulator or something like that and then it would just blast through these JavaScript commands and perform whatever automation they encode it according to Apple's API, that again, was to be implemented in JavaScript.

So what Dan did was figure out a way to actually sort of feed in commands to this text box from the outside, which wasn't supported by Apple, wasn't really in view. Apple clearly didn't intend for people to be using this text box for actual test automation or Continuous Integration. But Dan figured out a way to essentially feed in the commands that he wanted, when he wanted, from the outside. So it kind of opened up the possibility of programmatic automation of iOS apps. And he demonstrated this capability at Selenium Conference 2012 in London as a lightning talk. And there were some funny moments around that lightning talk where it actually ended up almost not happening. So we would be in a much different place today with Appium had that lightning talk not happened, I think. But after Dan gave the demo, well, it was Selenium Conference, it wasn't really about mobile automation, so people kind of thought, "Well that's a fun curiosity." Not too much came of it. He was using it, but more or less it was shelved.

That same year there was another testing conference. This one was really focused on mobile. It's called the Mobile Testing Summit in 2012, and it was hosted by a company called Sauce Labs. And the goal was to bring a bunch of different contributors to the mobile testing space together to talk about what mobile test automation could look like. And I helped put on this conference. I was working at Sauce Labs at the time and so we invited all these different projects and contributors and individuals. And what emerged from that conference was really that none of the existing solutions were good for the long haul. And a guy named Jason, Jason Huggins, who happens to also have been the creator of Selenium, and was a founder of Sauce Labs, was also at that conference. And he recalled Dan's lightning talk from Selenium Conference earlier that year. And he thought, "What if we took what Dan did and we kind of wrapped it in a Selenium WebDriver interface? That would sort of solve a lot of the problems, a lot of the drawbacks that these other projects suffer from."

So they got together over some drinks, and I'm not exactly sure what happened at that meeting, but certainly again, it was a fateful evening because they agreed to work together and share code and ideas and they decided, "Let's move forward with this Selenium WebDriver based iOS automation strategy." And they kind of hacked together a little proof of concept in Python.

Now a few months later, it was decided that this was a really good idea and in fact it was such a good idea that it kind of needed a team to work on it, and not just a couple of folks hacking on it in their spare time. And neither of them had a lot of time to work on it. So I was given leadership of the project at that point and chose to sort of start from scratch and build up Appium in Node.js, for a variety of reasons. There wasn't that much code to begin with at that point. So it was very easy to take a look at what existed and reimplement it in Node.js and then continue on from there. So at that point, Appium became an official open source project that lived on GitHub and that other people could contribute to. So we decided to go global with it. We solicited contributors, started going to conferences, started trying to tell the world about Appium. And that's also when I added Android support, making Appium, I believe at the time the second mobile automation framework to ever be cross platform. The first of which I believe was Cucumber. Although the APIs for Android and iOS on Cucumber were a bit different. So one of Appium's strong selling points at the time was that we used basically the exact same API across both platforms.

In 2014, our big accomplishment was releasing Appium 1.0. So the first official stable release of Appium. Also at the same time, we got a fair few contributors, and focused most of our attention on making sure that Appium was rock solid for even enterprise testing use. In 2015, we had the luxury of going back and fixing all the things that we hadn't gotten right the first time. So we did a complete code-based rewrite and we re-established a better architecture for our driver system, which will be very important, especially when we release Appium 2.0 hopefully later this year in 2020.

Moving a little bit quickly through the more recent years. In 2016 the big news for Appium was that Appium joined an open source foundation. Now at the time, this was called the JS Foundation. It's now called the OpenJS Foundation. The significance of this is that Appium's copyright up until that point was held by Sauce Labs, which might cause concern of course for companies that compete with Sauce Labs. If they wanted to contribute to Appium, they didn't necessarily have the legal guarantee that Appium was sort of a neutral thing that Sauce Labs couldn't use for their own purposes at some point. And because Sauce Labs didn't intend to do that and really wanted Appium to be a fully open, fully neutral project, Sauce Labs happily donated the project to the JS Foundation, which as I said, has merged with the Node Foundation into the OpenJS Foundation, and still is the manager and holder of the Appium copyright.

The last few years we really focused on stability and performance and making things solid, and at that point we felt that we were in a good spot to have our first ever user conference, which happened in 2018 in London, and which was a lot of fun. We followed it up last year in 2019 with our conference in India where the focus was really all the types of different platforms you could automate with Appium. And some of the newer technologies out there that had been integrated into Appium, for example, AI and machine learning, visual automation, and so on. The big news this year will hopefully be that Appium 2.0 we'll see a release, and that's the plan.

Appium's vision

Okay, so that's the history, the past and the present. What about the future? What is Appium's vision? What does Appium really see itself becoming in the future? Well, I think the first thing that we might say is that we would hope that Appium would, in some sense, become an official standard that is maintained not just by the Appium community, but by the world at large. By standards bodies, by people that care about everybody doing things in the same way to make life a little bit easier for everybody.

The most natural place for Appium to become standardized would be along the lines of the W3C WebDriver protocol. So this gets a little bit into the history of Selenium, which is a different project from Appium, but which Appium is based on, and at some point in its history Selenium decided that it really should become a sort of global standard rather than just being developed by one small team of maintainers. And so all the different browser vendors got together with the W3C, which is the World Wide Web Consortium, and they created a protocol called the WebDriver protocol. And it was based on the Selenium WebDriver API that already existed, but it was the result basically of an agreement between all these different companies that they would all support web browser automation in exactly the same way using the same API. So this was a great thing for the world. It meant that whenever a new version of Safari is released or Chrome or Firefox, that it ships with its own automation support, and all the different browsers, even though they're developed by completely different teams and competitive teams in fact, they are automated in the same way. So we hope one day that Appium will achieve the same status, at least with certain of its platforms. Mobile primarily being the goal here. And it would be great if it could be an official extension to the WebDriver protocol.

Next, we hope that Appium becomes less of a product and more of an ecosystem. So we hope that people take the vision of Appium for automating any kind of app and apply that to platforms that don't yet have support for Appium. So if somebody creates a new operating system, we hope that there's an Appium driver for it to allow people to write tests for that operating system using Appium. What that means is that we need for Appium to be really extensible and reusable so that anybody can take a look at our currently existing drivers and write their own new driver for some new platform. That's the only way that Appium will scale to all the different platforms that are out there. Because our small group of maintainers could never maintain support for automation of dozens of different platforms. But that is Appium's goal. So we want to make it really easy for everybody to be able to extend Appium for supporting different platforms. And so we want to make code libraries available for them to do that easily. And in fact, this is already the case.

Of course we do hope not just that the code libraries are available, but that people use them for creating automation support for all of these different platforms that we've been discussing. So the idea is that in the future there will be multiple drivers, each one of which is a WebDriver interface built on top of whatever automation technology is appropriate for a given platform, plus whatever other helper utilities are important for automating a particular platform. And then Appium really becomes the umbrella or the stable API on top of that sort of shifting sea of underlying automation technologies.

And of course in the future we hope that our community continues to be a really important part of the way Appium works and the way that people get help with Appium and something that just keeps Appium sort of on top in terms of its usability out there in the world. Because this is a pretty difficult space to work in. It can be hard to automate mobile devices. It can be hard to automate a lot of other platforms as well. So we all need help. We all need a community that is sharing what they've learned and the solutions that they've found to the problems that they've encountered.

The Appium project development process

So I'll just close out this section with an overview of the development process for Appium. This isn't super important for you unless you plan on becoming an Appium developer, which I hope you consider doing, but it is a little bit useful because throughout the course of using Appium you might find yourself in need of a certain bug fix or a certain improvement which has recently been committed and is not yet part of an Appium release. And so in that case, this diagram will hopefully help you. The basic way that the Appium project does development, very similar to many other open source projects. New code comes in as a pull request on GitHub. Of course, it's always a good idea to talk to maintainers first before any major changes to make sure that they're, at least in principle, going to be accepted. Once the new code is proposed, the maintainers review it manually, and then at the same time a variety of static analysis tools run like Lindt and unit tests and various other things. This might result in us requesting some changes to the code that's been proposed and this cycle can go on for quite a long time, in some cases, until eventually the pull request is either closed or merged. Usually it's merged. We put a lot of emphasis on helping people to emerge their code to Appium.

At that point we realized that the code has been merged to one particular repository, and Appium itself is made up of dozens of different repositories. So the repository where the code has been merged needs to be published, and because Appium is a Node.js application, all of its modules and sub-modules are published as NPM packages. NPM is package management tool for Node.js. Once that module has been published, it's usable unless you have an existing Appium installation. So if we have already made a release of Appium, re-downloading that release of Appium is not going to update any dependencies because each release of Appium has a set of completely frozen dependencies. But if you install the Appium beta, the Appium beta's dependencies are not frozen, which means that any time one of its dependencies is updated, for example with the code that was just merged in and published in the previous part of the slide, then reinstalling the Appium beta will bring in those new changes. So all that to say, if you ever find yourself in need of one of the fixes or improvements that's been merged and published NPM, but not yet available in an official version of Appium, then just install the Appium beta and you will almost certainly get the change that you're looking for.

So anyway, to round out this diagram, once we've decided that it's time for an official release, we update the change log of Appium, we make sure that all of the code has been merged in all the dependent repositories and that they've all had their NPM module published, and then we wrap up and publish a new version of Appium officially, at which point all the code within that version is generally available. So this is, I think, everything you need to know at a high level about Appium. So stay tuned and we will move into some of the details next.