XCTest: A Complete Guide

With iOS version 10, Apple introduced a new framework for automated user interface testing as part of XCTest (XCTest(UI)). This new framework allows you to write and run tests directly on your iPhone or iPad without needing a Mac or any coding expertise. XCTest is easy to use and intuitive, making it the perfect tool for anyone looking to take their app development skills to the next level. This guide will walk you through everything you need to know about XCTest, from installation to usage tips and tricks. So, if you're ready to start testing your apps like a pro, keep reading!

XCTest and Its History

XCTest is based on the WebDriver protocol and uses Apple's own XCTest testing framework. The WebDriver protocol is an industry standard for browser automation, and developers use XCTest to write unit tests for their apps. By combining these two technologies, Apple has created a powerful and easy-to-use testing framework perfect for anyone looking to automate their app testing.

XCTest was first announced at WWDC 2016, alongside the release of iOS 10. Before XCTest, automated testing on iOS was only possible using the UI Automation framework via instruments. UI Automation had some limitations, such as the need for a Mac and JavaScript coding expertise. With XCTest, Apple addressed these limitations by allowing tests to run directly on an iPhone or iPad without needing a Mac.

Check out: A Complete Guide to User Interface Testing

Benefits of using XCTest

There are many benefits to using XCTest over other testing frameworks.

• First and foremost, XCTest is very easy to use and requires only Swift experience, which is a standard development language. This makes it the perfect tool for anyone who wants to start writing automated tests but doesn't know where to begin.
• Secondly, XCTest is very intuitive and easy to understand. The syntax is similar to English, making it simple to read and write tests.
• Finally, XCTest is a fully-fledged testing framework that can be used for more than just UI testing. With XCTest, you can write unit tests, performance tests, and even test your app's compatibility with other apps and services.

Also Check: Why your app needs mobile compatibility testing?

Why was XCTest created?

XCTest was created to address these limitations by allowing tests to be run directly on an iPhone or iPad, without the need for a Mac. With XCTest, Apple wanted to develop a testing framework that was easy to use and didn't require any prior coding expertise. They also wanted to ensure that XCTest was based on industry-standard protocols, such as the WebDriver protocol. By doing this, they confirmed that XCTest would have a wide range of resources and support available.

XCTest vs. Other Testing Frameworks 

XCTest provides many improvements over different frameworks for testing iOS applications. Here's a small comparison between XCTest and its competitors:

XCTest vs. Appium

• XCTest is built into the iOS SDK, while Appium is a third-party library.
• XCTest only supports Apple products, while Appium can be used to test Android and iOS apps.
• XCTest can write in Swift/Objective-C, but Appium does not provide Swift client officially

XCTest vs. Calabash

• XCTest is built into the iOS SDK, while Calabash is a third-party library.
• XCTest only supports iOS apps, while Calabash can be used to test Android and iOS apps.
• XCTest doesn't require a Mac, while Calabash does.

XCTest vs. EarlGrey

• Both XCTest and EarlGrey are built into the iOS SDK.
• EarlGrey supports iOS 8 and above, while XCTest supports iOS 10 and above.

XCTest vs UI Automation

• XCTest is built into the iOS SDK, while UI Automation is a deprecated framework.
• XCTest doesn't require a Mac, while UI Automation does.

See: A complete guide to iOS application testing

How to get started with XCTest?

Now that you know what XCTest is and why it's so beneficial, you might be wondering how you can start using it. Luckily, getting started with XCTest is very easy! All you need is an iPhone or iPad running iOS 10 or later and Xcode 8 or later.

Once you have those two things, you're ready to start writing tests. To do so, simply open Xcode and create a new project. When prompted, choose the "iOS Unit Testing Bundle" template. This will create a basic test suite for you to work with.

If you're not familiar with how to write tests, don't worry! The XCTest documentation is very comprehensive and easy to follow. Apple also provides several sample tests you can use as a starting point.

Also See: Writing automated test scripts using Selenium

Once you've written your tests, you can run them directly on your iPhone or iPad by selecting the "Run" button in Xcode. Xcode will then deploy your app to the device and run the tests. You can view the results of the tests in the "Test Navigator" panel in Xcode.

Installing XCTest

XCTest is included in the Xcode 8.0+ release notes, so you will need to have Xcode 8.0 or later installed on your machine. You can check your Xcode version by opening the "About Xcode" dialog from the Xcode menu. Once you have verified that you have the correct version of Xcode installed, you are ready to install XCTest.

There are two ways to install XCTest:

Download the latest version of Xcode from the Apple Developer website. Once Xcode is installed, you will find XCTest under the "Developer Tools" section.

Read: Testing tvOS in Appium for Apple TV 1080 and 4K

Learning How to Use XCTest

Now that you have XCTest installed let's look at how it works. The first thing you can do is create a new project in Xcode. Once your project is created, open the "Tests" folder and create a new file called "MyFirstTest.swift".

XCTest uses the XCTest framework to write tests. This means that if you are familiar with writing unit tests in Xcode, then you will already be familiar with the syntax used by XCTest. If you have never written a unit test before, don't worry; it's straightforward to learn.

The basic structure of an XCTest is as follows:

import XCTest
class MyFirstTest: XCTestCase {
...
override func setUp() {
…
// Put setup code here. This method is called before the invocation of each test method in the class.
super.setUp()
}
override func tearDown() {
...// Put teardown code here. This method is called after the invocation of each test method in the class.
super.tearDown()
}
func testExample() {
…
// This is an example of a functional test case.
// Use XCTAssert and related functions to verify your tests produce the correct results
}
func testPerformanceExample() {
…
// This is an example of a performance test case.
self.measure {
…
// Put the code you want to measure the time of here.
}
}
}

As you can see, an XCTest consists of two parts: the setUp() method and the testExample() method.

• The setUp() method is called before each test is run. This is where you should put any code that needs to be executed before the test starts. For example, if you need to launch an app before running a test, you would do it in the setUp() method.
• The testExample() method is where the actual testing code goes. This is where you will write assertions to check that your app is behaving as expected.

Also read: How (And Why) To Create a Custom Expected Condition

Executing your first test

Now that you have written your first test, it's time to run it. In Xcode, select the "MyFirstTest.swift" file and click on the "Run" button in the toolbar. This will launch the simulator and execute your test.

You should see the following output in the console:

Test Suite 'MyFirstTest' started at 2017-02-12 23:04:21.191
Test Case '-[MyFirstTest testExample]' started.
t = 0.000s Start Test at 2017-02-12 23:04:21.192
t = 0.000s Set Up
2017-02-12 23:04:21.001 MyAppNameUITests-Runner.app/MyAppNameUITests-Runner (0x102829b00) launching
t = 0.057s Launch Application
(lldb)

This output shows that the test was successful. If the test had failed, you would see an "X" next to the test name in the console.

Read: Writing an Appium test in Kotlin

Assertions in XCTest

Assertions are used to verify that a specific condition is true. If the condition is not valid, the assertion will fail, and the test will stop executing. Assertions are written using the XCTest framework, and they have the following syntax:

XCTAssert(condition, message)

The condition is a Boolean value that determines whether the assertion passes or fails. The message is an optional String that is displayed if the assertion fails.

Check out: Fundamentals of Test Harness

For example, let's say we want to assert that 1 + 1 equals 2. We would write this assertion as follows:

XCTAssert(1 + 1 == 2, "1 + 1 does not equal 2")

If the assertion passes, nothing happens, and the test continues executing. If the assertion fails, the test will stop performing, and Xcode will print out the failure message. In this case, it would print out "1 + 1 does not equal 2".

Also check out: Validating Android Toast Messages

You can also use the XCTAssertEqual() method to check if two values are equal. This is often used to check if a value that we get from our app is the expected value. For example, let's say we want to check if the text "Hello World" is displayed on a label in our app. We would write this assertion as follows:

XCTAssertEqual(label.text, "Hello World", "Label text is not equal to 'Hello World'")

If the label's text is indeed "Hello World," then the assertion will pass, and the test will continue executing. If the label's text is anything other than "Hello World," the assertion will fail, and the test will stop running.

Check: Differentiating Factors That Make HeadSpin Unique

Matchers in XCTest

In addition to assertions, XCTest has several built-in methods that can be used for testing. These methods are called "matchers," which are used to find elements in the UI. For example, let's say we want to find a button with the text "Hello World" on it. We would use the following code:

let button = app.buttons["Hello World"]

This code will find the first button in the app with the text "Hello World" on it. The return value is XCUIElement. Matchers can also be used to check if an element exists or does not exist. For example, let's say we want to check if a button with the text "Hello World" exists. We would use the following code:

XCTAssertTrue(app.buttons["Hello World"].exists, "Button with text 'Hello World' does not exist")

If the button exists, the assertion will pass, and the test will continue executing. If the button does not exist, the assertion will fail, and the test will stop running.

Also Check: Automating Physical Buttons on iOS Devices

Debugging in XCTest

In addition to assertions and matchers, XCTest has several built-in methods that can be used for debugging. For example, let's say we want to print out all of the elements in the UI. We would use the following code:

print(app.debugDescription)

This code will print out a debug description of the app, including all of the UI elements. This can be helpful for troubleshooting purposes.

Now that you know how to write and run a basic XCTest, let's look at some of the more advanced features that XCTest offers.

Recommended post: How Artificial Intelligence is Revolutionizing UI Testing

Advanced Features of XCTest

XCTest has several advanced features that can make your tests more robust and reliable. In this section, we will look at some of these features and how they can be used.

Waiting For Elements to Appear

One common issue when writing automated tests is that elements may not appear on the screen immediately after launching the app. This can happen for many reasons, such as slow network conditions or lengthy database queries. In these cases, it is crucial to tell XCTest to wait for the element to appear before interacting with it. The return value is Boolean. This can be done using the XCUIElement.waitForExistence(timeout:) method.

The following example shows how this method can be used to wait for a button to appear:

let button = app.buttons["Login"]
let existence = button.waitForExistence(timeout: 5)
XCTAssertTrue(existence)
// The button will now be guaranteed to exist so we can interact with it.
button.tap()

In this example, we are waiting for a button labeled "Login" to appear. If the button does not appear within 5 seconds, an error will be thrown, and the test will fail.

See: A Comprehensive Guide to Mobile Application Security Testing

Accessing Elements That Are Not Visible on The Screen

Another common issue when writing tests is that some elements may not be visible on the screen when the test is run. This is because for several reasons, such as scrolling or opening a menu. In these cases, it is essential to tell XCTest to look for the element in the entire view hierarchy, not just the visible part of the screen. This can be done using the XCUIElement.descendants(matching:) method.

For example, suppose we have a button inside a scroll view. The button is not visible on the screen when the test starts, but it can be accessed by scrolling down. We can use the following code to access the button:

let scrollView = app.scrollViews["Content"]
let button = scrollView.descendants(matching: .button).element(boundBy: 0)
button.tap()

In this example, we use the descendants(matching:) method to find all descendant elements of type .button. We then use the element(boundBy:) method to get the first button in the list. Finally, we tap on the button to trigger the action.

Also read: Automating Physical Buttons on iOS Devices

Accessing Elements That Are Disabled

Another common issue when writing tests is that some elements may be disabled when the test starts. This can happen for several reasons, such as the user not being logged in or the app being in a specific state. In these cases, checking if the element is enabled before trying to interact with it is important. This can be done using the XCUIElement.isEnabled property.

The following example shows how this property can be used to check if a button is enabled:

let button = app.buttons["Login"]
if button.isEnabled {
button.tap()
} else {
// The button is disabled, so we cannot interact with it.
}

In this example, we check if a button with the label "Login" is enabled. If it is, we tap on the button to trigger the action. If it is not, we do not try to interact with it.

Check out: Accessing Android Logcat Logs with Appium

Accessing Elements That Do Not Exist Yet

Another common issue when writing tests is that some elements may not exist when the test starts. This can happen for several reasons, such as the element only being displayed after a specific action is taken. In these cases, checking if the element exists is essential before trying to interact with it. This can be done using the XCUIElement.exists property.

The following example shows how this property can be used to check if an element exists:

let element = app.buttons["Login"]
if element.exists {
element.tap()
} else {
// The element does not exist, so we cannot interact with it.
}

In this example, we check if a button with the label "Login" exists. If it does, we tap on the button to trigger the action. If it does not, we do not try to interact with it.

Read: Why real device is cloud critical in app testing

Waiting for an Element to Appear

Another common issue when writing tests is that some elements may not appear immediately when the test starts. This can happen for several reasons, such as the element only being displayed after a specific action is taken. In these cases, waiting for the element to appear before interacting with it is essential. This can be done using the XCUIElement.waitForExistence(timeout:) method.

The following example shows how this method can be used to wait for an element to appear:

let element = app.buttons["Login"]
let existence = element.waitForExistence(timeout: 30)
XCTAssertTrue(existence)
// The element will now be guaranteed to exist.
element.tap()

In this example, we are waiting for a button labeled "Login" to appear. We use a timeout of 30 seconds, which means the method will return after 30 seconds even if the element has not yet occurred. Once the method returns, we know that the element exists if the value was true, and we can tap on it to trigger the action.

Check: How to Make Digital Native App Testing Easier?

XCTest Best Practices

There are a few best practices to keep in mind when using XCTest:

• Make sure that your app is in a testable state before running tests.
• Use the XCUIApplication class to launch your app.
• Use the XCUIDevice class to get information about the device that is being tested.
• Use the XCUIElement class to interact with elements on the screen.
• Check for the existence of elements before interacting with them.
• Wait for elements to appear before interacting with them.
• Use the isEnabled, isSelected, and isHittable properties to check the state of elements.
• Use the XCUIElement.tap() method to trigger actions on elements.
• Use the XCUIElement.typeText() method to enter text into fields.
• Use the XCUIElement.swipeLeft(), .swipeRight(), .swipeUp(), and .swipeDown() methods to perform gestures.
• Use the XCTestCase class to write your tests.
• Use the setUp() and tearDown() methods to prepare for and clean up after each test.
• Use the XCTContext to structure your tests.
• Use the XCTAssert and XCTFail methods to verify the results of your tests.
• Use the XCTestRecorder tool to generate tests from user actions.
• Use the -logger parameter to generate logs during test execution.
• Use the -resultBundlePath parameter to generate a report after test execution.
• Use the -onlyTesting parameter to specify which tests should be run.
• Use the -skipTesting parameter to specify which tests should not be run.
• Use the --help or -h parameters to get help with command-line options.

How does XCTest integrate with HeadSpin Platform?

HeadSpin provides a comprehensive set of services to help you deliver amazing mobile experiences. One of these services is our mobile test automation platform, which allows you to write and run automated tests for your apps directly on your iPhone or iPad. 

XCTest is a new addition to the HeadSpin platform that makes it easy to get started with mobile test automation. With XCTest, you can write tests in Swift or Objective-C and run them on real devices without having to jailbreak them. You can also use XCTest to create and manage a testing farm of devices, so you can run your tests in parallel on multiple devices at the same time. 

In addition, XCTest integrates with popular CI/CD tools like Jenkins and Travis, so you can easily set up a continuous testing pipeline.

To execute XCTest native tests in HeadSpin, follow the step-by-step process below:

• Open the HeadSpin dashboard and set up your project.
• Add the HSConnect to your project and device.
• Write your tests in Swift or Objective-C.
• Run your tests on real devices without jailbreaking them.
• Use XCTest to create and manage a testing farm of devices in the HeadSpin dashboard.
• Integrate with popular CI/CD tools like Jenkins and Travis.
• Get started with mobile test automation today!

The Bottom Line

XCTest is a powerful tool that can be used to automate tests for your iOS apps. It is elementary to use and very intuitive, making it the perfect tool for those new to coding. In addition, XCTest can handle various common issues that can occur when writing tests, such as elements that are disabled or do not exist yet. With XCTest, you can ensure that your app is functioning correctly and that your users are having the best experience possible.

XCTest Framework FAQs

Q: What types of tests can I write with XCTest?

A: You can use XCTest to write unit, functional, and performance tests. Unit tests focus on specific classes or methods, while functional tests test the end-to-end functionality of your app. Performance tests measure the responsiveness and stability of your app under different conditions.

Q: I'm having trouble getting XCTest to work. What can I do?

A: If you're having trouble getting started with XCTest, we recommend checking out the official documentation or watching one of the many helpful online tutorials. You can also ask for help on the Apple Developer Forums or Stack Overflow.

Q: Where can I learn more about using XCTest?

A: You can find more information about using XCTest in the official documentation. You can also watch one of the many helpful tutorials available online. Finally, you can ask for help on the Apple Developer Forums or Stack Overflow.

Q: What are some common problems that can occur when using XCTest?

A: Some common problems when using XCTest include elements that are disabled or do not exist yet. Additionally, tests may fail if the app under test crashes or if it is not responding to user input.

Q: How do I handle these common problems?

A: To handle elements that are disabled or do not exist, you can use the isEnabled, isSelected, and isHittable properties to check the state of components. You can also use the waitForExistence() method to wait for an element to appear before interacting with it. If the app under test crashes or is unresponsive, you can use the XCTestRecorder tool to generate a test case that can be used to reproduce the issue.

Q: What are some tips for writing good tests?

A: Some tips for writing good tests include using descriptive names for your test methods, keeping your tests small and focused, and using the given-when-then keywords to structure your tests. Additionally, you should clean up after each test so that your tests are isolated from each other. Finally, you can use the XCTAssert and XCTFail methods to verify the results of your tests.