Xcode UI Testing vs. The UI Automation Instrument

Image by Yung-Luen Lan

Similarities

• Both systems are built upon the Accessibility layer in iOS and use elements that mimic, but do not directly represent, the same UIElement objects in your application.
• They both run in separate processes so they will not affect the performance of your application, but also are unable to programmatically set up or modify the state of your application.
• Both have an ability to record simulator interaction and convert those actions into code.
• Both navigate the view hierarchy by index or property comparison to locate elements.

Differences

• UIT uses a standard iOS development language (Swift) and the XCTest framework for evaluating assertions. However, UIA uses Javascript and its own UI Automation framework to provide testing functionality.
• When you access XCUIElements in UIT they are re-evaluated every access (even if they are a constant ‘let’) against the current application state. However, in UIA your element variables are a copy of the state at the moment they are assigned. This means developers have to grab a new copy for any future comparisons.
• UIA requires developers to handle responding to UIAlerts via special alert handlers, while UIT allows test writers to operate on those alert elements via the same mechanisms as any other object.
• UIT takes a screen shot on every simulated user action for free, while UIA only captures a screenshot automatically on failure. However, only UIA provides functionality to capture images on request which could be more helpful for debugging more dynamic pages.
• UIA requires the test writer to write in animation buffers that wait for elements to come on screen, while UIT handles all this complexity behind the scenes.
• UIT doesn’t have support for complex gestures such as pinch or tapping / drag at an X-Y coordinate. However, it does have support for the common tap, double tap, etc. gestures. UIA however, supports or has a hook to make whatever gesture you want.

What does UI Testing do better?

• UIT does a better job dynamically creating the code generated from the record feature compared to UIA.
• Writing in a native language is much nicer than JavaScript.
• Test writing in UIT feels a lot easier to get up and running without having to worry about animation delays, special alert handler or other particularities.
• UIT runs faster than UIA.

What does UIA do better?

• UIA may be more verbose and less nice to work with, but it has more power and flexibility.
• It has support for complex gestures and the more deterministic access to accessibility element copies.
• If you have a game you want to test, you have a chance with UIA testing, but likely not with UIT.

Which one should I use?

This is part 2 of a 3-part series on Xcode UI Testing…

Read Part 1: How to Use iOS UI Testing in Xcode

Read Part 3: Advanced UI Testing: Strategies, Hidden Gems and Limitations