Scrolling with Appium
Scrolling is necessary for automation whether it is mobile or web. Unlike Selenium, scrolling with Appium is a bit challenging as Appium is depending on other testing frameworks like XCUITest, UIAutomator2, etc and hence the scrolling supported by these frameworks are entirely different. Our main consideration in this article would be scrolling to an element, but we will briefly go through some other gestures of scroll and swipe as well. Also, remember that the direction of scrolling and swiping is exactly the opposite. i.e., when you swipe up, the screen will be scrolling down and take care of this while specifying scroll/swipe directions.
UiAutomator2
UIAutomator2 is one of the unit test frameworks for Android and is supported by Appium as well. One of the challenges with Android is that only the elements that are currently visible on the screen are available to be accessed. The scrolling is supported via mobile commands in appium. i.e., the same should be triggered as driver.execute(“<command>”, params);
- Using AndroidUIAutomator locator
While finding the element, we can specify to scroll the view to make the element visible. This is obtained by using the UiScrollable method within the MobileBy.AndroidUIAutomator locator.
AndroidElement element = (AndroidElement) appiumDriver.findElement(MobileBy.AndroidUIAutomator( "new UiScrollable(new UiSelector().scrollable(true).index(0)).scrollIntoView(new UiSelector().text(\"Splitting Touches across Views\"))"));The first UiSelector represents the parent element which is to be scrolled and the second inside scrollIntoView represents the child element to which the view should be scrolled. The syntax is new UiScrollable(<scrollable parent element>).scrollIntoView(<child element>), both the parent and child locators should be found with UiSelector.
2. mobile: scroll
strategy and selector are the parameters to indicate the element to which the view should be scrolled and are mandatory. Scrolling happens till this element is visible. The locator strategies accessibility id, class name, and -android uiautomator only are supported.
elementId(element till 1.21.0 appium version) can be given to specify the element which should be scrolled. This is the element being scrolled and not the element to which the scroll is expected. elementId is not the element and not even the locator id but the random hexadecimal code representing the element in that session returned by findElement API. This can be obtained with the method getId() on the element. This element should be scrollable so to make the scrolling. If this argument is not given then the first available scrollable element is taken automatically.
maxSwipes can be specified to limit the number of swipes on the screen.
HashMap<String, Object> scrollObject = new HashMap();
AndroidElement element = (AndroidElement) appiumDriver.findElement(MobileBy.AndroidUIAutomator("resourceId(\"io.appium.android.apis:id/list2\")"));
scrollObject.put("elementId", element.getId());
scrollObject.put("strategy", "-android uiautomator");
scrollObject.put("selector", "text(\"Beaufort\")");
appiumDriver.executeScript("mobile: scroll", scrollObject);3. mobile: scrollGesture
This won’t support scrolling till an element. This scrolls based on screen size. direction and percent are the mandatory parameters.
The direction can be up, down, left, and right. Percent can be expressed as float value from 0.1. 1 means 100 percent and will scroll the entire screen once. Values greater than 1 can also be given so as to scroll that much number of times. For example, 3.5 will scroll three full screens and then a half screen.
elementId can be given to specify the element which should be scrolled as we explained in mobile:scroll. If this argument is not given then the first available scrollable element is taken automatically. As per official documentation, the scrollable parent element should have been found using UI Automator locator, but others like XPath and id are also supported actually.
The scrolling area can also be specified using left, top, height, and width parameters instead of specifying the element. Preference is for elementId.
speed is also a non-mandatory parameter that is expressed in pixels per second which can be used to control the speed of scrolling.
scrollObject.put("elementId", element.getId());
scrollObject.put("direction", "down");
scrollObject.put("percent", 1);
appiumDriver.executeScript("mobile: scrollGesture", scrollObject);4. mobile: swipeGesture
Similar to scrollGesture but the percent value cannot exceed 1.0.
Espresso
In espresso, even though the full details of the elements outside the visible view are not available, some of the details are available as adapters to the parent element which is visible on the current view.
- Using androidDataMatcher locator
The details available in the adapters section can be used to scroll to an element that is outside the visible view. For the same, we have to locate the parent element to which the required element belongs and confirm that the child element is available in its adapters list.
appiumDriver.findElement(MobileBy.androidDataMatcher(new Json().toJson(ImmutableMap.of(
"name", "hasEntry",
"args", ImmutableList.of("title", "Popup Menu")
))));The above script will scroll to the element with the title Popup Menu. Here, we haven’t specified any parent. We can specify the parent by triggering the findElement on the parent as follows.
AndroidElement parent = (AndroidElement) appiumDriver.findElement(By.xpath("//*[@resource-id='app:id/list2']"));
parent.findElement(MobileBy.androidDataMatcher(new Json().toJson(ImmutableMap.of(
"name", "is",
"args", "Banon"
))));2. mobile: swipe
This will just swipe the given element in the specified direction. element is a mandatory parameter. Other parameters are startCoordinates, endCoordinates, swiper, and precisionDescriber. Start and End coordinates can have the valuesTOP_LEFT, TOP_CENTER, TOP_RIGHT, CENTER_LEFT, CENTER, CENTER_RIGHT, BOTTOM_LEFT, BOTTOM_CENTER, BOTTOM_RIGHT, andVISIBLE_CENTER.
swiper can be used instead of direction and values it can take are SLOW or FAST. This will swipe between the given start and end coordinates with the chosen swipe speed. precisionDescriber can take values PINPOINT (1px), FINGER (average width of the index finger is 16 – 20 mm), THUMB (average width of an adult thumb is 25 mm or 1 inch, the default value). The resulting view after the swipe is not predictable and hence I don’t recommend this method.
scrollObject.put("element", element.getId());
scrollObject.put("direction", "up");
appiumDriver.executeScript("mobile: swipe", scrollObject);XCUITest
XCUITest renders the entire elements in the view irrespective of whether it is visible or not. The biggest advantage with this is that we don't even need to scroll to the element if we don’t want to directly interact with this element.
- Using findElement with any locator
The element need not to be visible so as to get details of the element like getText(), getAttribute(), isSelected(), getSize() etc. Also, if we want to click on an element that is outside the visibility, just find the element and trigger the click, XCUITest will automatically scroll to the element and will perform the click.
appiumDriver.findElement(MobileBy.iOSNsPredicateString("label == \"TV Provider\"")).click();2. mobile: scroll
elementId(element prior to 1.22 appium version) parameter can be used to specify the element that needs to be scrolled. If not specified, the parent view XCUIElementTypeApplication is used for scrolling.
Either name, direction, predicateString, toVisible parameter could be used to specify the target location. The hierarchy goes in the same order and specifying all of them or more than one of them will be of no use as the first one according to the above priority will be taken and the rest will be ignored. name or predicateString can be used to indicate the element to which the scroll is to be made and will be scrolled till this element is visible.
direction alone can be mentioned to scroll in that direction, but the scroll length would be different across different devices. toVisible will scroll till the first element in the parent is visible.
Note: mobile: scroll seems to be not working as it keeps on scrolling even after passing the required child element and then ends up in a timeout error on my trials on iPhone’s Settings app.
HashMap<String, Object> scrollObject = new HashMap();
scrollObject.put("predicateString", "name == '70'");
appiumDriver.executeScript("mobile: scroll", scrollObject);3. mobile: swipe
Similar to scroll, but direction alone is supported and is mandatory.
elementId can be used to specify the element that needs to be scrolled. If not specified, the parent application view is used similar to scroll.
velocity which can take values greater than 0 and can be used to control the speed of the swipe.
4. mobile: dragFromToForDuration
This will drag the screen based on the coordinates fromX, fromY, toX, and toY given as parameters and duration that can be used to adjust the dragging speed.
Map<String, Object> params = new HashMap<>();
params.put("duration", 1.0);
params.put("fromX", 0);
params.put("fromY", 600);
params.put("toX", 0);
params.put("toY", 300);
appiumDriver.executeScript("mobile: dragFromToForDuration", params);TouchAction for scrolling
Irrespective of the test framework, TouchAction is a class supported by Appium that helps to trigger actions like press and moveTo within the screen. Even though it has the advantage that the same code and logic can be used across all platforms, if any of the above methods for scrolling is a perfect fit for your context, go with that as the same would be a better solution in terms of performance. But if the above methods are not working for you and you want to scroll to an element till it is visible, touch action could be used.
The logic I would be using is to scroll the screen by 50% starting from 3/4th of the screen till 1/4th of the screen and check whether the element is visible. This would be repeated until the element is visible.
TouchAction ta = new TouchAction(appiumDriver);
MobileElement parent = (MobileElement) appiumDriver.findElement(<parentBy>);var parentLocation = parent.getLocation();
var parentSize = parent.getSize();
var windowSize = appiumDriver.manage().window().getSize();int endHeight = windowSize.height < parentLocation.y+parentSize.height
? windowSize.height : parentLocation.y+parentSize.height;
int startHeight = parentLocation.y;
int heightArea = endHeight+startHeight;while(<check child element visibility(continue if not visible)>)
ta.press(new PointOption().withCoordinates(parentLocation.x, (int) (0.75*heightArea)))
.waitAction(new WaitOptions().withDuration(Duration.ofMillis(1500)))
.moveTo(new PointOption().withCoordinates(parentLocation.x, (int) (0.25*heightArea)))
.release().perform();
}
The while condition needs to be chosen accordingly as, for XCUITest, the child element would be available even though not visible and hence the element could be found once and isDisplayed() could be used to check the visibility whereas for Android, the findElement needs to be triggered each time and if not found, the loop should continue. Also, an additional check can be done based on a max time, so as to avoid the possibility of an infinite loop. The waitOption value can be changed accordingly to change the speed of scroll and hence the area of scrolling.
