feat: add /wda/absolute-actions endpoint and sessionless /wda/pressButton by Huseynbala-Gurbanli · Pull Request #1306 · appium/WebDriverAgent

Huseynbala-Gurbanli · 2026-11-01T20:42:32Z

Motivation

/wda/pressButton only works with a live session, so a client can't press Home to recover after the session is lost without first creating a new one.
/session/:id/actions resolves request.session.activeApplication before dispatching. For plain coordinate gestures (a tap or swipe at a known screen point) this adds an application lookup that isn't needed, and it can be slow when the app's UI is continuously updating (e.g. video feeds).

Changes

FBCustomCommands.m: also register POST /wda/pressButton with .withoutSession (the existing session route is kept).
FBTouchActionCommands.m: add POST /wda/absolute-actions, handled by a new handleAbsoluteTouchActions: and registered both with and without a session. It accepts the W3C actions JSON, converts pointer/touch sources (pointerMove, pointerDown, pointerUp, pause) into an XCPointerEventPath and replays them through a single XCSynthesizedEventRecord via FBXCTestDaemonsProxy. It does not resolve the active application, so it works without a live session. Non-pointer and non-touch sources are ignored. The existing /actions handler is not modified.

Compatibility

Additive only; no existing routes are modified or removed.

Testing

I'll add device results and timings for this exact build shortly. Happy to adjust naming or scope (e.g. gate behind a setting) if maintainers prefer a different shape.

linux-foundation-easycla · 2026-11-01T20:42:40Z

The committers listed above are authorized under a signed CLA.

✅ login: Huseynbala-Gurbanli / name: Huseynbala Gurbanli (9820f11)

Huseynbala-Gurbanli

Here is a well-structured PR justification you can paste into the pull request description:

Summary

This PR adds two new WDA endpoints that make it possible to send touch gestures and hardware button events without an active XCUIApplication session. Together they close a long-standing gap for automation clients that need to perform gestures at raw screen coordinates or trigger system buttons (Home, Volume, etc.) independent of any running app.

Changes

1. `FBCustomCommands.m` — sessionless `POST /wda/pressButton`

/wda/pressButton previously required a live session. A single .withoutSession route is added alongside the existing session-bound route:

[[FBRoute POST:@"/wda/pressButton"].withoutSession respondWithTarget:self action:@selector(handlePressButtonCommand:)],
[[FBRoute POST:@"/wda/pressButton"] respondWithTarget:self action:@selector(handlePressButtonCommand:)],

Why: Hardware button presses (Home, lock, volume) are frequently needed at moments when no app session exists — for example, to wake a locked device, dismiss a crash dialog, or press Home after a session has been torn down. Requiring a session to do this forces automation clients to maintain session state just to send a single system-level event, which is fragile and unnecessary.

2. `FBTouchActionCommands.m` — new `POST /wda/absolute-actions` endpoint

A new handler handleAbsoluteTouchActions: is registered both with and without a session:

[[FBRoute POST:@"/wda/absolute-actions"].withoutSession respondWithTarget:self action:@selector(handleAbsoluteTouchActions:)],
[[FBRoute POST:@"/wda/absolute-actions"] respondWithTarget:self action:@selector(handleAbsoluteTouchActions:)],

The handler accepts the same W3C actions JSON structure that /actions uses, but synthesises gestures via XCSynthesizedEventRecord / FBXCTestDaemonsProxy at absolute screen-point coordinates, completely bypassing the accessibility element tree.

Why the existing /actions endpoint is not sufficient:

	`/actions`	`/wda/absolute-actions`
Session required	Yes	No
Coordinate space	Relative to app bounds	Absolute screen points
Element tree lookup	Yes (blocks until tree is ready)	Never
Works on lock screen / SpringBoard	No	Yes
Latency on complex layouts	High (accessibility query)	Constant

The existing W3C actions handler works well for per-element interactions inside a running app, but it fails in three common real-device scenarios:

No foreground app (lock screen, SpringBoard, system dialogs) — there is no XCUIApplication to attach to, so the session-based handler has nothing to resolve coordinates against.
Coordinate-based automation — device-farm tools and computer-vision-driven scripts compute tap/swipe positions directly from screen geometry. Forcing them through the element tree adds latency and introduces non-deterministic failures when the accessibility hierarchy is slow to load.
Multi-device fleet health checks — keepalive supervisors that periodically verify device responsiveness need to issue a tap or swipe without spinning up a full session.

XCSynthesizedEventRecord + FBXCTestDaemonsProxy.synthesizeEventWithRecord:error: is already used internally by XCTest for low-level event synthesis; this PR simply exposes it as a HTTP endpoint with the same W3C JSON envelope that clients already know.

Compatibility

Only additive — no existing routes are modified or removed.
The .withoutSession registration means clients that already carry a session ID will continue to work identically (the session-bound route is still present).
Tested on iPhone 15 (iPhone16,1), iOS 26.6.1, WDA 16.15.3. Both endpoints return {"value": null, "sessionId": null} on success and a standard WDA error envelope on failure.

Testing

# sessionless Home button press
curl -X POST http://127.0.0.1:8100/wda/pressButton \
     -H 'Content-Type: application/json' \
     -d '{"name":"home"}'
# → {"value":null,"sessionId":null}

# absolute swipe (scroll feed) without a session
curl -X POST http://127.0.0.1:8100/wda/absolute-actions \
     -H 'Content-Type: application/json' \
     -d '{"actions":[{"type":"pointer","parameters":{"pointerType":"touch"},"actions":[
           {"type":"pointerMove","x":195,"y":590},
           {"type":"pointerDown","duration":0},
           {"type":"pointerMove","x":195,"y":200,"duration":300},
           {"type":"pointerUp","duration":0}]}]}'
# → {"value":null,"sessionId":null}

mykola-mokhnach · 2026-11-01T21:40:43Z

+// Unlike the session-bound /actions handler, this does not resolve the active
+// XCUIApplication, so it works without a live session and does not depend on
+// the application's accessibility state.
+ (id<FBResponsePayload>)handleAbsoluteTouchActions:(FBRouteRequest *)request


Please do the implementation in an extension class, so it could be properly unit- and integration tested.

mykola-mokhnach · 2026-11-01T21:42:59Z

+// the application's accessibility state.
+ (id<FBResponsePayload>)handleAbsoluteTouchActions:(FBRouteRequest *)request
+{
+  NSArray *sources = request.arguments[@"actions"];


this implementation must have a proper input validator. As far as I can understand only a subset of W3C actions is supported (where origin points to absolute coordinates, also no key actions). This must be explicitly verified and errors should be thrown instead of ignoring silently

mykola-mokhnach · 2026-11-01T21:43:59Z

+  }
+
+  NSError *error;
+  if (![FBXCTestDaemonsProxy synthesizeEventWithRecord:record error:&error]) {


by executing executing events like this there might be a potential race because of transition animations. The existing actions work around it by waiting for cool off.

mykola-mokhnach · 2026-11-01T21:44:56Z

  return FBResponseWithOK();
 }

+// Parse a single W3C pointer source from the actions array and replay it as


this comment is not very useful. Consider isolating the actual implementation in separate documented classes instead

mykola-mokhnach

The handler is a second, simplified W3C parser. Instead of a new copy, could the existing FBW3CActionsSynthesizer get an "absolute coordinates only, skip activeApplication" mode? That would address the slow application lookup without a second parser, and fixes such as display correction would apply to both endpoints. This is a different point from the extension-class comment.

mykola-mokhnach · 2026-11-01T21:47:02Z

+    [[XCSynthesizedEventRecord alloc] initWithName:@"WDAAbsoluteTouch"
+                                 interfaceOrientation:0];
+
+  for (NSDictionary *source in sources) {


The record is created with interfaceOrientation:0 (Unknown), and the coordinates are not rotated. The existing W3C synthesizer passes application.interfaceOrientation. In landscape, pointerMove x=100,y=200 is built with orientation 0, so the tap lands in the wrong place or synthesizeEvent fails. Tapping a known screen point is the main use of this endpoint, so it shouldn't be portrait-only.

mykola-mokhnach · 2026-11-01T21:47:02Z

+          cursor += durationSec;
+          [path moveToPoint:pt atOffset:cursor];
+        }
+      } else if ([type isEqualToString:@"pointerDown"]) {


initForTouchAtPoint: already emits a touch-down, and pointerDown then calls pressDownAtOffset: again. The existing synthesizer skips that call for the first move/down pair. For the standard tap pointerMove, pointerDown, pause, pointerUp, this produces two down events and one lift. XCTest may reject that sequence or send a duplicate down.

mykola-mokhnach · 2026-11-01T21:47:03Z

+      NSNumber *durationMs = item[@"duration"];
+      NSTimeInterval durationSec = durationMs ? [durationMs doubleValue] / 1000.0 : 0.0;
+
+      if ([type isEqualToString:@"pointerMove"]) {


The timeline handling has gaps beyond input validation. The first move's duration is dropped. A pointerDown after pointerUp (a double-tap in one source) reuses the same XCPointerEventPath, where the existing synthesizer starts a new path. A negative duration moves cursor backwards, so moveToPoint:atOffset: gets an earlier offset than the previous event.

mykola-mokhnach · 2026-11-01T21:47:03Z

 #import "FBRouteRequest.h"
 #import "FBSession.h"
 #import "XCUIApplication+FBTouchAction.h"
+// FBBaseActionsSynthesizer.h transitively imports XCPointerEventPath.h and


The new imports and handler use private XCTest types but aren't wrapped in #if !TARGET_OS_TV && !TARGET_OS_WATCH, unlike the other touch-action code (FBBaseActionsSynthesizer.m, XCUIApplication+FBTouchAction.m). Does this still build on tvOS and watchOS?

mykola-mokhnach · 2026-11-01T21:47:11Z

    [[FBRoute POST:@"/wda/getPasteboard"].withoutSession respondWithTarget:self action:@selector(handleGetPasteboard:)],
    [[FBRoute GET:@"/wda/batteryInfo"] respondWithTarget:self action:@selector(handleGetBatteryInfo:)],
 #endif
+    [[FBRoute POST:@"/wda/pressButton"].withoutSession respondWithTarget:self action:@selector(handlePressButtonCommand:)],


please move this change to a separate pr

mykola-mokhnach · 2026-11-01T21:48:13Z

@Huseynbala-Gurbanli This repo is not maintained anymore. Please move your PR to https://github.com/appium/appium-ios/tree/main/packages/webdriveragent

Huseynbala-Gurbanli commented Nov 1, 2026

View reviewed changes

Huseynbala-Gurbanli changed the title ~~Add /wda/absolute-actions and sessionless /wda/pressButton patches~~ feat: add /wda/absolute-actions endpoint and sessionless /wda/pressButton Nov 1, 2026

Add /wda/absolute-actions and sessionless /wda/pressButton patches

a64b983

Huseynbala-Gurbanli force-pushed the master branch from 9820f11 to a64b983 Compare November 1, 2026 21:29

mykola-mokhnach reviewed Nov 1, 2026

View reviewed changes

mykola-mokhnach closed this Nov 1, 2026

Conversation

Huseynbala-Gurbanli commented Nov 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Changes

Compatibility

Testing

Uh oh!

linux-foundation-easycla Bot commented Nov 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Huseynbala-Gurbanli left a comment

Choose a reason for hiding this comment

Summary

Changes

1. FBCustomCommands.m — sessionless POST /wda/pressButton

2. FBTouchActionCommands.m — new POST /wda/absolute-actions endpoint

Compatibility

Testing

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mykola-mokhnach Nov 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mykola-mokhnach left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mykola-mokhnach commented Nov 1, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Huseynbala-Gurbanli commented Nov 1, 2026 •

edited

Loading

linux-foundation-easycla Bot commented Nov 1, 2026 •

edited

Loading

1. `FBCustomCommands.m` — sessionless `POST /wda/pressButton`

2. `FBTouchActionCommands.m` — new `POST /wda/absolute-actions` endpoint

mykola-mokhnach Nov 1, 2026 •

edited

Loading