Modern Execution Context Reference

execute() receives a context object as its second argument. Which fields are present depends on the platforms you declare in applicationType and the permissions you request. Requesting nothing gives you logging, a scratch directory, runtime variables, and an abort signal.

Context Fields

Honor ctx.signal in loops and long waits so cancelled runs stop promptly.

Browser Adapter

Locator handles support click(), fill(text), textContent(), getAttribute(name), isVisible(), count(), boundingBox(), and string-based evaluate().

Mobile Adapter

locate() accepts a declared ElementRef or an ad-hoc { strategy, value } literal. Supported strategies are accessibilityId, id, xpath, className, androidUiAutomator for Android, and iosPredicate or iosClassChain for iOS.

What Is Exposed

ctx.ui.browser and ctx.ui.browser.page are the same object at two levels of abstraction. The adapter above covers common cases; when it does not reach far enough, every session also exposes the real automation object and the real assertion library.

No permission declaration is required for any of them. These are the genuine libraries rather than a curated subset, so every matcher, .not, .soft, and the generic value matchers work as documented.

Reach for these for frames, keyboard and mouse, dialogs, downloads, and accessibility snapshots on web, or raw Appium commands on mobile.

Three Rules That Differ from a Normal Test

Your addon runs in a worker while the browser lives in the host process. That has three consequences.

1. Await everything, including calls that are synchronous in Playwright such as page.url(). Composing without awaiting is still free, so page.getByRole("form").locator("input") reaches the host only once, when you finally await it.
2. Await every assertion. Generic matchers return a promise here. An un-awaited failing assertion becomes an unhandled rejection, and your step passes when it should not.
3. No closures. Function arguments are rejected. Anything that evaluates in the browser or on the device takes a string instead, including evaluate("el => el.textContent"), waitForFunction, $$eval, and WebdriverIO's execute.

Structural Validation with Aria Snapshots

Asserting the shape of a form or menu in one call beats a pile of per-element checks. Matching is a subset check, so unrelated markup can change freely, and it auto-retries like any Playwright assertion.

const page = ctx.ui.browser.page;
const expect = ctx.ui.browser.expect;

await expect(page.getByRole("form", { name: "Filters" })).toMatchAriaSnapshot(`
  - heading "Filters"
  - textbox "Search"
  - button "Apply"
`);

const snapshot = await page.getByRole("form", { name: "Filters" }).ariaSnapshot();

Waiting on Mobile

WebdriverIO's waitUntil is unavailable, because its condition callback runs in Node and no string form can replace it. Use ctx.ui.mobile.waitUntil, which re-evaluates a driver expression on the device each interval.

await ctx.ui.mobile.waitUntil(driver.$("~ok").isDisplayed(), { timeout: 10_000 });

Pass the expression without awaiting it. Awaiting first would check it once, and is refused with an explanation. Options are WebdriverIO's timeout, interval, and timeoutMsg.

What Is Refused

Your addon is one step inside someone else's session, so anything that outlives the step or hands over credentials is refused with a clear error.

One-shot waitForEvent(...) is available and covers dialogs, downloads, and popups. capabilities stays available, so you can branch on platformVersion.

page.context() and page.request are available, so cookies, storageState(), and API calls all work. Only the dangerous members inside them are refused.

On mobile, prefer ctx.ui.mobile.locate(...) over driver.$(...) where it suffices. locate() applies UiAutomator resourceId wrapping, the iOS class-name predicate workaround, and xpath normalization that the raw driver does not.

Runtime Variables

const v = await ctx.runtime.get("orderId");   // undefined when unset
await ctx.runtime.set("orderId", "ORD-1042");
await ctx.runtime.set("token", jwt, { isEncrypted: true });

AI

const answer = await ctx.ai.invoke("Summarize this receipt", {
  files: ["receipt.png"],
});

File paths are relative to ctx.tmpDir, or within your fs allowlist.

OCR

findImage accepts threshold, scale, and occurrence options.