Importable from halyard.phone. Coordinates and boxes are screen pixels: the phone’s frame, from the top-left, the space screenshot(), observe() and tap() all use.

Coords

A point. A TypedDict, so a plain dict works.
{"x": 540, "y": 1200}
KeyType
xint
yint

BBox

A box from its top-left corner. A TypedDict.
KeyType
xint
yint
widthint
heightint

Screen

One observed frame, from phone.observe(). A frozen dataclass.
FieldTypeDescription
textslist[Element]Every text region OCR found.
iconslist[IconBox]Every icon found.
hashstrA hash of the frame.
width, heightintThe frame’s size.
captured_atdatetimeWhen the frame was captured, in UTC.

screen.find_text()

The first text region whose text matches: a case-insensitive substring unless exact.
screen.find_text(text, *, exact=False)
Returns Element | None.

screen.find_all_text()

Every text region that contains contains, or matches the regular expression pattern. With neither, every region.
screen.find_all_text(*, contains=None, pattern=None)
Returns list[Element]. Raises ValueError when both are given.

Element

One text region OCR found. Plain data: to act on something, use a locator (phone.get_by_text(...)), which finds it again on the current screen.
FieldTypeDescription
bboxBBoxWhere it is.
centerCoordsIts centre, ready for phone.tap().
confidencefloatOCR confidence.
textstr | NoneThe text read.
source"ocr"Always "ocr".

IconBox

One detected icon. The icon model finds where icons are, not what they are.
FieldTypeDescription
bboxBBoxWhere it is.
centerCoordsIts centre.
confidencefloatDetection confidence.

DeviceInfo

What the phone is, from phone.device_info() and the handshake. Size behaviour from input_modalities and form_factor, not from a platform string.
FieldTypeDescription
device_idstrThe phone’s id.
platformstrThe OS family, such as "android".
form_factorstrThe physical class, such as "phone".
input_modalitiestuple[str, ...]What it accepts, such as ("touch", "keyboard").
modelstrBest effort; empty when unknown.
os_versionstrBest effort; empty when unknown.
screen_width, screen_heightintThe frame size in pixels.

HandshakeResult

The Protocol.handshake reply, from phone.handshake().
FieldTypeDescription
protocol_versionintThe protocol version the phone speaks.
deviceDeviceInfoWhat the phone is.
domainstuple[str, ...]The domains it advertises.
capabilitiestuple[str, ...]The exact methods it answers.
MethodReturns
supports(method)Whether method (such as "Touch.tap") is in capabilities.
has_domain(domain)Whether domain (such as "Accessibility") is in domains.