Chapter 2: How Assistive Technology Works on Android

If you did the homework from Chapter 1, you've now spent a few uncomfortable minutes driving your phone with TalkBack on. You heard your favorite app narrate itself, got lost somewhere in a settings screen, and switched it off with a small sigh of relief. Good. That discomfort is the single most useful thing you'll bring to this book.

But an experience isn't an explanation. To build accessible apps deliberately — rather than by trial, error, and superstition — you need to understand what's actually happening under the hood when an assistive service touches your UI. Where does the voice get its words? Why does focus jump where it jumps? Why does one button announce "Add note, button, double tap to activate" while the one right next to it just says "button"?

This chapter answers those questions. We're not writing much Compose yet; we're building the mental model that makes every later chapter obvious instead of magical. By the end you'll understand the one abstraction that all of Android's assistive technologies share — and, crucially, how your Compose code turns into something they can read.

Your app is not a picture

Here is the core idea, and if you take nothing else from this chapter, take this:

Assistive technologies do not see your screen. They read a description of it.

When you look at an app, your eyes take in a two-dimensional picture — colors, shapes, spatial arrangement — and your brain effortlessly reconstructs meaning from it. "That's a button. That's a heading. That's a list of five items. That one's selected." You do this so automatically that it's easy to forget you're doing it at all.

A blind user's phone can't do that. There's no eye, no picture-to-meaning leap. Instead, Android maintains a separate, structured description of everything on screen — a data structure, not an image — and assistive services read from that. Every button, every piece of text, every checkbox exists in this description as a node, and each node carries properties: what it is, what it says, what state it's in, what you can do to it.

This description is a tree, mirroring the nesting of your UI. In the classic Android View world it's built from objects called AccessibilityNodeInfo. In Compose, as you'll see, it comes from the semantics tree — but the principle is identical, and it's the thing every assistive service consumes.

So when we say "make your app accessible," what we very concretely mean is: make sure that structured description is complete, correct, and sensibly organized. A gorgeous screen with an empty or garbled description is, to an assistive technology, a blank wall. An ugly screen with a rich, correct description is perfectly usable. Accessibility lives in the description, not the pixels.

Let's meet the services that read it.

TalkBack: the screen reader

TalkBack is Android's built-in screen reader, and it's the assistive technology you'll test against most often. It's used primarily by people who are blind or have low vision, and it does two things: it speaks the interface aloud (or sends it to a refreshable Braille display), and it remaps touch gestures so the user can explore and control the phone without seeing it.

That gesture remapping is the part that surprises developers. With TalkBack on, a single tap no longer activates things. Instead:

Swipe right / swipe left moves accessibility focus to the next or previous node — a green rectangle you'll see outlining the currently focused element. This is linear navigation: the user walks through the screen one node at a time, in order.
Double-tap anywhere activates whatever currently has focus. So the flow is always "swipe to the thing, then double-tap to press it" — never "tap the thing directly."
Explore by touch: dragging a finger around the screen reads whatever is under it, letting the user build a spatial map.
Swipe up-then-down and similar angular gestures open menus and change navigation granularity — letting the user jump by heading, by paragraph, by word, by control, and so on.

Two consequences of this design matter enormously for how you build:

Order matters. Because linear navigation walks the tree in sequence, the order in which nodes are visited is the order the user experiences your screen. If your visual layout and your semantic order disagree, the user gets a confusing, out-of-sequence reading. (We devote real attention to controlling this in Chapter 6.)

Grouping matters. Every separate node is another swipe. If a single logical thing — a note card with a title, a date, and a preview — is exposed as three separate nodes, the user has to swipe three times to get past one card, hearing three disconnected fragments. If it's merged into one node, they hear "Groceries, edited yesterday, buy milk and eggs" in a single focus stop. Same pixels; wildly different experience. (This is Chapter 5.)

When TalkBack focuses a node, it assembles an announcement from that node's properties, roughly in this order: its label (the text or content description), its role ("button," "heading," "checkbox"), its state ("checked," "disabled," "selected"), and the available actions ("double tap to activate," "double tap to toggle"). This is why the humble favorite button from Chapter 1 announced "Add to favorites, button, double tap to activate" once we gave it a description — every clause in that sentence came from a specific semantic property. Your job, throughout this book, is to make sure each of those properties is set correctly.

Switch Access: control without touch

Not everyone can perform TalkBack's swipes and double-taps. Switch Access serves people with significant motor impairments who operate their device using one or more physical switches — a button, a foot pedal, a sip-and-puff device, even a detected head movement or facial gesture.

Because the user may have only one or two inputs available, Switch Access works by scanning: it moves a highlight through the on-screen elements one at a time (or group by group), and the user hits their switch to select whatever is currently highlighted. Think of it as the interface taking turns offering each control to the user, who says "yes, that one" when the right one comes around.

Switch Access reads from the same accessibility tree TalkBack does. So the improvements you make for screen-reader users — clear grouping, sensible order, every interactive element actually exposed as focusable and actionable — directly benefit switch users too. There's a curb-cut effect even within accessibility work: the structural fixes pay off across multiple services at once.

The specific thing Switch Access punishes is unreachable or fake-interactive elements. If you drew something that looks like a button but is really just a Box with an onClick buried in a way that doesn't register as a semantic action, a mouse or finger user might stumble onto it — but a switch user, whose entire world is "what does the scanner offer me," may never be able to reach it at all. Every tappable thing must be a real, actionable node.

Voice Access: driving by speaking

Voice Access lets users control their phone entirely with spoken commands — "open Beacon," "scroll down," "go back," "tap Add note." It serves people with motor impairments who find speaking easier than touching, and anyone whose hands are otherwise occupied.

Voice Access leans hard on your labels. When a user says "tap Add note," the system matches those words against the labels in the accessibility tree. If your button's label is a clear "Add note," the command just works. If the button is unlabeled, Voice Access falls back to overlaying numbers on interactive elements so the user can say "tap 7" instead — functional, but clumsy, and it forces the user to hunt for the number rather than name what they want.

The lesson lands in the same place as everything else: the label you write is the interface. A good content description isn't just narration for screen readers; it's the literal command a Voice Access user speaks. Vague or missing labels degrade the experience for a service you may never have thought about.

Magnification, font scale, and display size

The remaining tools are less about the tree and more about rendering, but they break just as many apps.

Magnification lets a user zoom into part of the screen and pan around it. It mostly Just Works — Android handles the zoom — but it exposes layouts that assume the user can see the whole screen at once. Content that only makes sense in full view, or that shifts unexpectedly, becomes hard to track.

Font scale is the big one. In system settings, users can enlarge text — often well beyond 200%. When they do, every piece of text sized in scale-aware units grows to match. The catch: text sized in the wrong units doesn't grow, and layouts built with fixed heights clip or overlap when text does grow. An enormous fraction of real-world accessibility bugs are simply "the developer hard-coded a size and the text no longer fits." Compose gives you the right tools here — the scalable sp unit for text, and layouts that flex — and we'll cover honoring font scale properly in Chapter 7.

Display size scales the whole UI, not just text, with similar consequences for rigid layouts.

The through-line: these users haven't changed your semantic tree at all. They've changed how big things need to be, and they've told the system so through settings your app is expected to respect. Building flexible, scale-aware layouts is how you respect them.

How Compose feeds all of this

So where does Compose fit? Here's the pipeline, end to end:

You write composables, some with explicit semantics {} modifiers, most relying on the built-in semantics that Material and Foundation components provide.
Compose builds a semantics tree — the parallel, meaning-focused description we introduced in Chapter 1. Every node carries its properties: label, role, state, actions.
Compose translates that tree into the platform's accessibility objects (AccessibilityNodeInfo), the same currency the old View system produced. This translation is why a Compose app and a View app look identical to an assistive service — they both ultimately hand the platform the same kind of node tree.
The accessibility services — TalkBack, Switch Access, Voice Access — read those objects and present them to the user through speech, scanning, or voice matching.

The beautiful part is that you operate almost entirely at layer 2. You don't touch AccessibilityNodeInfo; you don't write per-service code. You describe meaning in the semantics tree once, and Compose plus the platform fan it out to every service correctly. Get the meaning right, and TalkBack, Switch Access, and Voice Access all benefit from the same edit.

This is also why the rest of the book is organized around the semantics tree rather than around individual services. There is no "TalkBack chapter" and separate "Switch Access chapter," because you don't really program them separately. You program the tree. The services are just different readers of the same book.

A quick look at Beacon

Let's ground this in our running example. Beacon, at this stage, is a simple app: a screen titled "My Notes," a list of note cards, and a floating action button to add a new note. Nothing about it is accessibility-aware yet — it's a perfectly ordinary Compose app, which is exactly the point. Most apps start here.

If you turned TalkBack on and swiped through Beacon right now, here's the kind of thing you'd hit, and why:

The "My Notes" title reads fine as text — but it isn't marked as a heading, so a user who navigates by heading (a common way to skim a screen quickly) can't jump to it. It's just another stop in the linear walk.
Each note card is a Row of separate Text elements, so it reads as several disconnected fragments — three or four swipes per card instead of one.
The add button is an IconButton with a plus icon and contentDescription = null, so it announces "button" with no hint of what it does. A Voice Access user can't say "tap Add note"; they're stuck with "tap 4."

None of these are exotic bugs. They're the default result of building a normal screen without thinking about the tree — and every one of them maps to a chapter ahead: headings (Chapter 5), grouping (Chapter 5), labeling (Chapter 4). Over the next chapters we'll fix each, and watch Beacon's accessibility tree turn from a pile of fragments into a clean, navigable structure.

Try this before the next chapter

Last chapter you tried TalkBack. This time, meet its siblings — briefly, just to feel how differently each one consumes the same screen:

Switch Access. Enable it in Settings → Accessibility → Switch Access and set up the volume keys as switches (the setup wizard walks you through it). Open an app and watch the scanning highlight move. Notice how it's entirely at the mercy of what's reachable.
Voice Access. Enable it, open an app, and try naming things: "scroll down," "go back," and "tap [some button label]." Then find a poorly labeled control and watch it force you into numbered overlays.
Font scale. In Settings → Display → Font size (and Display size), crank both up near maximum, then reopen a few apps — including your own. Note every place text clips, overlaps, or gets cut off. That list is a preview of Chapter 7's work.

You don't need to master any of these. You just need to have felt them once, so that when we talk about "actions" and "focusable nodes" and "scale-aware units," you have a real memory to attach the words to.

Summary

Assistive technologies don't see your screen; they read a structured description of it — a tree of nodes, each carrying properties like label, role, state, and actions. Accessibility lives in that description, not the pixels.
TalkBack is the screen reader: it speaks the UI and remaps gestures so users navigate node by node (swipe) and activate by double-tapping. This makes order and grouping decisive.
Switch Access serves motor-impaired users via scanning; it depends on every interactive element being a genuinely reachable, actionable node.
Voice Access matches spoken commands against your labels, so the label you write becomes a literal voice command.
Magnification, font scale, and display size don't change the tree — they change how big things must be, and they break rigid layouts. Respecting them is a layout discipline.
In Compose, you work almost entirely at the level of the semantics tree. Compose translates it into the platform's accessibility objects, and every service reads from there — so one correct description serves all of them.

Now that you know what reads your app and why the tree is the thing that matters, we can look at the tree itself. In the next chapter we open up Compose's semantics system in detail — how it's built, how to inspect it, and how to bend it to your will.