AndroidHire
← Back to books

Chapter 2: The Testing Landscape

Chapter 1 argued for a philosophy: buy confidence at the lowest cost that still resembles real usage, and test behavior rather than implementation. That's the compass. This chapter is the map.

Because here's the practical problem the philosophy leaves open. You sit down to test a feature — say, the login screen we kept returning to in Chapter 1 — and you quickly realize it isn't one thing to test. There's a password rule. There's a ViewModel that turns a repository result into screen state. There's a composable that renders that state. There's how the whole screen looks in dark mode at a 2x font scale. And there's the question of whether a real person, on a real device, can actually open the app and sign in. That's five different behaviors, and — this is the part people miss — five different kinds of test serve them best. Reach for the wrong kind and you either pay too much for the confidence or fail to catch the bug at all.

So this chapter does two things. First, it defines the five categories precisely enough that we can talk about them without talking past each other for the rest of the book. Second, it gives you a repeatable decision procedure: I have a behavior in front of me — which test do I write? By the end you'll be able to look at any piece of an Android app and know, quickly and defensibly, where its tests belong.

2.1 A Vocabulary Problem

Before the map, a warning about the labels on it. The words "unit," "integration," "UI," and "end-to-end" are used inconsistently across the industry, and more energy has been wasted arguing about their definitions than almost any other topic in testing. The worst offender is "unit." Ask ten engineers what a unit is and you'll hear "a class," "a function," "a method," "a module" — and each answer quietly reshapes how they test.

This book takes a deliberate position, borrowed from Martin Fowler and Kent Beck: a unit is a behavior tested in isolation from slow or nondeterministic collaborators — not "one class." What makes a test a unit test isn't how many classes it touches; it's what it isolates. A test that exercises three small classes that always travel together, with no database, network, or clock in the loop, is still a unit test in every way that matters. Fowler draws the useful distinction between solitary unit tests, which stub out every collaborator, and sociable ones, which let real collaborators participate. Both are legitimate; the choice is about what you're trying to protect.

The payoff of this stance is that it dissolves a lot of pointless debate. When someone asks "is this a unit test or an integration test?" the honest answer is often "it's on the spectrum, and the label doesn't change whether it's a good test." What matters is: what does this test isolate, and what confidence does it buy? Hold onto that question. It's worth more than any taxonomy.

With that caveat stated plainly, here is the taxonomy — not as rigid boxes, but as five useful reference points along a continuum from "one behavior in a vacuum" to "the whole app in the world."

2.2 The Five Categories

Unit tests

A unit test exercises a single behavior with its slow and nondeterministic collaborators removed. It runs on the JVM, in milliseconds, and it fails for exactly one reason.

This is the right tool for pure logic: validation rules, calculations, date and money math, parsers, mappers, and the small state-shaping functions that live in your domain layer. These have no reason to touch Android, and testing them anywhere but the JVM is pure waste.

class PasswordValidatorTest {
    private val validate = PasswordValidator()

    @Test
    fun `rejects a password shorter than eight characters`() {
        assertEquals(
            PasswordValidation.Invalid(Reason.TooShort),
            validate("hunt2"),
        )
    }

    @Test
    fun `rejects a password with no digit`() {
        assertEquals(
            PasswordValidation.Invalid(Reason.NoDigit),
            validate("hunterhunter"),
        )
    }

    @Test
    fun `accepts a password that meets every rule`() {
        assertEquals(PasswordValidation.Valid, validate("hunter2secure"))
    }
}

No coroutines, no Android, no test rule — just a function and its outputs. What a unit test can't tell you is whether your units wire together correctly, or whether anything ever renders on screen. It proves a part works; it says nothing about the whole.

Integration tests

An integration test lets several real units collaborate across a seam, replacing only the genuinely expensive boundaries — the network, the disk, the clock — with fast fakes. In the trophy from Chapter 1, this is the tall middle section, and on modern Android it usually runs on the JVM (often under Robolectric) in tens of milliseconds.

This is where most bugs actually live. A ViewModel individually works; a repository individually works; the bug is in the seam between them — the mapping, the error handling, the order of state emissions. The "good" login test from Chapter 1 — the one driving LoginViewModel with a FakeAuthRepository and asserting the state stream through Turbine — is an integration test in this sense. It never mentions a single implementation detail, and it exercises the ViewModel and its collaborator together, the way they really run.

@Test
fun `failed login surfaces an error state`() = runTest {
    val repository = FakeAuthRepository(validCredentials = "ada@example.com" to "hunter2secure")
    val viewModel = LoginViewModel(repository)

    viewModel.uiState.test {
        assertEquals(LoginUiState.Idle, awaitItem())

        viewModel.login("ada@example.com", "wrong-password")

        assertEquals(LoginUiState.Loading, awaitItem())
        assertEquals(LoginUiState.Error("Invalid credentials"), awaitItem())
    }
}

Integration tests buy a lot of confidence per millisecond, which is exactly why the trophy leans on them. What they don't prove is that the fully assembled app — real database, real network, real navigation graph — holds together, or that any of it looks right.

UI tests (Compose)

A Compose UI test renders real composables and interacts with them through the semantics tree — the same accessibility model a screen reader consumes — then asserts on what a user would see or do. Thanks to Robolectric, most of these can run on the JVM; when you need real graphics or system integration, the identical test can run instrumented on a device.

@Test
fun errorMessage_isShown_whenLoginFails() {
    composeTestRule.setContent {
        LoginScreen(uiState = LoginUiState.Error("Invalid credentials"))
    }

    composeTestRule
        .onNodeWithText("Invalid credentials")
        .assertIsDisplayed()
}

Notice the overlap with the previous category. Drive a real LoginViewModel from inside a Compose test and you have something that is simultaneously a UI test and an integration test. That's not a contradiction to resolve — it's the fuzzy boundary from Section 2.1, and it's a feature. A UI test proves the screen shows and reacts to the right things for a given state. What it doesn't prove is that the screen looks correct pixel-for-pixel — a Text can be present in the semantics tree and still be clipped, mis-coloured, or overlapping something else.

Screenshot tests

A screenshot test renders a composable to actual pixels and compares the image against a previously approved "golden." It runs on the JVM through tools like Paparazzi, Roborazzi, or Compose Preview Screenshot Testing, in the low hundreds of milliseconds, with no device in the loop.

This category catches an entire class of defect the others are blind to: a label truncated at a long locale, broken contrast in dark mode, a layout that shatters at a large font scale, spacing that drifts after a theme change. As noted in Chapter 1, the classic pyramid has no slot for this at all.

class LoginScreenScreenshotTest {
    @get:Rule
    val paparazzi = Paparazzi(deviceConfig = DeviceConfig.PIXEL_5)

    @Test
    fun loginScreen_errorState_darkMode() {
        paparazzi.snapshot {
            AppTheme(darkTheme = true) {
                LoginScreen(uiState = LoginUiState.Error("Invalid credentials"))
            }
        }
    }
}

The strength and the limit are the same fact: a screenshot test only knows whether the rendering matches the approved baseline. It proves the screen looks like what you signed off on. It proves nothing about behavior or logic — a screen can look flawless and do entirely the wrong thing.

End-to-end tests

An end-to-end test drives the fully assembled app through its real UI, ideally on a real device, across a complete user journey. These run through tools like Maestro or instrumented Compose tests, and they are the slow, precious cap at the top of the trophy.

# Maestro flow: login.yaml
appId: com.example.app
---
- launchApp
- tapOn: "Email"
- inputText: "ada@example.com"
- tapOn: "Password"
- inputText: "hunter2secure"
- tapOn: "Log in"
- assertVisible: "Welcome back, Ada"

An E2E test proves the thing that nothing else can: that all the pieces, wired together and running for real, actually work for a journey a user cares about. That is genuine, irreplaceable confidence. But it comes at a price — these tests are slow, they're the most prone to flake, and when one fails it points at "somewhere in this whole journey," not at a line of code. That cost profile is why E2E is a small, deliberate handful of critical paths, never your primary safety net.

The landscape at a glance

Category What it isolates / includes Runs on Typical speed What it proves What it can't prove Primary tools
Unit One behavior; collaborators faked or absent JVM ~1–10 ms Logic is correct in isolation Units wire together; anything renders JUnit, MockK, Truth, Turbine
Integration Several real units across a seam JVM (Robolectric) or device ~10–100 ms Units collaborate correctly The whole app assembles; visual correctness JUnit, Robolectric, Turbine, fakes
UI (Compose) Rendered semantics + interaction JVM or device ~10–100 ms (JVM) Screen shows/reacts correctly for a state It looks right to the pixel; real backend works Compose UI Test, Robolectric
Screenshot Pixel rendering of a composable JVM ~50–200 ms It matches the approved baseline Any behavior or logic Paparazzi, Roborazzi, Preview Screenshot Testing
End-to-end Whole app, real UI, full journey Device / emulator Seconds and up The assembled app works for a real journey Exactly where a bug is; edge cases economically Maestro, Espresso, Compose on device

2.3 The Boundaries Are Fuzzy — On Purpose

Look at that table and you'll notice the categories bleed into each other. A Compose test with a fake ViewModel is a UI test and an integration test. A screenshot test is a UI test that happens to assert on pixels instead of semantics. An instrumented Compose journey is a UI test and an E2E test. This bothers people. It shouldn't.

Fowler's observation applies directly: the value of these labels is in communication, not classification. You do not earn confidence by correctly filing a test under the right heading. You earn it by knowing what the test isolates and what it proves. When a test spans two categories, ask the two questions that actually matter — what does it isolate? and what confidence does it buy? — and let the label be whatever's convenient. Any minute spent debating whether something is "really" a unit or integration test is a minute not spent catching a bug.

The one rule the categories exist to serve: every test you keep should catch something the others can't. If a screenshot test and a UI test and an integration test all fail on the same regression, you're paying three maintenance bills for one signal. Diversity of failure, not volume of tests, is what a healthy suite optimizes for.

2.4 A Decision Framework

Here is the procedure to run whenever you're staring at a behavior and wondering where its test belongs. It's three questions, applied in order.

1. Name the observable behavior you're protecting. Not "test the ViewModel" — that's a class, not a behavior. Say it as an outcome someone could notice: "an eight-character minimum is enforced," "a failed login shows an error," "the screen is readable in dark mode," "a user can sign in and reach home." If you can't state the behavior as an observable outcome, you don't yet know what you're testing, and no category will save you.

2. Find the cheapest test that resembles real usage closely enough to catch that behavior's regression. Start at the bottom of the trophy and climb only as far as you must. Pure logic? A unit test resembles usage perfectly and costs nothing — stop there. Behavior that only emerges when collaborators interact? Climb to integration. Behavior a user can only perceive on a rendered screen? Climb to a UI test. Something only pixels can catch? Screenshot. Something only the fully assembled app can prove? E2E — and only then.

3. Ask what could break this behavior that nothing else in your suite is covering. This is the check against redundancy from Section 2.3. If an integration test already proves the error state is produced, a UI test's job is narrower — that the state renders as a visible message — and a screenshot's job is narrower still: that the message is legible. Each level earns its place by covering a failure mode the others can't see.

That procedure collapses into a lookup table you'll internalize within a week:

If the behavior is… Reach for… Because…
A pure calculation, validation, parse, or mapping Unit No collaborators or UI involved; fastest possible feedback
State produced by orchestrating collaborators Integration (ViewModel + fakes) The bug lives in the seam, not in the pieces
"The screen shows or does X when state is Y" Compose UI test (JVM first) You need the semantics tree, not just the state object
"It looks correct across themes, sizes, and locales" Screenshot Only pixels can catch a visual regression
"A whole critical journey works when assembled" E2E (sparingly) Only the assembled app proves the wiring end to end

The heuristic underneath the whole table is simple enough to say in one line: start at the cheapest level, and climb only when the behavior genuinely lives higher up. Most behaviors live lower than your instinct suggests.

2.5 One Feature, Five Ways

Let's make the framework concrete by running it across a single feature. Login has behaviors at every level of the trophy, and seeing the same feature tested five ways is the clearest possible picture of how the categories divide labor. You've already met two of these in Chapter 1; here they are in their full context.

Unit — the password rule. The behavior: a password must be at least eight characters and contain a digit. Pure logic, no collaborators. This is the PasswordValidatorTest from Section 2.2 — microseconds to run, and it would be malpractice to test it any higher up.

Integration — the ViewModel's state transitions. The behavior: valid credentials drive the screen from idle to loading to authenticated, and bad credentials drive it to an error state. This only emerges when the ViewModel and repository interact, so we test them together with a fake repository and assert the state stream through Turbine — the LoginViewModel tests from Chapter 1 and Section 2.2.

UI — the error renders. The behavior: when the state is Error, the user sees the message. That's a claim about the rendered semantics tree, not about a state object, so it climbs to a Compose UI test — errorMessage_isShown_whenLoginFails. Note how narrow its job is: the integration test already proved the error state is produced, so the UI test only has to prove that state becomes a visible message.

Screenshot — it's legible in the dark. The behavior: the error state is readable and correctly laid out in dark mode. No text assertion can see contrast or clipping, so this is a screenshot test — loginScreen_errorState_darkMode. Again, narrower than the level below it: not "is the message present" but "is it legible."

E2E — a real person can sign in. The behavior: launch the real app, type real credentials, tap log in, land on home. Only the assembled app proves the navigation graph, the dependency wiring, and the real screen all cooperate — so this one, and only this one, climbs all the way to a Maestro flow.

Five behaviors, five levels, and each test catches something the ones below it cannot. That is a healthy vertical slice through the trophy — and it's the pattern every feature in this book will follow.

2.6 Anti-Patterns

A few failure modes recur often enough to name.

The ice-cream cone. The pyramid, inverted: a mountain of slow E2E tests balanced on a sliver of unit tests. It's how teams end up with a suite that takes an hour, flakes constantly, and still catches bugs late and vaguely. It usually grows by accident, one "let's just add an end-to-end test for this" at a time. The decision framework in 2.4 is the antidote — climb only when you must, and most behaviors won't ask you to.

Testing the same behavior at every level. If your integration test, UI test, and screenshot test all fail on the identical regression, you're paying three maintenance bills for one signal, and every refactor now breaks three tests instead of one. Each level should own a distinct failure mode (Section 2.3). Redundant coverage feels thorough and is actually a tax.

Treating the trophy's proportions as hard numbers. There is no correct ratio of unit to integration to E2E. The shape is a heuristic about where confidence tends to be cheap, not a quota to hit. A data-heavy domain layer will be unit-heavy; a thin app over a rich backend will lean integration and E2E. Let the app's actual risk profile set the mix.

Choosing a level by habit instead of by need. "We always write an instrumented test for screens" is how behavior that could've been a 10 ms JVM test becomes a 10-second device test. Run the framework each time; don't autopilot into your team's default level.

Testing the framework instead of your code. A test that asserts Room can store a row, or that Compose can display a Text, is testing Google's code, not yours. Those libraries have their own test suites. Your tests should assert your logic, your state, your screens — the things that can actually regress when you change your code.

Key Takeaways

  • A unit is a behavior isolated from expensive collaborators, not "one class." What a test isolates matters more than what label it wears.
  • The five categories — unit, integration, UI, screenshot, end-to-end — are reference points on a continuum, and their boundaries are deliberately fuzzy. Don't debate the labels; ask what each test isolates and what confidence it buys.
  • The decision framework is three questions: name the observable behavior, find the cheapest test that resembles real usage closely enough to catch its regression, and confirm it covers a failure mode nothing else does.
  • Start at the cheapest level and climb only when the behavior genuinely lives higher. Most behaviors live lower than instinct suggests.
  • A healthy suite optimizes for diversity of failure, not volume of tests — every test you keep should catch something the others can't.
  • Watch for the ice-cream cone, redundant coverage across levels, cargo-culted ratios, autopilot level-selection, and tests that exercise the framework instead of your code.

What's Next

You now have the map and the compass. Chapter 3 gets your project ready to act on them: source sets and where each kind of test actually lives (test versus androidTest), the Gradle configuration that makes JVM and device tests coexist, the JUnit 4 versus JUnit 5 decision this book has been deferring, and the Robolectric setup that lets so many of these categories run on the JVM in the first place. By the end of it, every category from this chapter will have a real home in a real build — and we'll introduce the example app that carries us through the rest of the book.

Prev chapterNext chapter