4. Probes

The question. Which experiments can a user run on an app, and what does each one expose?

4.1 A probe is an experiment, not a session of use

Normal use shows you what an app does when everything works. That is the least informative condition, because most designs look the same when the network is fast, the process is alive and one device is in play. A feed that refetches on every visit and a feed that reads from a local store both show a list of items.

Designs separate under stress. Take the network away and the first feed shows an error while the second shows content. End the process and one app returns to where you were while another starts over. Each of the seven constraints in Chapter 2 is a force the team had to answer, and the answer stays hidden until the force is applied.

A probe applies one such force on purpose. It is a repeatable experiment a normal user can run on an app to expose one aspect of its design. It has three parts: an action you take, a behavior you watch, and a reading that connects each possible behavior to a design.

The method uses only what a normal user has: the app, device settings, a second device or account, and time. You do not inspect traffic and you do not open the app’s package. This limit is deliberate. It keeps the method legal and safe, it works on every platform, and it trains the skill the book is about, which is reasoning from behavior to design.

4.2 What makes a good probe

A probe is worth running when it has four properties.

Property	Meaning	A probe that fails it
Cheap	It takes minutes and needs nothing you do not already have	Leaving a spare device untouched for six weeks
Safe	It cannot lose data, cost money or affect another person	Deleting your only copy of something to see whether it returns
Repeatable	The same steps give the same result, so a surprise can be checked	An effect that depends on a network glitch you cannot cause again
Discriminating	Its possible outcomes point to different candidate designs	Watching a screen load on a fast network

The fourth property matters most and is the easiest to miss. A probe is discriminating when at least two candidate designs predict different outcomes for it. If every design you are considering would produce the same behavior, the probe tells you nothing, however dramatic the behavior looks.

Consider a “like” button that turns red the moment you tap it. Tapping it again and again is cheap, safe and repeatable, and it discriminates nothing. Both an optimistic update and a fast round trip to the backend turn the button red at once. Tapping it in airplane mode is the discriminating version. The optimistic design still turns the button red. The round-trip design shows an error or leaves the button unchanged.

The four properties trade against each other. The most discriminating probes are often the expensive ones: a long absence, an old app version, a clock set ahead. Section 4.6 gives the order that manages this trade.

Not every probe has to score well on all four. An expensive probe is worth running when it is the only one that separates two designs you care about. An unsafe probe is never worth running.

4.3 The Do, Watch, Read format

Every probe in this book is written in the same three parts.

Do. The steps, in order, each one an instruction. The steps state the starting condition, because a probe that begins from an unknown state is not repeatable. “Force-quit the app, then turn on airplane mode, then open the app” is a different probe from “turn on airplane mode while the app is open”.

Watch. What to look at while you do the steps. This part exists because the informative detail is usually small: a pending indicator on one item, a timestamp that shifts by a second, a launch screen that appears for a moment. Without direction you look at the content and miss the mechanism.

Read. A list of outcomes. Each outcome is something you can see, the outcomes are mutually exclusive, and each has a reading: what it means about the design, with a confidence word. The three words are fixed across the book.

Word	Meaning
Observed	Seen directly in a probe. A fact about behavior.
Inferred	The design that best explains the observations, with the alternatives ruled out by a probe.
Speculative	Consistent with the observations, but at least one other design fits equally well.

The outcome is always a fact about behavior. The reading is a claim about design, and it carries the word that says how far the claim goes. Chapter 5 covers how a reading earns its word.

The format has a practical use beyond tidiness. Writing the outcomes down before you run the steps forces you to decide what each result would mean. A probe whose outcomes you cannot list in advance is an exploration. Explorations are useful for finding signals, and they are not evidence for a design until you turn them into a probe with stated outcomes.

On this site a probe is also a form. You select the outcome you saw, and the selection is recorded in your worksheet. Chapter 6 describes the worksheet.

4.4 The eight probe families

Probes fall into eight families. Seven of them apply a force to the app. The eighth applies none and reads what the device and the store already report.

Figure 4.1 places the active families on the universal app model of Chapter 3. Each family acts on a different part of the model, which is why each one exposes different components.

Figure 4.1. Where each active probe family acts on the universal app model. Passive observation acts on nothing and reads what the device reports.

Family	You change	It exposes
1. Network probes	The connection between client and backend	What the client keeps, what it queues, how it retries
2. Lifecycle probes	Whether the process and its memory exist	What is stored durably and what is held in memory
3. Time probes	The clock, the length of an absence, the app version	Expiry rules, whose clock is trusted, how old clients are treated
4. Multi-device probes	The number of clients on one account or one item	Delivery paths, conflict rules, session rules
5. Fresh-state probes	Everything stored on the device	What lives only on the device and what the backend can restore
6. Resource probes	Storage, power and data budgets	What the app gives up first when rationed
7. Permission probes	The app’s access to platform services	How features degrade, and when the app asks
8. Passive observation	Nothing	Footprint, background activity, release habits, stated behavior

Network probes

Airplane mode, a weak connection, a switch between networks.

Airplane mode is the cheapest strong probe there is. It splits every feature into what works from the device alone and what needs the backend, which maps directly onto the local store, the cache and the outbox. A weak connection is a different probe, not a milder one. With no network a request fails at once. With a weak network a request hangs, half completes or succeeds while its response is lost, and that exposes timeouts, retries, request priorities and duplicate handling. A switch between networks, for example from wireless to mobile data in the middle of an upload, exposes whether transfers and persistent connections resume or restart.

These probes carry most of Chapter 11, Chapter 12 and Chapter 13. Chapter 14 uses them to see how a live connection recovers, and Chapter 15 to see how far a list was loaded ahead. The media chapters, 16 to 19, use a weak connection to expose quality adaptation and resumable transfers.

Lifecycle probes

Force-quit, a long time in the background, a device restart.

Process death removes everything the app held in memory. Whatever is still there afterwards was written to storage. Lifecycle probes therefore draw the line between screen state held in memory and data in the local store, and they do it one feature at a time.

The three probes in the family are not interchangeable. A force-quit is a deliberate act by the user, and many apps choose not to restore the previous screen after one. A long time in the background lets the operating system end the process on its own, which is the case teams design state restoration for. A device restart ends everything at once and shows what starts again without you opening the app, such as notifications still arriving or an upload continuing.

Chapter 7 uses lifecycle probes to compare a cold start with a warm start. Chapter 8 uses them to find what survives when the app is killed, and Chapter 9 to find where each piece of on-screen data lives. Chapter 12 combines them with network probes to test whether pending mutations are durable. Chapter 23 uses them to study background work.

Time probes

A changed clock, a long absence, an old app version.

Many designs contain a rule about time that never fires in a ten-minute session: a cache entry expires, a session ends, a cursor becomes too old to use, an old app version is refused. Time probes make those rules fire.

A changed clock shows whose clock a rule trusts. If setting the device clock ahead expires a cache or changes which edit wins a conflict, the rule runs on client time. If nothing changes, the backend’s clock decides. A long absence is the honest version of the same probe and it costs calendar time. An old app version, kept on a spare device with updates turned off, shows how the backend treats clients it can no longer change.

Chapter 11 uses time probes for cache expiry, and Chapter 12 for clock skew and long absence. Chapter 20 uses them for session lifetime, Chapter 30 for how long fetched configuration is kept, and Chapter 33 for forced upgrades and old-version support.

Multi-device probes

A second device, a second account.

One client can never show you a delivery path or a conflict, because both need at least two parties. A second device signed in to the same account shows how a change travels from one client to another, how fast, and what triggers it. A second account shows the same for shared data, and adds what one user can see of another: presence, typing indicators, read state, access changes.

The second device does not have to be a phone. A tablet, an old handset or the app’s web version all serve. For a second account, use one you own or one that belongs to someone who has agreed to help.

Chapter 14 is built on this family, and Chapter 12 uses it for every conflict probe. Chapter 9 uses it to see which screens react to a change made elsewhere, and Chapter 39 uses it throughout. Chapter 20 uses it to find the session rules.

Fresh-state probes

A fresh install, cleared data, sign out and sign in.

These probes empty the device and watch what comes back. Whatever returns came from the backend, or from a backup. Whatever does not return lived only on the device. The order and speed of the return show what the team loads first and what it leaves for a backfill.

The three probes empty different amounts. Sign out and sign in removes one account’s data and keeps the app’s own settings, or should. Cleared data returns the app to its first launch on a device the backend may still recognize. A fresh install on a device that has never had the app is the cleanest state and the only one that shows true first-run behavior, including what the app allows before sign-in.

Chapter 7 uses a fresh install to see a first launch, Chapter 11 to watch a cache fill from empty, and Chapter 12 to watch a first sync. Chapter 20 uses sign out and sign in. Chapters 27 and 30 use a fresh install to see what a new user is shown.

Resource probes

Low storage, low-power mode, data-saver mode.

An app that is rationed has to give something up, and what it gives up first shows what the team considers optional. Under low-power mode, look for background refresh that stops, animations that simplify and video that no longer plays on its own. Under data-saver mode, look for lower media quality and prefetching that stops. Under low storage, look for a cache that shrinks and downloads that are refused.

This family also separates the app’s own choices from the operating system’s. Some effects are imposed on every app, and you learn those by running the same probe on two unrelated apps.

Chapter 11 uses low storage to find what is discarded, and Chapter 13 uses data-saver mode to find which requests are treated as optional. Chapters 16 to 18 use this family for media quality and transfers, Chapter 23 for background work, and Chapters 35 and 36 for resource budgets and constrained devices.

Permission probes

Deny, grant later, revoke.

A permission is a switch on one of the platform services, and the app’s reaction shows how tightly a feature depends on it. Deny a permission and see whether the feature degrades, blocks or keeps asking. Grant it later from device settings and see whether the app notices without a restart. Revoke it while the app is in use and see whether the app copes.

The moment the app asks is a signal too. An app that asks for everything on first launch and an app that asks at the point of need were designed with different priorities.

Chapter 22 is built on this family. Chapter 24 uses it for location and sensors, and Chapter 23 uses the notification permission to separate what the app does visibly from what it does silently.

Passive observation

Storage used, data used, battery used, app size, notifications received, settings screens, store listing.

Passive observation changes nothing. It reads what the device and the store already record about the app. Storage used shows how much the app keeps. Data used and battery used show how much it does, including when you are not looking. App size hints at what is bundled. Notifications received show what the backend can send to a device with no app open. The app’s own settings screens list the choices the team exposed, such as a download quality or an offline mode, and each exposed choice is a stated design fact. The store listing gives a version history, a category, declared data practices and a description written by the maker.

Passive observation is rarely conclusive, and it is the best source of questions. It appears in almost every Part II chapter. Chapter 11 starts from storage used, and Chapters 34, 35 and 38 depend on it heavily because their concerns leave few other traces.

4.5 Safety rules

A probe must be safe to run on your own account. Four rules keep it that way.

Use your own account. Probe with an account you own, on a device you own. When a probe needs a second account, use another one of yours or one whose owner has agreed and knows what you will do.

Restore device settings afterwards. Several probes change a device-wide setting: airplane mode, the clock, low-power mode, a revoked permission. The setting affects every app on the device, not only the one under test. A wrong clock can sign you out of other apps or make secure connections fail. Restore each setting as soon as the probe ends, and check it. Probes in this book that change a device-wide setting carry a caution line.

Never test on data you cannot afford to lose. Fresh-state probes delete what is on the device. Conflict probes make one edit lose on purpose. Create test content for the purpose: a note called “probe test”, a message to yourself, a draft you do not need. Before a fresh install on your main device, confirm with a second device that your data reaches the backend. Do not run probes inside a flow that moves money, places an order or sends something to another person, unless you intend the real result.

Do not probe other people’s accounts. The method stops at what the app shows a normal user. Do not try to reach data that is not yours, do not guess identifiers, do not send requests the app’s own UI would not send, and do not automate actions at a volume that could burden the service. Content you create while probing should be visible only to you, or to someone who has agreed to take part.

A fifth point is a habit and not a rule: tell the difference between a probe and an attack. Turning off your own network is a probe. Trying to make the backend accept something it should refuse is not part of this method, and it is outside what a service’s terms allow.

4.6 The order to run probes in

Run probes from passive to active, from cheap to expensive, and from reversible to hard to undo. Figure 4.2 shows the order.

Figure 4.2. The order of probes, from cheapest and most reversible to most expensive.

There are three reasons for this order.

First, passive observation must come before anything else because active probes destroy what it reads. A fresh install resets the storage figure. A long test session inflates the data and battery figures. Read the numbers while they still describe your normal use.

Second, cheap probes decide which expensive probes are worth running. If airplane mode shows a feature is online only, every conflict probe for that feature is pointless, because a feature that cannot write offline cannot produce an offline conflict. The first-pass probes in Section 4.8 exist to make these decisions early.

Third, destructive probes end the chance to observe the current state. Run the fresh install after you have learned everything the populated app can show you.

Start the slow probes early, all the same. A spare device left signed in and untouched costs nothing while you do the rest of the teardown. Set it aside on the first day and return to it at the end.

4.7 Running a probe well

A probe is an experiment, and the usual discipline of experiments applies.

Take a baseline. Before you apply the force, do the same steps without it. To judge how an app behaves on a weak connection you need to know how it behaves on a good one. Without a baseline you will attribute ordinary behavior to the probe.

Change one thing. A probe that turns on airplane mode and force-quits the app tests two things, and its result cannot be assigned to either. Some probes combine forces on purpose, such as the test for durable pending mutations in Chapter 12. Those state the combination and run the single-force probes first.

Repeat anything surprising. One run is an anecdote. Apps have bugs, networks have moods, and the operating system makes its own decisions about memory and background time. A result that appears three times out of three is a signal. A result that appears once is a note.

Record the conditions. Write down the app version, the device, the operating system version, the network and the date. Apps change behavior between versions, and sometimes between launches of the same version when remote config changes. A finding without its conditions cannot be checked later, by you or by anyone else.

Probe one feature at a time. Designs belong to features and not to apps. The message list and the profile screen of one app can give opposite results on the same probe. Name the feature in every record.

Write the outcome before the reading. Record what you saw in plain words first, then decide what it means. If you write “has an outbox” in your notes, you have skipped a step. Write “the edit made offline was still visible after a force-quit”, and let the reading follow.

4.8 The first-pass probes

Six probes work on any app. They take about twenty minutes together, apart from the waiting in P4.4, and they need no knowledge of what the app is for. Their purpose is routing: each outcome tells you which Part II chapters will repay your time on this app.

Run them in the order given. P4.1 is passive and must come first. P4.6 removes data and comes last. Choose one feature to run them on, usually the app’s main screen, and record the feature’s name.

New teardownOpen worksheet

P4.1 Footprint survey

Do

1. In device settings, open the list of storage used per app. Note the size of the app itself and the size of the data it has stored.
2. Open the lists of mobile data used and battery used per app, and note the app's figures.
3. Use the app for ten minutes, opening content you have not opened before.
4. Read the storage figure again.

Watch

How large the stored data is next to the app itself, and whether it grew during the ten minutes.

Read

Stored data is negligible and does not growSpeculativeThe app keeps little on the device and fetches what it shows. P4.2 confirms this. Start with Chapters 10 and 13.Stored data is modest and grows as you browseInferredEither a cache filled by visits, or data the app writes as you use it, such as items you create and logs. Growth in the cache figure while you only browse points to the first. Growth in user data while you create points to the second. Chapter 11 covers what a cache holds and when it is discarded.Stored data is already large, far beyond what one session explainsSpeculativeEither a local replica of your account's content or downloaded media. P4.2 and P4.6 separate the two. Read Chapters 11 and 12.

Select what you saw to record it in your worksheet. Select it again to clear it.

Read the figures before any other probe. The mobile data and battery figures have no outcome of their own here, because they only mean something next to how much you use the app. Record them in the worksheet with the period they cover.

P4.2 Launch in airplane mode

Do

1. Use the app online for a minute, then force-quit it.
2. Turn on airplane mode.
3. Open the app.
4. Try to view content you saw earlier, then try one small change, such as a draft or a reaction on your own content.
5. Turn airplane mode off.

Watch

What the first screen shows, whether the app says it is offline, and whether the change is accepted.

Caution. This probe changes a device-wide setting. Turn airplane mode off when you finish.

Read

An error, an endless spinner or an empty screenObservedThe first screen needs the network. Either nothing useful is kept on the device, or the app will not show it without a check. Read Chapters 7 and 13.Earlier content appears, and the change is refusedInferredReads come from a cache or a local store. Writes need the network. Read Chapter 11, then Chapter 12.Earlier content appears, and the change is acceptedInferredAn optimistic update on top of stored content. Chapter 12 settles whether an outbox stands behind it.

Select what you saw to record it in your worksheet. Select it again to clear it.

This is the cold start version of the probe, with the process ended first. An app that is already open in memory can show content it never stored, and that would hide the difference you are looking for.

P4.3 Force-quit and reopen

Do

1. Go at least three screens deep into the app, and scroll partway down a list.
2. Type some text into an input field and do not submit it.
3. Force-quit the app.
4. Open it again from its icon.

Watch

Which screen opens, whether the scroll position is kept, and whether the typed text is still there when you return to the field.

Read

The start screen opens, and the text is goneObservedScreen state and drafts are held in memory only. A cold start builds everything again. Read Chapters 7 and 8.The start screen opens, and the text is still in its fieldInferredDrafts are written to the local store as you type. Navigation is not restored after a quit by the user. Read Chapters 8 and 9.The app reopens where you were, with the textInferredBoth the navigation position and the draft are stored on the device. Read Chapter 8.

Select what you saw to record it in your worksheet. Select it again to clear it.

A force-quit is the user’s own act, and an app may deliberately start over after one. A start screen here does not prove that the app loses its place after process death. P4.4 tests that case.

P4.4 Long background and return

Do

1. Note the screen you are on, your scroll position and the newest item you can see.
2. Leave the app without quitting it. Use several demanding apps, such as the camera and a game.
3. Wait at least an hour. Overnight is better.
4. Return to the app from its icon.

Watch

Whether a launch screen appears, which screen you land on, and whether the content changes without you asking.

Read

The same screen and the same content, and nothing changesInferredThe process survived, and the app does not refresh on return. The probe has not yet tested process death. Repeat it with a longer wait.The same screen, and the content refreshes on its ownInferredThe app pulls when it returns to the foreground, or treats its data as stale after a set time. Read Chapters 9 and 11.A launch screen, then the start screenObservedProcess death occurred and screen state was not restored. Read Chapter 8.A launch screen, then the screen you leftInferredProcess death occurred and the app restored its screen state from storage. Read Chapter 8.

Select what you saw to record it in your worksheet. Select it again to clear it.

You cannot force the operating system to end a process, only make it likely. Using demanding apps in between raises the odds. If the process survives every attempt, record that and move on.

P4.5 Second device

Do

1. Sign in to the same account on a second device, or on the web version if there is one.
2. Put both in the foreground, side by side, on the same screen.
3. Make a small change on device A, using content only you can see.
4. Watch device B for 30 seconds without touching it.
5. If nothing changes, leave the app on B and return to it. If still nothing changes, refresh by gesture.

Watch

Whether the change reaches B, how long it takes, and what you had to do to make it appear.

Read

B updates within a few seconds, untouchedInferredThe backend can reach a client in the foreground, by a persistent connection or by frequent polling. Read Chapter 14.B updates only after you return to it or refreshInferredThe client pulls on a trigger and is not told about changes while it is open. Read Chapters 12 and 15.B never shows the changeInferredThis data lives on the device that made it. The feature has no backend store, or does not sync it.Signing in on B signed A outObservedThe backend allows one session per account. Multi-device probes need a second account instead. Chapter 20 covers the session design.

Select what you saw to record it in your worksheet. Select it again to clear it.

Time the delay with a clock if B updates on its own. A delay that is always short suggests a persistent connection. A delay that varies from nothing up to some fixed limit suggests polling at that interval. Chapter 14 takes this further.

P4.6 Fresh install

Do

1. Install the app on a device that has never had it.
2. Open it and note what you can use before signing in.
3. Sign in.
4. Watch what appears, and in what order, without touching anything.

Watch

Whether your existing content comes back, whether a restore step is needed, and how long the app takes to become usable.

Caution. Removing an app deletes its data on that device. Prefer a device that has never had the app. Do not remove the app from your main device unless P4.5 showed that your data reaches a second device.

Read

Your content returns after sign-in, with no extra stepInferredThe backend store holds your content and is the source of truth. Chapter 12 covers the order in which it loads.Your content returns only after a restore from a backupInferredThe device holds the content, and the backend does not keep it in a form it can serve. A backup stands in for a backend store.The app starts empty, or has no sign-in at allInferredThe content exists only on the device that created it. See Chapter 54.

Select what you saw to record it in your worksheet. Select it again to clear it.

Note what the app offers before sign-in as well. Content that is usable without an account tells you which backend reads need no session.

Reading the six together

No single outcome places an app. The combination does.

Combination	The app is probably	Go first to
Blocked offline, start screen after quit, B updates on refresh	A thin client over backend reads	Chapter 10 and Chapter 13
Content offline and writes refused, B updates on refresh	A cached reader	Chapter 11 and Chapter 15
Content and writes offline, large stored data, content returns on a new device	A local replica with a sync engine	Chapter 12
B updates untouched within seconds	Built around real-time delivery	Chapter 14
No propagation, starts empty on a new device	An app with no backend store for this feature	Chapter 54

Treat the table as a starting hypothesis. Each row is Speculative until the probes of the chapter it points to have run.

4.9 Signal-to-design table

These signals come from the first-pass probes and from passive observation. Each one points to a design and to the chapter that tests it properly.

Signal	Design it points to	Confidence
Stored data far exceeds the size of the app itself	Content is kept on the device as a cache, a local replica or downloads	Speculative
Stored data shrinks on its own when the device runs low on storage	A cache that the app or the operating system may discard	Inferred
Mobile data used is high for the time you spend in the app	Media, prefetching or background workers	Speculative
Battery is used on days you did not open the app	Background workers or continued use of platform services	Inferred
A notification arrives after a force-quit	Async delivery through the platform's push channel	Inferred
The start screen fills at once in airplane mode	The UI reads from a cache or a local store	Inferred
A typed draft survives a force-quit	Drafts are written to the local store as you type	Inferred
The app returns to its start screen after a long time in the background	Screen state is not restored after process death	Inferred
A second device updates while you watch it	A real-time delivery path to clients in the foreground	Inferred
Content appears on a new device after sign-in	The backend store is the source of truth	Inferred
Signing in on a second device signs the first one out	One session per account	Observed
Behavior changes between two launches with no update installed	Remote config or server-driven UI	Speculative

Search the signals from every chapter in the signal lookup.

4.10 Worksheet entries

These are the fields this chapter adds to the teardown worksheet. The first-pass probes fill the first six. Complete the rest by hand.

New teardownOpen worksheet

Mobile data and battery figures, with the period they coverTest conditions: devices, network, app versionChapters to read next

4.11 In practice

At the start of a teardown, do four things.

1. Read the passive figures and the store listing before you touch anything else, and write them down.
2. Set a spare device aside, signed in, for the long-absence probes you may want later.
3. Run P4.2 to P4.5 on the app’s main feature, selecting the outcome you saw each time.
4. Use the combination of outcomes to choose the three or four Part II chapters that matter most for this app, and write them in the “Chapters to read next” field.

Inside a Part II chapter, pick probes by what they discriminate. Each chapter says which probes to run on every feature and which to run only after a given earlier outcome. Follow that order. It is the cheap-first rule applied to one concern.

When a probe from the book does not fit the app in front of you, write your own in the same format. State the steps and the starting condition. State what to watch. List the outcomes before you run it, and check that at least two candidate designs predict different ones. If they do not, you have a demonstration and not a probe.

4.12 Summary

• A probe is a repeatable experiment a normal user can run to expose one aspect of an app’s design. It applies a force that normal use does not.
• A good probe is cheap, safe, repeatable and discriminating. Discriminating means that candidate designs predict different outcomes for it.
• Every probe is written as Do, Watch, Read. The outcome is a fact about behavior, and the reading is a design claim with a confidence word.
• There are eight probe families: network, lifecycle, time, multi-device, fresh-state, resource, permission, and passive observation. Each acts on a different part of the universal app model.
• Probe only your own account, restore device settings afterwards, never test on data you cannot afford to lose, and do not probe other people’s accounts.
• Run passive observation first, then cheap and reversible probes, then probes that remove data. Start probes that need calendar time early and read them last.
• Take a baseline, change one thing, repeat surprises, record the conditions, and probe one feature at a time.
• Six first-pass probes work on any app and route you to the Part II chapters that matter for it.