Every question the Testing pages answer

The 14 testing pages answer 69 named questions between them. Every one is listed below, and every link lands on the paragraph that answers it rather than the top of the page.

How to Test a HealthKit Integration

Is HKHealthStore final, and does that stop you injecting a fake?No on both counts. Apple declares it as class HKHealthStore with no final keyword, and documents no prohibition on subclassing it, so the…
Should the protocol seam return HKQuantitySample or your own sample type?Your own type, and the reason is concrete rather than stylistic. Apple's public initializers for HKQuantitySample take type, quantity,…
Can you seed ordinary step samples into the iOS Simulator for a test run?Not with any Apple tool. Apple's only documented simulator sample data is clinical health records, supplied as three sample accounts you…
Which HealthKit paths are worth an end-to-end XCUITest?Our recommendation is three, and a hard stop after them. First run with authorization granted, where the app moves from empty state to…
Why must an empty HealthKit read be a first-class test case rather than an error?Because Apple hides read-authorization state deliberately, so a denied read and a type with no samples both come back as an empty array…

Test Data for Health Connect: Fakes, the Toolbox, and the Generators You Write

Does FakeHealthConnectClient compute aggregates from the records I insert?No. Google documents that aggregation calls have no fake implementations and use stubs you program instead, accessed through the overrides…
Which version of the Health Connect testing library is current, and is it still being developed?As of 2026-07-30 the latest published version is 1.0.0-alpha03, released April 9 2025. The AndroidX release notes list exactly three…
Can I drive the Health Connect Toolbox from a CI script to seed data?Not in any documented way. The only scriptable step is installing the APK with adb after extracting it from the downloaded ZIP. Every other…
Do I have to complete Google's recommended Health Connect test cases before shipping?Google frames them as recommendations: the page says you are responsible for testing your applications and that Health Connect recommends a…
How do I test what happens when a user revokes a Health Connect permission mid-sync?Construct FakePermissionController with grantAll set to false, pass it into FakeHealthConnectClient as the permissionController, and assert…

Mock Wearable Data That Is Ugly Enough to Find Bugs

Should a fixture generator draw a new random seed on every CI run?No. Pin the seeds in source and add new ones rather than rotating them. A generator that reseeds itself each run will eventually produce a…
Why won't you publish typical ranges or distributions for synthetic wearable data?Because we have not measured them, and an invented range does not stay on the page it was invented on. It becomes the tolerance in…
What does a two-minute device clock skew break that a clean fixture never will?Three things. Any duplicate matcher that treats an identical start instant and value as the same sample stops matching, so the same walk is…
Is it safe to keep scrubbed real user payloads as test fixtures?Treat them as production data, because that is what they are. Fixtures get committed to repositories, echoed into CI logs, and pasted into…
Where do new fixture cases come from once the obvious ones are written?From incidents, not from imagination. Every case worth having exists because something like it happened to someone, which means you cannot…

Testing Background Sync: Three Tests, and Only One Is Real

Why does HealthKit background delivery never fire during a CI run?Because hosted CI runners give you simulators, and Apple states on both the enableBackgroundDelivery method page and the HKObserverQuery…
What should a unit test for an HKObserverQuery handler actually assert?Five things, all of which are decisions rather than side effects. A failed read advances no cursor and marks a retry. The same delivery…
Does WorkManagerTestInitHelper prove my background job will run on a real phone?No, and it is important to say why. Google documents that Doze does not let JobScheduler run and that WorkManager uses JobScheduler…
How often should we do the manual on-device background delivery run?Our recommendation is to trigger it on events rather than on a calendar: a new iOS major version, any change to the background-delivery…
Why can I not assert that a background wake arrived within a time limit?Because there is no documented number to assert against. Apple documents the frequency parameter as a maximum, at most one wake per period,…

Testing an OAuth Integration: The Token Lifecycle, Not the Login Screen

Should my CI pipeline ever call a real provider's token endpoint?No, and the reason is stronger than flakiness. Under refresh token rotation a refresh is destructive and non-idempotent, so a CI run that…
How do I keep recorded OAuth fixtures from leaking a real user's health data?Scrub two classes of thing, not one. Credentials are the obvious class: client secret, access and refresh tokens, Authorization headers,…
Why would two workers refreshing one user cost that person a full re-authorization?Because under rotation a refresh is destructive and non-idempotent. Both workers notice the same expired access token and both post the…
How do I test refresh token rotation when my fake always hands back the same token?Make the fake hostile. Script the token endpoint as an ordered sequence in which the first call returns a different refresh token and the…
What should the nightly rollup write for a day when the provider grant was dead?A no-data state, never a zero, and this is the assertion most token-lifecycle suites never write. A dead grant in a checkout flow shows…

Provider Sandboxes: Build the Fake, Keep One Real Account

Should I wait for a provider sandbox before writing integration tests?No, and the answer does not depend on whether one exists. Even a perfect hosted sandbox is the wrong target for continuous integration: its…
Why did a test that passed for three weeks against a real provider account suddenly go red?Almost certainly because the data moved, not because your code did. Wearable days get retro-edited when a device syncs late, sleep gets…
How often should the contract test against the real provider run?On a named trigger rather than a cadence you will forget: before a release, and whenever the provider announces an API version change.…
Can one real staging account replace the fake?No, and the ceiling is low. One account is one body, one timezone, one device mix and one history depth. It cannot give you a user whose…
What should a fake return that a provider's own sandbox probably would not?Ugly days and rude faults. Two overlapping sleep segments from two devices, a day retro-edited between two reads, a night crossing a DST…

Testing Webhooks Locally: Replay Signed Payloads, Not Just the Handshake

Do I need a tunnel like ngrok to test a webhook handler?Only for the subscription handshake and a first manual smoke test, because the provider has to reach a public HTTPS URL. Every test that…
How do I prove a replayed delivery did not double a user's step count?POST byte-identical bodies with the same delivery id twice, then assert two things that have nothing to do with the response: exactly one…
What should a webhook test assert on when the payload is only a change pointer?On the job that got enqueued. Most fitness notifications carry a subject, a metric and a window rather than values, so the handler resolves…
How do I test signing key rotation before the rotation happens?Standard Webhooks makes the signature header a space-delimited list of signatures precisely to allow zero-downtime secret rotation, so sign…
Why does my signature check pass in tests but fail against the real provider?Almost always because the test signs the parsed object rather than the bytes. The Standard Webhooks spec calls this out as a very common…

Testing 429 Rate-Limit and Outage Handling in a Health Backfill

How do I test 429 handling when the provider never sends a Retry-After header?RFC 6585 section 4 defines the 429 status code and says a response MAY include Retry-After, so a compliant provider is allowed to omit it.…
Should fault injection live in the retry helper's unit test or in the fake provider?In the fake. A unit test that hands a synthetic 429 to your backoff function asserts that your code slept the number your code computed,…
How do I prove a backfill resumes from its checkpoint instead of restarting?Have the fake log every request it serves, tagged with a run id. Run the job until the storm kills it, record which date windows committed,…
Why is a slow provider response worse than an outage in a backfill test?An outage fails fast, so the window re-queues and the worker is free almost immediately. A response that arrives just inside your client…
What should a test assert about recovery after a rate-limit storm?That the resumed traffic is spread out. Rate-limit every user in the fixture, restore success responses at a known instant, bucket the…

Testing Camera Features When You Have No Camera

Why is adding a frame-source seam later a rewrite rather than a refactor?Because a capture session silently supplies five things besides pixels, and your analysis code will have come to depend on all of them.…
Does VNVideoProcessor let me run body pose detection over a video file instead of a camera?Yes, and it is Apple's documented path for it. Apple describes VNVideoProcessor as an object that performs offline analysis of video…
Can the Android emulator play a recorded workout video into my camera pipeline?No. Google documents importing a still image in PNG or JPEG format into the virtual scene through the Extended controls window, the Camera…
Are third-party tools that stream a Mac camera into the iOS Simulator worth depending on?They exist and they are all unofficial. As of 30 July 2026 we could read one of them directly: a macOS menu-bar app from Software Mansion…
What breaks on real hardware after a green video-fixture run?Five things, and they are the reason the fixture suite is a floor rather than a ceiling. Auto-exposure hunting in a dim room produces…

Testing Pose Estimation Accuracy with a Regression Corpus

Why would a mean keypoint error improve while the rep counter gets worse?Because the mean is averaging the joint you depend on against fifteen you never read. Your rep logic probably reads four joints. Apple's…
What pixel tolerance should I set for each keypoint?One you measured, in a scale-normalized unit, and nobody can give it to you. The tolerance is not a computer-vision constant, it is a…
How should the suite score a keypoint that was genuinely occluded?By scoring confidence rather than position, otherwise you will tune the suite until it prefers a model that guesses. Ground truth should…
Should a pose accuracy regression fail the build, or just get reported?Both, split by clip category, and the split is a product decision rather than a technical one. The real question is whether it is…
How do I make a pose SDK upgrade show up as a visible diff?Pin the model version in both production code and the test, then commit the numbers. On iOS the body-pose request carries an explicit…

How to Test a Rep Counting Algorithm

Why is comparing the final rep count not enough?Because errors cancel. A clip where the counter misses one rep and double-counts another produces the correct total and a passing test,…
How many clips does a rep-counting corpus need?We are deliberately not giving you a number, and you should distrust anyone who does without saying how they measured it. Coverage of…
Who should label the ground truth for a rep corpus?The person who performed the set should do the first pass, immediately, while they still remember it. Ground truth is what the human did,…
Is over-counting or under-counting the worse rep-counting failure?They feel completely different to a user and you should refuse to collapse them into a single F-score, because that lets a gain in one…

CI for an App That Needs a Real Device

Can I run my pose regression corpus on a hosted CI runner instead of a phone?Partly, and the split is worth making explicit. The logic that consumes keypoints — your rep state machine, your form thresholds, your…
Does Firebase Test Lab do camera image injection?We cannot tell you, and we would rather say so than guess. Our research environment could not reach firebase.google.com or cloud.google.com…
How many devices belong in a fitness app's test matrix?Fewer than you want, chosen by criteria rather than by model name. The criteria we would use: the oldest chipset your app claims to…
Why does my rep counter pass in CI and miss reps late in a long session on a mid-tier Android?Almost certainly thermal throttling changing your frame interval. A camera pose pipeline pins the camera, the ISP, the accelerator and the…
Which health sync tests actually need physical hardware?Far fewer than the camera tests. Token rotation races, webhook replays and duplicates, timezone day boundaries, deduplication and…

Testing Offline Sync and Conflict Resolution

Why would adding content-hash deduplication to a workout log delete real sets?Because two identical sets are normal. Five reps at sixty kilos, logged twice ninety seconds apart, is a person doing their second set —…
What should an offline workout sync test assert on?Three properties, all stated as things a user would see. Conservation: every set, weight correction and deletion the person performed is…
What breaks when someone is offline across an app update?The migration you did not notice was a migration. A shape change — bare kilograms becoming a value-and-unit pair — is caught by any…
Where does a set logged at 23:58 belong after it syncs at 00:40?On the previous civil day, and its rollup has to be recomputed to say so. The offline flush that crosses midnight is the common way this…
Do device samples need the same conflict tests as user entries?No, and the practical rule is to never share an assertion helper between them. A no-duplicates helper written for a step stream will one…

Testing That a User's Health Data Is Actually Deleted

Why does asserting a 200 from a token revocation endpoint prove nothing?Because RFC 7009 requires it. The specification says the authorization server responds with HTTP status code 200 if the token has been…
How do I make a deletion test fail when someone adds a new store and forgets to purge it?Do not hand-write the store list in the test. Keep a registry file in the repo with one entry per store, then write a test that discovers…
What is the right assertion for a store whose delete is asynchronous, like a search index?Poll a read until a bounded timeout, and assert on the read result rather than on the delete call's status. Index deletions are frequently…
What does a restore drill have to check before backup erasure is more than a claim?That a previously deleted user is still absent after the restore completes. Nobody surgically edits a snapshot, so the honest design is a…
What can only a real provider account prove about a revoked wearable grant?That the grant is genuinely dead upstream, which is the single thing a fake cannot tell you, because a fake returns whatever you programmed…

Back to the Testing hub, or see every question this site answers.