playback or ambient: choosing the AVAudioSession category for workout cues

The first audio bug most workout apps ship is not in their own code. It is in the session they never configured. The cue works on a desk with the screen on, then goes silent on the first run with the phone in an armband, or it plays and the user's music stops dead. Both are the documented behaviour of the category the app is using, usually the one it got by default.

This page is about choosing that category and mode. Lowering the user's music while a cue plays is the next page, duckOthers and notifyOthersOnDeactivation; keeping the app allowed to play after the screen locks is UIBackgroundModes audio.

The default you are already shipping#

Apple's AVAudioSession overview lists what every iOS app gets before it touches the session. It "supports audio playback, but disallows audio recording." When the app plays audio, "it silences any other background audio." In iOS, "setting the Ring/Silent switch to silent mode silences any audio the app is playing," and "locking a device silences the app's audio."

Read that list as a workout app. The user starts their music, starts your workout, locks the phone and puts it away. Under the default session, your first cue silences their music, and every cue after the lock is silenced. Apple's verdict on the default is polite: it "generally doesn't provide the audio behavior a media app needs."

The default has a name. Apple documents soloAmbient as "The default audio session category," whose audio "is silenced by screen locking and by the Silent switch," and which "implies that your app's audio is nonmixable—activating your session will interrupt any other audio sessions which are also nonmixable."

The four categories a workout app considers#

Each cell is from Apple's category pages. ambient: "audio from other apps mixes with your audio. Screen locking and the Silent switch (on iPhone, the Ring/Silent switch) silence your audio." playback: "your app audio continues with the Silent switch set to silent or when the screen locks," and "To continue playing audio when your app transitions to the background (for example, when the screen locks), add the audio value to the UIBackgroundModes key in your information property list file." playAndRecord adds that "The user must grant permission for audio recording," which is a permission prompt no cue-only app should trigger.

Apple's Playing audio guidelines give the same split in design terms: ambient is for when sound "isn't essential, and it doesn't silence other audio"; playback is for when sound "is essential and might mix with other audio," and it "Can play in the background."

Why ambient is tempting and usually wrong#

ambient looks like exactly what a polite workout app wants. It mixes with the user's music without being asked, so there is no ducking to configure and no option to get wrong. For a cue that is genuinely optional, that is a fine choice: a metronome tick while the app is open, a celebratory sound on the summary screen.

The problem is the moment the cue exists for. A rest-over cue matters most when the phone is locked in a pocket, and Apple documents that screen locking silences an ambient session. That is not a bug you can work around inside the category; it is the category's definition. Our judgement: if a cue has to be heard with the screen off, it cannot live in ambient.

Why playback is right and has one trap#

playback keeps going through the Silent switch and the lock, which is what a coaching cue needs. The trap is the sentence that follows on Apple's page: "By default, using this category implies that your app's audio is nonmixable—activating your session will interrupt any other audio sessions which are also nonmixable. To allow mixing for this category, use the mixWithOthers option."

So a workout app that sets .playback and nothing else has traded one bug for another: instead of going silent when the phone locks, it now stops the user's music every time it speaks. The fix is a category option. Apple documents three that matter here, and two of them imply the third:

• mixWithOthers: "If you set this option, your app mixes its audio with audio playing in background apps, such as the Music app." Apple notes that setting duckOthers or interruptSpokenAudioAndMixWithOthers "also enables this option."
• duckOthers lowers the other audio while yours plays.
• interruptSpokenAudioAndMixWithOthers pauses spoken audio, such as a podcast, instead of talking over it.

The last two are the subject of the duckOthers page. Apple's guidelines put the principle plainly: "don't make people stop listening to music from another app if you don't need to."

The mode: voicePrompt, not spokenAudio#

The mode refines the category. Apple documents default as usable "with every audio session category," and two spoken-audio modes that are easy to confuse:

• voicePrompt: "A mode that indicates that your app plays audio using text-to-speech." Apple says setting it "allows for different routing behaviors when your app connects to certain audio devices, such as CarPlay," and that apps of this type "also configure their sessions to use the duckOthers and interruptSpokenAudioAndMixWithOthers options."
• spokenAudio: "A mode used for continuous spoken audio to pause the audio when another app plays a short audio prompt," appropriate for "podcasts or audio books."

spokenAudio is the mode of the app you are interrupting, not yours. A coaching app that speaks short cues over the user's music is the "short audio prompt" in that sentence. If your app also plays long guided sessions, those are the continuous spoken audio, and the mode can differ between the two activities.

Apple scopes voicePrompt to text-to-speech. For cues recorded as audio files, Apple's page does not say whether the mode applies; our judgement is to use default for recorded clips and tones, and voicePrompt when the cue is synthesized speech, as on the AVSpeechSynthesizer page.

let session = AVAudioSession.sharedInstance()
try session.setCategory(.playback,
                        mode: .voicePrompt,
                        options: [.duckOthers, .interruptSpokenAudioAndMixWithOthers])

Apple's setCategory(_:mode:options:) page says you "Typically" set the category and mode "before activating the session," and that setting them while active "results in an immediate change." Configure once, at workout start, and do not activate yet: Apple's overview notes it is "generally preferable to defer this call until your app begins audio playback," which "ensures that you won't prematurely interrupt any other background audio that may be in progress."

Knowing whether the user already has music on#

Many workout apps ship their own music bed under the coaching. When the user starts their own music, the bed should get out of the way and the cues should stay. Apple documents exactly this split for games.

secondaryAudioShouldBeSilencedHint is "A Boolean value that indicates whether another app, with a nonmixable audio session, is playing audio," to be used "as a hint to silence audio that's secondary to the functionality of the app." Apple's example: a game "can use this property to mute the soundtrack while leaving sound effects unmuted." Swap soundtrack for music bed and sound effects for cues.

Apple steers you to it over isOtherAudioPlaying, which "returns true if any other audio is playing, including audio from an app using the ambient category." For change events, silenceSecondaryAudioHintNotification is posted "when the primary audio from other apps starts and stops," but only "to registered listeners who are currently in the foreground and have an active audio session," so re-read the hint when the app returns to the foreground rather than relying on the notification alone.

The Silent switch question#

Apple's guidelines describe what silent mode means to the person who flips it: "When a device is in silent mode, it plays only the audio that people explicitly initiate, like media playback, alarms, and audio/video messaging." A playback session plays through the switch. Whether a workout cue counts as audio the person explicitly initiated is a product decision Apple's text does not make for you. Our judgement: starting a workout with voice cues switched on is an explicit request, and a setting that turns cues off, paired with haptics as the fallback channel, respects the person who wants silence.

Monday morning#

Find where your app first touches AVAudioSession. If the answer is nowhere, you are on soloAmbient and every symptom above is expected. Set .playback with a mixing option at workout start, choose voicePrompt for synthesized speech and default for clips, defer activation until the first cue, and add the audio background mode. Then lock the phone with music playing and listen to one full interval.