TextToSpeech workout cues on Android: initialization, focus, queue and completion

Android's built-in TextToSpeech is the quickest way to make a workout app talk, and it is asynchronous in more places than its method names suggest. The constructor returns before the engine is ready. speak() returns before anything is spoken. Completion arrives on a listener, possibly on another thread. And the reference says nothing about audio focus, so ducking the user's music is your job.

This page walks the lifecycle for coaching cues. The focus request itself is on the MAY_DUCK page, and the background restriction for apps targeting Android 15 is on the AUDIOFOCUS_REQUEST_FAILED page.

Initialization, and the callback that can come early#

Google's TextToSpeech reference: "A TextToSpeech instance can only be used to synthesize text once it has completed its initialization. Implement the TextToSpeech.OnInitListener to be notified of the completion of the initialization."

The constructor's parameter documentation adds the detail that causes a confusing crash: "In a case of a failure the listener may be called immediately, before TextToSpeech instance is fully constructed." If your onInit reads the property you are about to assign the instance to, a failure path can hit it while it is still null. Treat init status as data and keep onInit from touching the instance until construction has returned.

Create the engine when the workout screen opens, not when the first cue is due, so initialization is finished before anything needs saying.

Android 11: the queries element#

Apps targeting Android 11 "that use text-to-speech should declare TextToSpeech.Engine.INTENT_ACTION_TTS_SERVICE in the queries elements of their manifest," and Google gives the block:

<queries>
  <intent>
    <action android:name="android.intent.action.TTS_SERVICE" />
  </intent>
</queries>

Speaking a cue#

tts.setAudioAttributes(cueAttributes)          // same attributes as the focus request

fun speakCue(text: String, id: String) {
    if (audioManager.requestAudioFocus(cueFocus) != AudioManager.AUDIOFOCUS_REQUEST_GRANTED) return
    tts.speak(text, TextToSpeech.QUEUE_FLUSH, null, id)
}

Google documents setAudioAttributes as setting "the audio attributes to be used when speaking text or playing back a file," returning ERROR or SUCCESS. Use the same attributes your focus request carries, as Google's focus guide recommends for any player.

speak(CharSequence, int, Bundle, String) "Speaks the text using the specified queuing strategy and speech parameters," and Google is explicit that it "is asynchronous, i.e. the method just adds the request to the queue of TTS requests and then returns. The synthesis might not have finished (or even started!) at the time when this method returns." The text must be "No longer than getMaxSpeechInputLength() characters," and the return value is ERROR or SUCCESS "of queuing the speak operation," not of speaking it.

QUEUE_FLUSH or QUEUE_ADD#

Our judgement: default to QUEUE_FLUSH for workout cues. A queue that holds a cue past its moment delivers it late, and a late cue is wrong. Google notes the flush only affects your own app's entries: "Entries in the queue from other callees are not discarded."

Knowing when it finished, so the music can come back#

Completion arrives through UtteranceProgressListener, keyed by the utterance id you passed to speak. Google's speak documentation: "In order to reliably detect errors during synthesis, we recommend setting an utterance progress listener." The listener's class note matters for threading: "The callbacks specified in this method can be called from multiple threads."

tts.setOnUtteranceProgressListener(object : UtteranceProgressListener() {
    override fun onStart(utteranceId: String) {}
    override fun onDone(utteranceId: String) = release(utteranceId)
    override fun onStop(utteranceId: String, interrupted: Boolean) = release(utteranceId)
    @Deprecated("Deprecated in Java")
    override fun onError(utteranceId: String) = release(utteranceId)
    override fun onError(utteranceId: String, errorCode: Int) = release(utteranceId)
})

onDone is the right release point because of Google's wording: by then "All audio will have been played back." Do not use isSpeaking() for this. Google warns that "a speech item is considered complete once it's audio data has been sent to the audio mixer," and "There might be a finite lag between this point, and when the audio hardware completes playback."

Languages and voices that are not there#

setLanguage(Locale) returns a support code, and Google notes the engine "will try to use the closest match to the specified language as represented by the Locale, but there is no guarantee that the exact same Locale will be used." Two return values mean you will get no speech: LANG_MISSING_DATA, "the language data is missing," and LANG_NOT_SUPPORTED. Separately, an utterance can fail with ERROR_NOT_INSTALLED_YET, "a failure caused by an unfinished download of the voice data." Our judgement: check the language when the workout screen opens, and if it is missing, tell the user before the workout starts that cues will be tones and haptics, rather than discovering it at the first cue.

Rate, pauses and shutdown#

• setSpeechRate(float): "1.0 is the normal speech rate, lower values slow down the speech (0.5 is half the normal speech rate), greater values accelerate it (2.0 is twice the normal speech rate)." Our judgement: expose it as a user setting rather than choosing a rate for everyone.
• playSilentUtterance(durationInMs, queueMode, utteranceId) "Plays silence for the specified amount of time," which is how to put a deliberate gap between two halves of a cue without a timer.
• stop() "Interrupts the current utterance (whether played or rendered to file) and discards other utterances in the queue." Call it when the user pauses.
• shutdown() "Releases the resources used by the TextToSpeech engine," and Google suggests calling it in onDestroy(). For a workout app, call it when the workout ends, from whatever owns the engine, which is often the foreground service rather than an activity.

Monday morning#

Create the engine when the workout screen opens and confirm onInit cannot touch a half-built instance. Add the TTS_SERVICE queries element. Set the cue attributes on the engine and match them in the focus request. Speak with QUEUE_FLUSH and release focus in onDone, onStop and onError. Check the language before the workout starts. Then run a workout with music playing and listen for the music coming back after every cue.