Text to speech

`SpeechRecognition` web standard#

webkitSpeechRecognition / SpeechRecognition — speech to text

Example

The standard Web Speech API for dictation: the user talks, the page gets text. Chrome on Android has it and Android's WebView does not, so the app provides it over Android's own speech recogniser. Code written for Chrome works unchanged.

Returns: start() begins listening; result events carry event.results[i][0].transcript (and .confidence, .isFinal); error events carry event.error (not-allowed, no-speech, audio-capture, network, aborted, language-not-supported, service-not-allowed); end always comes last. Needs: the microphone. The build turns it on by itself when your code mentions SpeechRecognition, and Android asks the user the first time start() runs.

<button id="mic">Dictate</button>
<textarea id="note"></textarea>
<script>
const Rec = window.SpeechRecognition || window.webkitSpeechRecognition;
const mic = document.getElementById('mic');
if (!Rec) {
  mic.hidden = true;                       // no speech recogniser on this phone
} else {
  mic.addEventListener('click', () => {
    const rec = new Rec();
    rec.lang = 'en-IN';                    // leave empty for the page / phone language
    rec.interimResults = true;             // live text while the user speaks
    rec.onresult = (e) => {
      let text = '';
      for (const r of e.results) text += r[0].transcript;
      document.getElementById('note').value = text;
    };
    rec.onerror = (e) => {
      if (e.error === 'not-allowed') alert('Allow the microphone to dictate.');
    };
    rec.onend = () => { mic.disabled = false; };
    mic.disabled = true;
    rec.start();
  });
}
</script>

Continuous dictation that stops on request and reports each error:

const Rec = window.SpeechRecognition || window.webkitSpeechRecognition;
let rec = null;
function toggleDictation() {
  if (!Rec) return showMessage('Voice typing is not available on this phone.');
  if (rec) { rec.stop(); return; }                   // stop() delivers what was heard
  rec = new Rec();
  rec.continuous = true;
  rec.onresult = (e) => {
    for (let i = e.resultIndex; i < e.results.length; i++) {
      if (e.results[i].isFinal) appendToNote(e.results[i][0].transcript);
    }
  };
  rec.onerror = (e) => {
    const msg = { 'not-allowed': 'Allow the microphone first.', 'no-speech': 'Nothing was heard.',
                  'audio-capture': 'No microphone found.', 'network': 'Speech service unreachable.' }[e.error];
    if (msg) showMessage(msg);
  };
  rec.onend = () => { rec = null; };
  rec.start();
}

Say a command and act on it, in the phone's language:

const Rec = window.SpeechRecognition || window.webkitSpeechRecognition;
function listenForCommand() {
  if (!Rec) return;
  const rec = new Rec();
  rec.lang = navigator.language;
  rec.onresult = (e) => {
    const said = e.results[0][0].transcript.toLowerCase();
    if (said.includes('next')) goNext();
    else if (said.includes('back')) goBack();
    else showMessage('Heard: ' + said);
  };
  rec.onerror = (e) => {
    if (e.error === 'language-not-supported') showMessage('This language is not supported here.');
  };
  rec.start();
}

Notes: continuous = true keeps listening across sentences until you call stop(); after three silent rounds it ends with no-speech, so a forgotten recogniser does not listen forever. stop() delivers what was heard; abort() discards it. Recognition stops when the app goes to the background, as it does when a Chrome tab is hidden. Only pages that belong to the app (its own files, or the creator's website) can use the microphone this way; any other site gets not-allowed. The API is missing when the phone has no speech recognition service, so check for it as above. Internal transports: __srStart, __srStop, __srAbort, __srAvailable; results arrive as appmint:sr. AI-built apps use speech.start() from @/lib/appmintNative.

`speechSynthesis` web standard#

window.speechSynthesis + SpeechSynthesisUtterance

Example

Reads text aloud with the phone's own text-to-speech engine, using the normal Web Speech API.

Returns: speechSynthesis.speak(u) returns nothing; the utterance's start / boundary / end / error / pause / resume events (as onstart … properties or with addEventListener) tell you what happened. getVoices() returns an array of { voiceURI, name, lang, localService, default } synchronously. Needs: nothing. Android WebView has no working speech API of its own; the app installs this one at document start in every frame.

Speak a sentence and know when it is done:

function say(text) {
  if (!('speechSynthesis' in window)) { alert('Speech is not available here'); return; }
  var u = new SpeechSynthesisUtterance(text);
  u.lang = 'en-US';   // BCP-47 tag: 'ta-IN', 'hi-IN', 'fr-FR' ...
  u.rate = 1.2;       // 0.1 .. 4 (1 = normal)
  u.pitch = 1;        // 0.1 .. 4 (1 = normal)
  u.onstart = function () { document.body.classList.add('speaking'); };
  u.onend = function () { document.body.classList.remove('speaking'); };
  u.onerror = function (e) {
    document.body.classList.remove('speaking');
    console.log('Speech failed:', e.error);   // 'synthesis-failed'
  };
  speechSynthesis.speak(u);   // queues after anything already speaking
}

document.getElementById('stop').onclick = function () {
  if ('speechSynthesis' in window) speechSynthesis.cancel();   // stops now, clears the queue
};

Voices arrive a moment after start-up. Read them again when voiceschanged fires:

var voices = [];
function loadVoices() {
  voices = speechSynthesis.getVoices();   // [] until the engine is ready
  var tamil = voices.filter(function (v) { return v.lang.indexOf('ta') === 0; });
  document.getElementById('tamil-btn').hidden = tamil.length === 0;
}
if ('speechSynthesis' in window) {
  loadVoices();
  speechSynthesis.addEventListener('voiceschanged', loadVoices);
}

function sayWithVoice(text, voice) {
  var u = new SpeechSynthesisUtterance(text);
  u.voice = voice;           // a voice from getVoices(): that exact voice is used
  u.volume = 0.8;            // 0 .. 1
  speechSynthesis.speak(u);
}

An article reader with one Pause / Resume button that highlights the paragraph being read:

var toggle = document.getElementById('pause-toggle');

toggle.onclick = function () {
  if (!('speechSynthesis' in window) || !speechSynthesis.speaking) return;
  if (speechSynthesis.paused) speechSynthesis.resume();   // continues from the last word
  else speechSynthesis.pause();
};

function readArticle() {
  if (!('speechSynthesis' in window)) return;
  speechSynthesis.cancel();                               // start over
  document.querySelectorAll('article p').forEach(function (p) {
    var u = new SpeechSynthesisUtterance(p.textContent);
    u.onstart = function () { p.classList.add('reading'); };
    u.onend = u.onerror = function () { p.classList.remove('reading'); };
    u.onpause = function () { toggle.textContent = 'Resume'; };
    u.onresume = function () { toggle.textContent = 'Pause'; };
    speechSynthesis.speak(u);                             // paragraphs queue in order
  });
  toggle.textContent = 'Pause';
}

Notes:

volume (0-1), voice (the exact voice from getVoices()), rate, pitch and lang all reach the phone's engine. With a voice and a lang, the voice wins; with neither, the engine's default voice speaks.
boundary fires before each word with charIndex / charLength (Android 8+; engines that report no word positions send none).
pause() / resume(): Android's engine cannot pause mid-word, so pause() stops it and resume() speaks the rest of the sentence from the last word the engine reported (pause / resume events fire). On an engine that reports no word positions, resume() starts that utterance again from its beginning. speaking stays true while paused; pending is true while queued utterances wait.
cancel() fires error with error: 'interrupted' on the utterance being spoken and 'canceled' on queued ones, as the standard says - do not show those as failures.
If the phone has no voice data for the language, the engine may still speak in its default language. Check getVoices() for the language first.
The raw events behind this are appmint:tts (see ttsSpeak). You do not need them when you use this API.

`__srAbort` bridge#

window.WebToApk.__srAbort(id: String)

SpeechRecognition.abort(): stop at once, discard.

Example

Documented together with WebToApk.__srStart - the example there shows this one too.

Internal transport behind webkitSpeechRecognition / SpeechRecognition (start, stop, abort, and whether a recogniser exists on the phone). Pages use the standard API; never call these directly, because the __ methods may change without notice.

Returns: __srStart(id, options) returns "" when listening starts, or a Web Speech error name (not-allowed, service-not-allowed). Everything after that arrives as appmint:sr events: {id, type} for start/audiostart/speechstart/speechend/audioend/end, {id, type:'result', final, alternatives:[{transcript, confidence}]}, and {id, type:'error', error}. Needs: the microphone (armed from the call shape).

Use the public API:

const rec = new (window.SpeechRecognition || window.webkitSpeechRecognition)();
rec.onresult = (e) => console.log(e.results[0][0].transcript);
rec.start();

Continuous dictation with live text and a language:

const SR = window.SpeechRecognition || window.webkitSpeechRecognition;
if (SR) {
  const rec = new SR();
  rec.lang = 'en-US';
  rec.continuous = true;
  rec.interimResults = true;
  rec.onresult = (e) => {
    const r = e.results[e.results.length - 1];
    note.value = r[0].transcript + (r.isFinal ? '\n' : '');
  };
  micBtn.onclick = () => rec.start();
  stopBtn.onclick = () => rec.stop();
}

Report errors such as a refused microphone:

rec.onerror = (e) => {
  if (e.error === 'not-allowed' || e.error === 'service-not-allowed') showMessage('Allow the microphone to dictate.');
  else showMessage('Speech error: ' + e.error);
};
rec.onend = () => micBtn.classList.remove('live');

Notes: Implemented in the app shell (SpeechRecognitionBridge.kt) over Android's SpeechRecognizer. __srStart is refused for pages that do not belong to the app.

`__srAvailable` bridge#

window.WebToApk.__srAvailable(): Boolean

Whether a speech recognition service is installed (webkitSpeechRecognition exists only then).

Example

Documented together with WebToApk.__srStart - the example there shows this one too.

Internal transport behind webkitSpeechRecognition / SpeechRecognition (start, stop, abort, and whether a recogniser exists on the phone). Pages use the standard API; never call these directly, because the __ methods may change without notice.

Returns: __srStart(id, options) returns "" when listening starts, or a Web Speech error name (not-allowed, service-not-allowed). Everything after that arrives as appmint:sr events: {id, type} for start/audiostart/speechstart/speechend/audioend/end, {id, type:'result', final, alternatives:[{transcript, confidence}]}, and {id, type:'error', error}. Needs: the microphone (armed from the call shape).

Use the public API:

const rec = new (window.SpeechRecognition || window.webkitSpeechRecognition)();
rec.onresult = (e) => console.log(e.results[0][0].transcript);
rec.start();

Continuous dictation with live text and a language:

const SR = window.SpeechRecognition || window.webkitSpeechRecognition;
if (SR) {
  const rec = new SR();
  rec.lang = 'en-US';
  rec.continuous = true;
  rec.interimResults = true;
  rec.onresult = (e) => {
    const r = e.results[e.results.length - 1];
    note.value = r[0].transcript + (r.isFinal ? '\n' : '');
  };
  micBtn.onclick = () => rec.start();
  stopBtn.onclick = () => rec.stop();
}

Report errors such as a refused microphone:

rec.onerror = (e) => {
  if (e.error === 'not-allowed' || e.error === 'service-not-allowed') showMessage('Allow the microphone to dictate.');
  else showMessage('Speech error: ' + e.error);
};
rec.onend = () => micBtn.classList.remove('live');

Notes: Implemented in the app shell (SpeechRecognitionBridge.kt) over Android's SpeechRecognizer. __srStart is refused for pages that do not belong to the app.

`__srStart` bridge#

window.WebToApk.__srStart(id: String, optionsJson: String): String

SpeechRecognition.start(): "" when starting (events arrive as `appmint:sr`), else the Web Speech error name. Origin-gated: what the user says is user data.

Example

Internal transport behind webkitSpeechRecognition / SpeechRecognition (start, stop, abort, and whether a recogniser exists on the phone). Pages use the standard API; never call these directly, because the __ methods may change without notice.

Returns: __srStart(id, options) returns "" when listening starts, or a Web Speech error name (not-allowed, service-not-allowed). Everything after that arrives as appmint:sr events: {id, type} for start/audiostart/speechstart/speechend/audioend/end, {id, type:'result', final, alternatives:[{transcript, confidence}]}, and {id, type:'error', error}. Needs: the microphone (armed from the call shape).

Use the public API:

const rec = new (window.SpeechRecognition || window.webkitSpeechRecognition)();
rec.onresult = (e) => console.log(e.results[0][0].transcript);
rec.start();

Continuous dictation with live text and a language:

const SR = window.SpeechRecognition || window.webkitSpeechRecognition;
if (SR) {
  const rec = new SR();
  rec.lang = 'en-US';
  rec.continuous = true;
  rec.interimResults = true;
  rec.onresult = (e) => {
    const r = e.results[e.results.length - 1];
    note.value = r[0].transcript + (r.isFinal ? '\n' : '');
  };
  micBtn.onclick = () => rec.start();
  stopBtn.onclick = () => rec.stop();
}

Report errors such as a refused microphone:

rec.onerror = (e) => {
  if (e.error === 'not-allowed' || e.error === 'service-not-allowed') showMessage('Allow the microphone to dictate.');
  else showMessage('Speech error: ' + e.error);
};
rec.onend = () => micBtn.classList.remove('live');

Notes: Implemented in the app shell (SpeechRecognitionBridge.kt) over Android's SpeechRecognizer. __srStart is refused for pages that do not belong to the app.

`__srStop` bridge#

window.WebToApk.__srStop(id: String)

SpeechRecognition.stop(): stop listening, deliver what was heard.

Example

Documented together with WebToApk.__srStart - the example there shows this one too.

Internal transport behind webkitSpeechRecognition / SpeechRecognition (start, stop, abort, and whether a recogniser exists on the phone). Pages use the standard API; never call these directly, because the __ methods may change without notice.

Returns: __srStart(id, options) returns "" when listening starts, or a Web Speech error name (not-allowed, service-not-allowed). Everything after that arrives as appmint:sr events: {id, type} for start/audiostart/speechstart/speechend/audioend/end, {id, type:'result', final, alternatives:[{transcript, confidence}]}, and {id, type:'error', error}. Needs: the microphone (armed from the call shape).

Use the public API:

const rec = new (window.SpeechRecognition || window.webkitSpeechRecognition)();
rec.onresult = (e) => console.log(e.results[0][0].transcript);
rec.start();

Continuous dictation with live text and a language:

const SR = window.SpeechRecognition || window.webkitSpeechRecognition;
if (SR) {
  const rec = new SR();
  rec.lang = 'en-US';
  rec.continuous = true;
  rec.interimResults = true;
  rec.onresult = (e) => {
    const r = e.results[e.results.length - 1];
    note.value = r[0].transcript + (r.isFinal ? '\n' : '');
  };
  micBtn.onclick = () => rec.start();
  stopBtn.onclick = () => rec.stop();
}

Report errors such as a refused microphone:

rec.onerror = (e) => {
  if (e.error === 'not-allowed' || e.error === 'service-not-allowed') showMessage('Allow the microphone to dictate.');
  else showMessage('Speech error: ' + e.error);
};
rec.onend = () => micBtn.classList.remove('live');

Notes: Implemented in the app shell (SpeechRecognitionBridge.kt) over Android's SpeechRecognizer. __srStart is refused for pages that do not belong to the app.

`__ttsSpeak` bridge#

window.WebToApk.__ttsSpeak(json: String)

The speechSynthesis polyfill's full speak: `{text, lang, rate, pitch, volume (0-1), voice (a getVoices() name, or ''), id}`. [ttsSpeak] stays for pages that call the bridge directly; it has no volume or voice.

Example

Internal transport behind speechSynthesis.speak(utterance): the full utterance - text, language, rate, pitch, volume and the chosen voice - as one JSON string. Pages use the standard speechSynthesis API; never call this directly.

Returns: nothing. Progress arrives as appmint:tts events, which the speechSynthesis layer turns into the utterance's start / boundary / end / error events. Needs: nothing.

The standard API this carries:

function readAloud(text) {
  if (!('speechSynthesis' in window)) return;
  var voice = speechSynthesis.getVoices().filter(function (v) { return v.lang === 'en-GB'; })[0];
  var u = new SpeechSynthesisUtterance(text);
  if (voice) u.voice = voice;   // that exact voice
  u.volume = 0.6;               // 0 .. 1
  u.addEventListener('boundary', function (e) { highlightWordAt(e.charIndex, e.charLength); });
  u.addEventListener('end', function () { clearHighlight(); });
  speechSynthesis.speak(u);
}

A voice picker the user chooses once; every later utterance carries that exact voice:

var picker = document.getElementById('voice-picker');

function listVoices() {
  var saved = localStorage.getItem('voiceName') || '';
  picker.innerHTML = '';
  speechSynthesis.getVoices().forEach(function (v) {
    picker.add(new Option(v.name + ' (' + v.lang + ')', v.name, false, v.name === saved));
  });
}
if ('speechSynthesis' in window) {
  listVoices();                                   // [] until the engine is ready
  speechSynthesis.onvoiceschanged = listVoices;
}
picker.onchange = function () { localStorage.setItem('voiceName', picker.value); };

function speakWithSavedVoice(text) {
  var name = localStorage.getItem('voiceName');
  var voice = speechSynthesis.getVoices().filter(function (v) { return v.name === name; })[0];
  var u = new SpeechSynthesisUtterance(text);
  if (voice) u.voice = voice;                     // its language is used too when u.lang is empty
  speechSynthesis.speak(u);
}

Workout cues: quiet and quick for countdowns, full volume and higher for the switch:

function cue(text, loud) {
  if (!('speechSynthesis' in window)) return;
  var u = new SpeechSynthesisUtterance(text);
  u.lang = 'en-US';
  u.rate = 1.4;                  // 0.1 .. 4
  u.pitch = loud ? 1.3 : 1;      // 0.1 .. 4
  u.volume = loud ? 1 : 0.4;     // 0 .. 1
  u.onerror = function (e) {
    if (e.error === 'synthesis-failed') console.log('Cue not spoken:', text);
  };
  speechSynthesis.speak(u);      // queues after the previous cue
}

cue('Ten seconds left', false);
cue('Switch sides!', true);

Notes: The JSON is {text, lang, rate, pitch, volume, voice, id} (voice is a getVoices() name or ''). The older ttsSpeak(text, lang, rate, pitch, id) stays for pages that call the bridge directly; it has no volume or voice.

`ttsCancel` bridge#

window.WebToApk.ttsCancel()

Backs the window.speechSynthesis polyfill, which is installed at document start in every frame - pages should use the STANDARD speechSynthesis API and these will be called for them. Android WebView ships no working speechSynthesis of its own, which is why the reroute exists.

Described by its group, Text-to-speech, rather than on its own.

Example

Stops speech right now and throws away everything still waiting to be spoken. Pages normally call the standard speechSynthesis.cancel(), which calls this for you.

Returns: nothing. Needs: nothing.

The public way (use this):

document.getElementById('stop-reading').addEventListener('click', function () {
  if ('speechSynthesis' in window) speechSynthesis.cancel();
});

The raw call:

if (window.WebToApk && typeof window.WebToApk.ttsCancel === 'function') {
  window.WebToApk.ttsCancel();
}

Say only the newest status and drop anything older still queued (a delivery tracker, a live score):

function announceLatest(text) {
  if (!('speechSynthesis' in window)) return;
  speechSynthesis.cancel();   // the old line gets error 'interrupted', queued ones 'canceled'
  var u = new SpeechSynthesisUtterance(text);
  u.onerror = function (e) {
    if (e.error === 'interrupted' || e.error === 'canceled') return;   // replaced, not a failure
    showToast('Could not read that aloud');
  };
  speechSynthesis.speak(u);
}

announceLatest('Your driver is 5 minutes away');
announceLatest('Your driver has arrived');   // cuts the first one off

Notes: Speech that is cut off or dropped sends no appmint:tts end or error event, so an utterance's onend does not run after a cancel. If you track raw ttsSpeak ids yourself, forget them when you call ttsCancel().

`ttsGetVoices` bridge#

window.WebToApk.ttsGetVoices(): String

JSON array of installed voices: [{name, lang, default}]. Empty until the engine finishes init - the page gets 'voiceschanged' then and re-queries, exactly like Chrome's async getVoices() contract.

Example

Lists the text-to-speech voices installed on the phone. Pages normally use the standard speechSynthesis.getVoices(), which calls this for you.

Returns: a JSON string, synchronously: [{ "name": "en-us-x-sfg-local", "lang": "en-US", "default": false }, ...]. It is "[]" until the engine has started. Needs: nothing.

The public way (use this):

function fillLanguages() {
  var langs = {};
  speechSynthesis.getVoices().forEach(function (v) { langs[v.lang] = true; });
  document.getElementById('langs').textContent = Object.keys(langs).join(', ');
}
if ('speechSynthesis' in window) {
  fillLanguages();
  speechSynthesis.onvoiceschanged = fillLanguages;
}

The raw call (remember JSON.parse):

function nativeVoices() {
  if (!(window.WebToApk && typeof window.WebToApk.ttsGetVoices === 'function')) return [];
  try { return JSON.parse(window.WebToApk.ttsGetVoices()); } catch (e) { return []; }
}

window.addEventListener('appmint:tts', function (e) {
  if (e.detail && e.detail.event === 'voiceschanged') {
    var hindi = nativeVoices().filter(function (v) { return v.lang.indexOf('hi') === 0; });
    document.getElementById('hindi-btn').hidden = hindi.length === 0;
  }
});

Speak in the phone's language when a voice for it is installed, otherwise let the engine use its default voice:

function bestSpeechLang() {
  if (!(window.WebToApk && typeof window.WebToApk.ttsGetVoices === 'function')) return '';
  var voices;
  try { voices = JSON.parse(window.WebToApk.ttsGetVoices()); } catch (e) { return ''; }
  var want = navigator.language || 'en-US';            // e.g. 'ta-IN'
  var exact = voices.filter(function (v) { return v.lang === want; })[0];
  if (exact) return exact.lang;
  var base = want.split('-')[0];
  var near = voices.filter(function (v) { return v.lang.split('-')[0] === base; })[0];
  return near ? near.lang : '';                        // '' = the engine's default voice
}

if (window.WebToApk && typeof window.WebToApk.ttsSpeak === 'function') {
  window.WebToApk.ttsSpeak(greetingText(), bestSpeechLang(), 1.0, 1.0, 'greeting');
}

Notes: lang is a BCP-47 tag such as en-US or ta-IN. A voice's name is the engine's own id and cannot be chosen by name when speaking; only its language is used.

`ttsIsSpeaking` bridge#

window.WebToApk.ttsIsSpeaking(): Boolean

Backs the window.speechSynthesis polyfill, which is installed at document start in every frame - pages should use the STANDARD speechSynthesis API and these will be called for them. Android WebView ships no working speechSynthesis of its own, which is why the reroute exists.

Described by its group, Text-to-speech, rather than on its own.

Example

Tells you if the phone's text-to-speech engine is speaking at this moment. Pages normally use speechSynthesis.speaking or the utterance's onend instead.

Returns: true / false, synchronously. false before the engine has started. Needs: nothing.

The public way (use this):

if ('speechSynthesis' in window && speechSynthesis.speaking) {
  speechSynthesis.cancel();
}

The raw call asks the engine itself, so it is exact even if speech was started with the raw ttsSpeak:

function isSpeaking() {
  if (window.WebToApk && typeof window.WebToApk.ttsIsSpeaking === 'function') {
    return window.WebToApk.ttsIsSpeaking() === true;
  }
  return false;
}

var btn = document.getElementById('read-btn');
setInterval(function () {
  btn.textContent = isSpeaking() ? 'Stop' : 'Read aloud';
}, 500);

Wait until the app has stopped talking before you start recording, so the mic does not pick up its own voice:

function whenQuiet(fn) {
  if (!(window.WebToApk && typeof window.WebToApk.ttsIsSpeaking === 'function')) { fn(); return; }
  (function check() {
    if (window.WebToApk.ttsIsSpeaking()) setTimeout(check, 300);
    else fn();
  })();
}

document.getElementById('answer-btn').addEventListener('click', function () {
  whenQuiet(startRecordingAnswer);   // e.g. after the question was read aloud
});

Notes: Do not poll fast; an event (onend, or appmint:tts with event: 'end') is better when you started the speech yourself.

`ttsSpeak` bridge#

window.WebToApk.ttsSpeak(text: String, lang: String, rate: Double, pitch: Double, utteranceId: String)

Backs the window.speechSynthesis polyfill, which is installed at document start in every frame - pages should use the STANDARD speechSynthesis API and these will be called for them. Android WebView ships no working speechSynthesis of its own, which is why the reroute exists.

Described by its group, Text-to-speech, rather than on its own.

Example

Starts speaking text with the phone's text-to-speech engine. Pages normally use the standard speechSynthesis API, which calls this for you.

Returns: nothing. Progress arrives later as the appmint:tts event (and the window.onAppMintTts(detail) hook) with detail = { id, event }, where event is 'start', 'end' or 'error', and id is the utteranceId you passed. Needs: nothing.

The public way (use this):

if ('speechSynthesis' in window) {
  var u = new SpeechSynthesisUtterance('Your order is ready');
  u.lang = 'en-GB';
  u.onend = function () { console.log('done'); };
  speechSynthesis.speak(u);
}

The raw call, if you want your own ids. Arguments: text, lang, rate, pitch, utteranceId:

var ttsWaiting = {};

window.addEventListener('appmint:tts', function (e) {
  var d = e.detail || {};
  if (d.event === 'voiceschanged') return;        // engine ready, id is ''
  var done = ttsWaiting[d.id];
  if (!done || d.event === 'start') return;
  delete ttsWaiting[d.id];
  done(d.event === 'end');                         // 'end' = spoke it, 'error' = failed
});

function speakRaw(text) {
  return new Promise(function (resolve) {
    if (!(window.WebToApk && typeof window.WebToApk.ttsSpeak === 'function')) { resolve(false); return; }
    var id = 'mine_' + Date.now();
    ttsWaiting[id] = resolve;
    window.WebToApk.ttsSpeak(String(text), 'en-US', 1.0, 1.0, id);
  });
}

speakRaw('Hello').then(function (ok) { console.log(ok ? 'spoken' : 'could not speak'); });

Read a list of steps in order with your own ids, and highlight the step being spoken:

var steps = Array.prototype.slice.call(document.querySelectorAll('.recipe-step'));

window.addEventListener('appmint:tts', function (e) {
  var d = e.detail || {};
  var m = /^recipe_(\d+)$/.exec(d.id || '');
  if (!m) return;                                  // not one of ours
  var el = steps[Number(m[1])];
  if (d.event === 'start') el.classList.add('reading');
  if (d.event === 'end' || d.event === 'error') el.classList.remove('reading');
});

document.getElementById('read-recipe').addEventListener('click', function () {
  if (!(window.WebToApk && typeof window.WebToApk.ttsSpeak === 'function')) return;
  steps.forEach(function (el, i) {
    // '' = the engine's default voice; 0.9 = a little slower for cooking
    window.WebToApk.ttsSpeak(el.textContent, '', 0.9, 1.0, 'recipe_' + i);
  });
});

Notes: Utterances queue one after another. lang may be '' (the engine's default voice); rate and pitch are clamped to 0.1-4. If the engine is still starting, the text waits and is spoken when it is ready; if the engine cannot start, each waiting utterance gets 'error'. Use your own id prefix; ids starting wta_u belong to the speechSynthesis layer. On Android 8+ appmint:tts also carries {id, event: 'boundary', charIndex, charLength} before each word. This raw call has no volume or voice - speechSynthesis has both.

`ttsWarmUp` bridge#

window.WebToApk.ttsWarmUp()

Backs the window.speechSynthesis polyfill, which is installed at document start in every frame - pages should use the STANDARD speechSynthesis API and these will be called for them. Android WebView ships no working speechSynthesis of its own, which is why the reroute exists.

Described by its group, Text-to-speech, rather than on its own.

Example

Starts the phone's text-to-speech engine early, so the first speak() has no delay and getVoices() fills in. The app's speechSynthesis layer already calls this at page start, so pages almost never need it.

Returns: nothing. When the engine is ready, the appmint:tts event fires with detail = { id: '', event: 'voiceschanged' } (and speechSynthesis.onvoiceschanged runs). Needs: nothing.

The public way (use this): just read voices when they are ready.

if ('speechSynthesis' in window) {
  speechSynthesis.onvoiceschanged = function () {
    console.log('voices:', speechSynthesis.getVoices().length);
  };
}

The raw call, then wait for the ready event:

window.addEventListener('appmint:tts', function (e) {
  if (e.detail && e.detail.event === 'voiceschanged') {
    var list = JSON.parse(window.WebToApk.ttsGetVoices());
    console.log('engine ready,', list.length, 'voices');
  }
});

if (window.WebToApk && typeof window.WebToApk.ttsWarmUp === 'function') {
  window.WebToApk.ttsWarmUp();   // safe to call many times
}

Start the engine when a "Read aloud" screen opens and enable its button once the engine is ready:

var readBtn = document.getElementById('read-aloud');

function engineReady() {
  try { return JSON.parse(window.WebToApk.ttsGetVoices()).length > 0; } catch (e) { return false; }
}

function openReaderScreen() {
  showReaderScreen();
  if (!(window.WebToApk && typeof window.WebToApk.ttsWarmUp === 'function')) return;
  readBtn.disabled = !engineReady();       // already started earlier? enable right away
  window.WebToApk.ttsWarmUp();             // no-op when the engine is already running
}

window.addEventListener('appmint:tts', function (e) {
  if (e.detail && e.detail.event === 'voiceschanged') readBtn.disabled = false;
});

Notes: voiceschanged fires once, when the engine finishes starting. If it already started (for example because the speechSynthesis layer warmed it), you will not get it again; call ttsGetVoices() directly.

SpeechRecognition web standard#

speechSynthesis web standard#

__srAbort bridge#

__srAvailable bridge#

__srStart bridge#

__srStop bridge#

__ttsSpeak bridge#

ttsCancel bridge#

ttsGetVoices bridge#

ttsIsSpeaking bridge#

ttsSpeak bridge#

ttsWarmUp bridge#

`SpeechRecognition` web standard#

`speechSynthesis` web standard#

`__srAbort` bridge#

`__srAvailable` bridge#

`__srStart` bridge#

`__srStop` bridge#

`__ttsSpeak` bridge#

`ttsCancel` bridge#

`ttsGetVoices` bridge#

`ttsIsSpeaking` bridge#

`ttsSpeak` bridge#

`ttsWarmUp` bridge#