The two speaking engines

In local voice mode Kotoba can speak through either of two ElevenLabs engines. Pick one in Settings → Personality → Voice engine (local mode), or with KOTOBA_TTS_ENGINE.

api/src/kotoba/core/app_settings.py
"tts_engine": ("KOTOBA_TTS_ENGINE", "expressive", _one_of("expressive", "fast"))

The default is expressive. The setting only affects local mode; in agent mode the agent in your ElevenLabs account does the synthesis and this control does nothing.

Measured by calling create_tts_client() with each setting:

expressive -> ExpressiveTtsClient
fast       -> TagStrippingTts

Why the split exists

It is not a design preference. eleven_v3 is the model that performs audio tags, and ElevenLabs answers 403 for eleven_v3 on the stream-input WebSocket. It is reachable over REST only. So "the engine that performs emotion" and "the engine that streams down a socket" could not be the same engine, and Kotoba carries both.

api/src/kotoba/core/voice/config.py
# eleven_v3 performs [audio tags] but EL returns 403 for it on the stream-input WebSocket (a platform
# restriction, verified live) — it is only reachable over REST, hence the expressive engine.

expressive: sentence by sentence

REST has no partial input, so a request needs a whole sentence. core/voice/sentences.py cuts the reply into sentences as it streams, and the engine posts them, up to two requests in flight at once (MAX_PARALLEL = 2), coalescing whatever is already queued into one request up to BATCH_CHARS = 250.

Measured, feeding four fragments 0.6 s apart, these are the requests that actually went out:

'[warmly] Hola, que tal.'
'[warmly] Te cuento una cosa.'
'Mmm...'
'[warmly] Y esto es lo ultimo.'

Three things are visible in that output.

• One request per sentence. They were separate because they arrived separately; back-to-back sentences merge into one request instead.
• The tag is carried forward. Each REST request is context-free, so a [warmly] in the first would only colour the first. _carry_emotion() re-prepends the last sustained tag to later tagless requests, which is what eleven_v3 does by itself when it sees a whole reply at once.
• A wordless hum gets no tag. Mmm... came through bare. A rare tag on a six-character body is the least reliable shape ElevenLabs documents, and its failure mode is speaking the tag's name. One-shot tags — laughs, laughs softly, giggles, sighs, gasps, pause — are never carried either, or she would laugh at every sentence.

The sentence rules

Measured against SentenceSplitter:

A stream that never terminates a sentence — lyrics, an unpunctuated run — is force-cut at 480 characters at the last natural pause. Measured on 720 characters with no full stop: one 479-character piece released, 239 held.

When the stream simply goes quiet mid-turn (she announced a tool and is now running it), a watchdog forces out a held sentence after HOLD_SECS = 0.4 if it already looks finished. It never forces out a fragment.

When a request fails

• No audio delivered and no retry used yet: one retry, after 0.35 s.
• Still nothing: that text is dropped and the stream carries on.
• Two consecutive losses (FAILURE_STREAK_LIMIT = 2): the stream fails and the dispatcher stops launching, so a dead stream cannot keep spending your quota.
• A rejected key or an exhausted quota fails immediately and is never retried.
• A byteless 200 past 6 seconds is treated as a suspected stall and pauses the dispatcher — nothing launched behind a stalled request could be heard first anyway.

Recovery is asymmetric, and honestly so

If a TTS stream dies mid-turn, the session opens a fresh one and re-feeds whatever the dead client accepted but provably never spoke. Only the REST engine can answer that question. Its text stays local until a request's audio is read back. Text sent down the flash WebSocket is gone the moment it is sent, so on fast nothing is recovered. From core/voice/session.py:

engines without the accounting … recover nothing: that is the honest floor, not an oversight.

fast: one socket, no tags

TtsClient primes the socket, feeds text as it arrives with try_trigger_generation, and ends with an empty {"text": ""} — the protocol's end-of-stream marker.

Flash models read [happily] aloud, literally. So the fast engine wraps the client in a filter that removes every bracket first, and it is streaming-safe. Measured, feeding a tag one token at a time:

sent:   "[hap" "pily] Ho" "la [ya" "y] mun" "do"
reached the socket:  " Ho" "la " " mun" "do"    →  " Hola  mundo"

Both the valid tag and the invented one are gone before ElevenLabs sees them.

Voice settings

• fast sends {"stability": 0.5, "similarity_boost": 0.8}.
• expressive sends nothing unless you set it — measured, expressive_voice_settings() returns {} on a fresh install and the request body omits the field. The code's reason is that eleven_v3 accepts only certain values, so nothing is guessed. Set KOTOBA_TTS_STABILITY, KOTOBA_TTS_SIMILARITY or KOTOBA_TTS_STYLE and only those are sent.

Choosing

fast is a legitimate trade and Settings labels it as one: "Fast (flash — lowest latency, flat voice)". A sentence-at-a-time REST engine cannot begin speaking until a sentence boundary exists, where a socket can start on a fragment.

If you switch to fast, do it from the Voice engine control and leave Expressive voice alone — the two are coupled, and switching engines is enough. See Audio tags for why.