Voice and avatar1 of 9
What the voice costs
On this page
Kotoba's voice is ElevenLabs. Both directions — hearing you and speaking back — go to ElevenLabs, and both use the same API key. There is no other speech engine in the code.
The key
One key, two jobs.
| Job | ElevenLabs model | Where |
|---|---|---|
| Hearing you | scribe_v2_realtime | api/src/kotoba/core/voice/stt.py |
| Speaking, default engine | eleven_v3 | api/src/kotoba/core/voice/tts_rest.py |
| Speaking, fast engine | eleven_flash_v2_5 | api/src/kotoba/core/voice/tts.py |
Those three model ids are what Kotoba itself calls, which is the local voice mode. In agent mode
the agent in your own ElevenLabs account does the listening and the speaking, with whatever models
you configured it with.
The key is resolved in one place, core/voice/config.py:resolve_api_key(), in this order:
- a key saved inside the app — written by the setup wizard, encrypted in her database
(
db/database.py:save_keyruns it throughcore/keystore), - the
ELEVENLABS_API_KEYenvironment variable, - nothing.
The API key box in Settings → Brain is not this key: it saves your model provider's key.
The voice key is entered by kotoba setup in the terminal, or by the browser's first-run screen —
reachable again later through Settings → Brain → Run setup again. Both call the same code,
core/voice_key.py.
A placeholder counts as nothing. api/.env.example ships ELEVENLABS_API_KEY=el_... on line 12, and
copied unedited that string used to be sent to ElevenLabs as if it were real. It is now filtered out,
and the error says which of the two problems you have:
ELEVENLABS_API_KEY still holds the placeholder copied from .env.example — run
`kotoba setup` to put your real ElevenLabs key in, or edit the variable
Setup checks a key against GET /v1/voices, not /v1/user. ElevenLabs keys carry scopes, and a key
that speaks perfectly answers /v1/user with 401 missing the permission user_read. Only two
answers make Kotoba throw a key away: ElevenLabs itself saying invalid_api_key, and a 429 against
that key. A captive portal, a proxy or a 500 keeps the key and says so
(api/src/kotoba/core/voice_key.py).
Without a key
She reads and writes. She does not speak, and she cannot hear.
Measured on a voice socket opened with no key present: the socket accepts, sends its ready frame,
and the very first microphone frame comes back as a fatal error.
{"type": "ready", "session_id": "...", "audio_in": {...}, "audio_out": {...}}
{"type": "error", "code": "stt_auth", "fatal": true,
"message": "No ElevenLabs API key configured — run `kotoba setup`"}
Nothing was sent to ElevenLabs — the key is checked before any network call, in config.auth_headers().
What still works with no key:
- Typing in the browser. Typed messages ride the same voice socket, so you must start the call first; the call connects fine without a key. Her replies arrive as captions.
- The whole terminal.
kotoba,kotoba --once "..."and every tool need no voice key.kotoba doctorreports a missing key as a warning, not a failure — "no ElevenLabs key, so no voice … Text still works." The terminal's own VOICE row readstext only · no key(api/src/kotoba/cli/facts.py).
What stops:
- The microphone. The first frame latches
stt_authand the mic is off for that call. - Her voice. The first thing she tries to say latches
tts_authand the rest of the call is captions only.
There is no free or local speech engine
Not "coming soon" — not written. There is no local speech-to-text and no local text-to-speech
anywhere in the repository: no Whisper, no Piper, no Kokoro, no Coqui, no espeak, no bundled model
weights. Searching the source for any of those names returns only the audio tag [whispers].
The dependency list says the same thing. The whole voice extra is one package:
api/pyproject.toml
voice = ["websockets>=16,<18"]
A WebSocket client. Nothing that could synthesize or transcribe anything on its own.
ROADMAP.md lists a local engine under Next, with no date. Until someone writes it, an
ElevenLabs key is the price of hearing her voice.
What it does not cost
- No tunnel, no public URL, in the default mode. Everything Kotoba needs from ElevenLabs is reached outbound. See Voice modes.
- No ElevenLabs agent to register in the default mode. The agent id field in Settings is unused there.
- No key for the face. The avatar renders and changes expression with no ElevenLabs key at all — emotions arrive on a separate channel from the backend. The mouth is the exception: it is driven from the spectrum of audio that is already playing, so with no key there is no audio and no mouth movement. Nothing in the avatar path talks to ElevenLabs itself.
Choosing her voice
The voice id is resolved by config.default_voice_id(): the voice_id stored in her soul config
wins, then KOTOBA_VOICE_ID, then a built-in fallback id. soul/default.md seeds the stored value
only while it is still empty, so once you pick a voice in Settings → Personality → Voice ID the
file stops overriding you.
There is no voice picker. You paste an ElevenLabs voice id, which means leaving the app to go and
find one. ROADMAP.md names this as a known gap.
