Kotoba

Where to start

What Kotoba isWho she is and where she runs, in one page
InstallingOne package, two commands
First runA key, a model, her first words
The approval gateHow she asks before she acts
The two voice modesLocal voice, or the agent tunnel
The soul fileChange who she is
RoadmapWhat grows next, and what was cut on purpose

Or a section

↑↓ move openesc close124 pages
All pages

Voice and avatar1 of 9

What the voice costs

About 4 minutes to read

On this page

Kotoba's voice is ElevenLabs. Both directions — hearing you and speaking back — go to ElevenLabs, and both use the same API key. There is no other speech engine in the code.

The key

One key, two jobs.

JobElevenLabs modelWhere
Hearing youscribe_v2_realtimeapi/src/kotoba/core/voice/stt.py
Speaking, default engineeleven_v3api/src/kotoba/core/voice/tts_rest.py
Speaking, fast engineeleven_flash_v2_5api/src/kotoba/core/voice/tts.py

Those three model ids are what Kotoba itself calls, which is the local voice mode. In agent mode the agent in your own ElevenLabs account does the listening and the speaking, with whatever models you configured it with.

The key is resolved in one place, core/voice/config.py:resolve_api_key(), in this order:

  1. a key saved inside the app — written by the setup wizard, encrypted in her database (db/database.py:save_key runs it through core/keystore),
  2. the ELEVENLABS_API_KEY environment variable,
  3. nothing.

The API key box in Settings → Brain is not this key: it saves your model provider's key. The voice key is entered by kotoba setup in the terminal, or by the browser's first-run screen — reachable again later through Settings → Brain → Run setup again. Both call the same code, core/voice_key.py.

A placeholder counts as nothing. api/.env.example ships ELEVENLABS_API_KEY=el_... on line 12, and copied unedited that string used to be sent to ElevenLabs as if it were real. It is now filtered out, and the error says which of the two problems you have:

ELEVENLABS_API_KEY still holds the placeholder copied from .env.example — run
`kotoba setup` to put your real ElevenLabs key in, or edit the variable

Setup checks a key against GET /v1/voices, not /v1/user. ElevenLabs keys carry scopes, and a key that speaks perfectly answers /v1/user with 401 missing the permission user_read. Only two answers make Kotoba throw a key away: ElevenLabs itself saying invalid_api_key, and a 429 against that key. A captive portal, a proxy or a 500 keeps the key and says so (api/src/kotoba/core/voice_key.py).

Without a key

She reads and writes. She does not speak, and she cannot hear.

Measured on a voice socket opened with no key present: the socket accepts, sends its ready frame, and the very first microphone frame comes back as a fatal error.

json
{"type": "ready",  "session_id": "...", "audio_in": {...}, "audio_out": {...}}
{"type": "error",  "code": "stt_auth", "fatal": true,
 "message": "No ElevenLabs API key configured — run `kotoba setup`"}

Nothing was sent to ElevenLabs — the key is checked before any network call, in config.auth_headers().

What still works with no key:

  • Typing in the browser. Typed messages ride the same voice socket, so you must start the call first; the call connects fine without a key. Her replies arrive as captions.
  • The whole terminal. kotoba, kotoba --once "..." and every tool need no voice key. kotoba doctor reports a missing key as a warning, not a failure — "no ElevenLabs key, so no voice … Text still works." The terminal's own VOICE row reads text only · no key (api/src/kotoba/cli/facts.py).

What stops:

  • The microphone. The first frame latches stt_auth and the mic is off for that call.
  • Her voice. The first thing she tries to say latches tts_auth and the rest of the call is captions only.

There is no free or local speech engine

Not "coming soon" — not written. There is no local speech-to-text and no local text-to-speech anywhere in the repository: no Whisper, no Piper, no Kokoro, no Coqui, no espeak, no bundled model weights. Searching the source for any of those names returns only the audio tag [whispers].

The dependency list says the same thing. The whole voice extra is one package:

api/pyproject.toml
voice = ["websockets>=16,<18"]

A WebSocket client. Nothing that could synthesize or transcribe anything on its own.

ROADMAP.md lists a local engine under Next, with no date. Until someone writes it, an ElevenLabs key is the price of hearing her voice.

What it does not cost

  • No tunnel, no public URL, in the default mode. Everything Kotoba needs from ElevenLabs is reached outbound. See Voice modes.
  • No ElevenLabs agent to register in the default mode. The agent id field in Settings is unused there.
  • No key for the face. The avatar renders and changes expression with no ElevenLabs key at all — emotions arrive on a separate channel from the backend. The mouth is the exception: it is driven from the spectrum of audio that is already playing, so with no key there is no audio and no mouth movement. Nothing in the avatar path talks to ElevenLabs itself.

Choosing her voice

The voice id is resolved by config.default_voice_id(): the voice_id stored in her soul config wins, then KOTOBA_VOICE_ID, then a built-in fallback id. soul/default.md seeds the stored value only while it is still empty, so once you pick a voice in Settings → Personality → Voice ID the file stops overriding you.

There is no voice picker. You paste an ElevenLabs voice id, which means leaving the app to go and find one. ROADMAP.md names this as a known gap.