Kotoba

Where to start

What Kotoba isWho she is and where she runs, in one page
InstallingOne package, two commands
First runA key, a model, her first words
The approval gateHow she asks before she acts
The two voice modesLocal voice, or the agent tunnel
The soul fileChange who she is
RoadmapWhat grows next, and what was cut on purpose

Or a section

↑↓ move openesc close124 pages
All pages

Discord7 of 10

Voice in a room

About 6 minutes to read

On this page

Getting her in

discord_voice is a tool she holds, so you ask her in words: "come into voice", "join the vc". With no channel named she joins the one you are in — which is why she needs the voice_states intent. Any member can ask; it is on the guest list.

Asking her to leave calls the same tool. If she is asked from inside a turn she is still answering, she finishes speaking first and then goes, rather than cutting off the goodbye she has not said yet.

What voice needs beyond the text setup:

  • The voice extra. Without it she joins and then neither hears nor speaks, because her transcriber and her synthesiser cannot be imported.
  • An ElevenLabs key. Voice is ElevenLabs both ways: their realtime transcriber in, their synthesis out.
  • libopus, the system library. Windows gets the DLL inside the wheel; on Linux and macOS install it, or she can neither decode what she hears nor encode what she says.

Discord voice is end-to-end encrypted, and she decrypts it herself, per speaker. PyNaCl and the DAVE library come with the discord extra and the bot will not start without them.

How she hears people

One transcriber per person, opened on their first sound and kept open while they are in the room. Closing it between sentences reopened it more than four times in ten within five seconds, and each handshake took a fifth of a second out of the front of whatever they said next — which is exactly where a name goes. A frame of silence every ten seconds costs less than that.

Discord sends nothing at all while somebody is quiet. There are no silent packets to wait on, so the end of a sentence is noticed on a local clock:

IntervalMeaning
0.15 s with no audioThey stopped talking
1.0 s moreThe sentence is over; commit it
2.0 s instead...unless it might not be: a phrase under 1.2 s, nothing transcribed yet, or a last word that is grammatically waiting for the next one ("and", "pero", "porque", "um")
10 s idleOne frame of silence, to keep the transcriber's socket alive

Transcription is pinned to her own configured language rather than left to auto-detect — unless that language is auto, or you override it with KOTOBA_STT_LANGUAGE. It matters more than it looks: her name is a real Japanese word, and a Spanish "Kotoba, hola" came back written in katakana, which matched nothing.

Her name is how you reach her, and how you stop her

In a voice channel she answers only when somebody says her name. Not a mention — the spoken name. What counts as her name is read off the gateway: her display name in that server, her nickname, her username, and the names of the roles she wears. Whoever runs her can rename her, and a hardcoded name would be wrong for everybody but its author.

Where the name may sit in what you said:

  • at the front of a clause (the first two words),
  • at the end of it,
  • anywhere inside a short clause of five words or fewer.

Her name buried mid-sentence in a long one is a room talking about her, and she stays out of it.

The match is deliberately fuzzy, because a name is the word transcription gets wrong: it has no dictionary to fall back on and comes out spelled how it sounded. The tolerance scales with the length of the name, so a short one cannot be reached from an ordinary word — and one extra edit is allowed only when the punctuation shows the word was used to address somebody.

Her name is also the interrupt. Say it at the front of what you say while she is talking and she stops. That is heard in a partial transcript, seconds before the finished sentence arrives, so there is nothing to add after it and no phrase to remember. Whatever follows the name is then an ordinary turn: ask her to be quiet and she will, in her own words.

Two more shapes:

  • Her name and nothing else, while she is idle: she answers with a short sound in her own voice and then listens for 8 seconds without needing the name again. Making you say it twice is rude.
  • Her name and nothing else, while she is talking: the interrupt was the whole message. She stops and says nothing.

She has one voice in that room, so a second person calling her replaces the answer in flight rather than queueing behind it. A queued answer arrives after the room has moved on and reads as her malfunctioning.

What she says, and what is filtered out

A spoken reply goes through the same chain as every other voice surface before it reaches the synthesiser: tool-call leaks, code fences, URLs and forbidden phrases are taken out, and bracketed text is judged — an audio tag the expressive engine can actually perform is kept for it to perform, anything else in brackets is dropped rather than read aloud. On the fast engine the tags go too, since it would read them out literally.

Given a reply that reads look at ((https://example.test/a?utm_source=x)) and run followed by a bash code block and the word ok, what reaches the synthesiser is exactly:

look at and run ok

She never reads a link or a block of code aloud. In writing she keeps both — there a link is the useful part.

She is told, for a spoken turn, that everybody in the room hears everything and nobody can skim it: two or three sentences, then stop, and offer the long version rather than delivering it. She also narrates while a tool runs in voice, which she does not do in writing — in a channel those canned lines are three English sentences posted ahead of an answer nobody asked in English.

A caption of what she said is posted in the voice channel's own text chat, up to 1900 characters, with her audio tags stripped and the citation brackets and links tidied.

The voice channel is its own conversation

A voice channel has its own session, separate from the text channel where you asked her to join. What was said there is her memory of that room.

When a spoken turn gives up

A turn is abandoned when nothing has happened for 75 seconds — nothing written, no step, no card open — or after 10 minutes in total. She says she got stuck and asks you to try again. An open approval card counts as something happening: that is a person reading, and its own window is longer than any total limit worth setting.

Leaving

She leaves a channel she has been alone in for five minutes — no non-bot member — because sitting there bills a transcriber and a gateway for nobody. She also leaves when you ask.