Kotoba

Where to start

What Kotoba isWho she is and where she runs, in one page
InstallingOne package, two commands
First runA key, a model, her first words
The approval gateHow she asks before she acts
The two voice modesLocal voice, or the agent tunnel
The soul fileChange who she is
RoadmapWhat grows next, and what was cut on purpose

Or a section

↑↓ move openesc close124 pages
All pages

Architecture10 of 10

What is deliberately not in the architecture

About 5 minutes to read

On this page

Architecture pages are where stale diagrams go to be believed. This one exists so nobody documents a ghost, or waits for a subsystem that was refused rather than deferred.

Refused outright

ROADMAP.md names three, and they are refusals rather than backlog items:

  • A hosted version you sign into. Kotoba is self-hosted. There is no account system, no server to sign into, and no code path anywhere in the tree that sends a conversation to a Kotoba-operated service. The only sign-in that exists is the local web gate, which is a password you set and an HMAC-signed cookie your own backend mints.
  • Shipping a Live2D model in the repository. Models belong to their authors and their licences forbid redistribution. The repository carries an expression map so any model can be fitted, and the installer — never the repository — fetches one onto your machine, on request, once you accept the sample's terms.
  • Telemetry. Not now, not later. Nothing in the tree phones home. Next's own build-time telemetry is explicitly disabled in both scripts/build_web.py and the dev server kotoba serve launches.

The three-layer 3D scene does not exist

Design prose from before the first release described three stacked screen layers: a React DOM overlay, a Live2D canvas, and a Three.js canvas rendering a nocturnal garden built in Blender.

Only the first two shipped. There is no three and no @react-three/* dependency in package.json, no Scene3D component, and no .glb in the tree. What sits behind the avatar is components/RoomBackdrop.tsx: a still PNG (public/scene/room.png) with drifting dust motes and a slow warm light, in CSS. If you find a diagram with a Three.js layer in it, the diagram is older than the code.

Voice does not go through a tunnel by default

Another piece of design-era prose has ElevenLabs' Conversational AI as the voice pipeline, which implies a public URL pointing at your backend.

The default today is voice_mode=local: the browser captures microphone audio and ships it to your own backend over WS /api/voice/{session_id}, and the backend talks outbound to ElevenLabs for transcription and speech. Every connection leaves your machine; nothing listens for the outside world.

The agent mode is still there and still supported — it is the path that posts to /v1/chat/completions — and it genuinely does need a public URL, because in that mode ElevenLabs' cloud is the one calling you. It is a deliberate alternative, not the default, and the whole inline-versus-deferred distinction exists because of it.

Products that were cancelled

Older material describes a three-product line: an open-source edition, a managed cloud subscription, and a B2B offering. Only the open-source project exists. There is no billing code, no tenancy, no licence tier and no revenue-gated term anywhere in the repository; the licence is MIT, full stop, and "a hosted version you sign into" is on the refused list above.

Cadence numbers that were resized

Two sets of timing numbers in older prose were sized by ElevenLabs constraints that no longer apply. The live values, all in core/loop.py:

Old proseIn the code
Tool calls per companion turn"max 10"_MAX_TOOL_CALLS = 8; work mode has its own cap of 40
First heartbeat"never more than 5 s of silence"_HEARTBEAT_FIRST = 9.0
Subsequent heartbeats"every 3–4 s"_HEARTBEAT_EVERY = 21.0

The old numbers were sized against a protocol floor that is gone. Nothing cuts a turn in voice_mode=local, and the roughly-7-second cut on the ElevenLabs agent path is answered by the buffer word in server.py's /v1 generator alone — never by model narration. The current cadence is about how long a wait goes unacknowledged before it reads as a hang, and four hums inside twelve seconds was measured as grating.

Emissions that look missing and are not

Every tool declares three expressions — focus, done, fail — and the loop emits only two of them. done is read by nothing.

That is deliberate and should not be "fixed". The face for a finished turn comes from the audio tag she actually wrote, resolved through stream.TAG_TO_FACE, so voice and face have one source and cannot disagree. Emitting a done expression here would fight the tag a moment later. fail is the asymmetry with a reason: a stumble has to show mid-turn, before she has said anything.

Code that is present and uncalled, on purpose

interaction.dismiss() has no production caller, and its docstring says so. It stays because it is the one safe shape for a wave-away gesture — resolving the Future with a sentinel, never cancelling the task behind it, since a detached approval task has two lives and cancelling it could stop a command the user had just said yes to. The obvious rewrite is the defect. Delete it only together with the DISMISSED ending.

Database.repeated_turn_report() is the same kind of thing: a read-only audit with no production caller, deliberately not a sweeper, because which of a user's own conversation records get rewritten is their call and not a de-duplication verdict.

Known gaps, named rather than hidden

These are real limitations of the architecture as built, and README.md lists them plainly:

  • MCP tool calls are not carded. Installing or connecting a server asks; the tools it then exposes are audited, not gated.
  • You can switch off a whole toolset, not a single command. Only the allow side of a grant is persisted.
  • The interactive terminal needs a POSIX terminal. setup, doctor, serve, discord and --once run natively on Windows; the full session does not.
  • The report viewer and pending attachments live in memory and are gone when the process restarts. The report's own HTML file is in the workspace and stays.
  • Orphaned interaction cards. In-memory card state is keyed per session id, so if a new session begins a turn while another session still holds an unanswered card, those cards never resolve. interaction.note_turn logs a warning naming them; it is an accepted limitation, not a fixed bug.
  • A DNS-rebinding TOCTOU against the SSRF guard stays open. core/ssrf.py says so in its own docstring.

For what is planned versus merely wished for, ROADMAP.md in the repository is the source, and it separates the two explicitly.