Kotoba

Where to start

What Kotoba isWho she is and where she runs, in one page
InstallingOne package, two commands
First runA key, a model, her first words
The approval gateHow she asks before she acts
The two voice modesLocal voice, or the agent tunnel
The soul fileChange who she is
RoadmapWhat grows next, and what was cut on purpose

Or a section

↑↓ move openesc close124 pages
All pages

The agent2 of 15

How a turn works

About 10 minutes to read

On this page

One turn is one call into the agentic loop. Inside it the loop iterates: ask the model, run whatever tools it called, feed the results back, ask again. It stops when the model answers without calling a tool — or when a cap stops it.

Around every tool call there are three moments: announce, heartbeat, confirm. What reaches you at each depends on whether you are listening or reading.


The register decides what you hear

register is "voice or text", and it is not the same axis as channel. A message typed into the web app is channel="text" but register="voice" — she still speaks it.

The loop sets narrate_tools = narrate_tools and register == "voice". So:

  • register="voice" — the canned before/after lines are spoken.
  • register="text" — none of them are. They exist so a voice does not go silent, and written into a chat they are English sentences in front of an answer that was in another language.

The web app and the voice socket use the default, voice. The terminal passes text and draws a spinner instead. Discord passes text for a written message and voice when she is speaking in a voice channel.


Announce — before the tool runs

Two different things happen depending on the tool's risk.

An action (risk is write, exec or network) gets:

  • the once-per-turn working chip and a determined face;
  • the tool's own focus expression (thinking unless the tool declares otherwise);
  • a step card on the wire: {id, step_kind, action}, opened with phase: "start" and closed later with phase: "done".

A read (risk is read) gets no row. It gets a peek frame instead — one field naming the tool she is inside, emitted only when the value changes, and cleared by the next action and by the turn's exit. Without it, "what do you know about me" fires five reads and puts nothing on the wire at all, which a client cannot tell apart from a hang.

The action line is written from the real arguments:

ToolRow
shell$ ls -la
execute_codepython> print(1) (plus when the snippet has more lines)
read_file / write_file / patchread_file notes.txt
search_filessearch 'needle'
web_search / web_extractweb_extract https://example.com
mcp_findsearch MCP registry: spotify
mcp_installconnect MCP: browser
anything elsethe tool's name

Each row also carries a step_kind so the client can style it: shell, code, web, file, mcp, skill, memory, or tool.

Then, on a voice register only, she says the tool's ANNOUNCE line — once per tool name per turn:

shell → "Let me run that real quick." web_extract → "Let me actually read that page for you." memory_write → "Oh, that's worth remembering. I'll keep it."

Those are her soul file's lines, and they are the ones you hear: _voice_for reads the SOUL override first and only falls back to the tool module's own ANNOUNCE when the soul file is silent about that tool. Edit them in soul/default.md, not in the module.

The line is skipped entirely for a tool the turn has withheld, for a switched-off family, for an MCP tool, and for a reasoning model.

On a default install, that last one applies. The shipped model is gpt-5.6-luna and reasoning_effort defaults to low, so is_reasoning_model() returns True out of the box and the canned ANNOUNCE and COMPLETE lines above are suppressed. The model narrates in its own words, in your language, instead. You see them as written only on a model that does not reason, or with reasoning_effort: off.


Heartbeat — while the tool runs

The tool runs as a task. The loop polls it every 3 seconds. If it is still running at 9 seconds, she says the first heartbeat. After that, one every 21 seconds.

_HEARTBEAT_TICK  = 3.0
_HEARTBEAT_FIRST = 9.0
_HEARTBEAT_EVERY = 21.0

There is no protocol floor behind those numbers. Nothing cuts a turn for silence in the default local voice mode. The curve is only about how long a wait goes unacknowledged before it reads as a hang, and each phrase is a separate speech request — four hums inside twelve seconds was measured as grating.

Beats are deferred, not skipped, while the tool is blocked on you. If the turn is waiting on a card that blocks — an approval, an ask_secret or request_credential box, an OAuth wait — the next beat is pushed out rather than spoken: the card already says everything, and narrating over it read as work.

The non-blocking cards are excluded by construction, not by a list that could fall behind. The test is "does this turn hold a future somebody has to resolve", and open_input_card / open_link_card register none, because their answer is meant to arrive as an ordinary next turn.

Who gets which heartbeat

Measured directly from core/loop.py::_heartbeat_lines:

ModelRegisterToolHeartbeat
non-reasoningvoicebuilt-inthe tool's own phrases — "Working on it...", "Still running...", "Almost there..."
non-reasoningvoiceMCPa wordless hum — "Mmm...", "Hmm...", "Mm...", "Hmmm..."
reasoningvoiceanythe wordless hum
anytextanynothing

The hum needs voice_mode: local. On voice_mode: agent the same rows return nothing at all, because an ElevenLabs agent is already filling the gap.

The reasoning-model rule exists because such a model narrates its own tool use in your language. The canned English would be a duplicate — but suppressing it outright once left a 10-second tool with 10.09 seconds of measured silence, so the hum replaced it rather than nothing.

One caveat about the helper itself: _heartbeat_lines("shell", …, register="text") returns the canned phrases for a non-reasoning model. It never reaches you, because narrate_tools is already false on a text register and the caller does not build a heartbeat list at all. The effective behaviour is the table above.

Timeouts

The default tool budget is 30 seconds. Three things widen it:

  • A tool that declares its own TIMEOUTdelegate takes 420 s, discord_apply_plan 600 s, mcp_find 180 s, mcp_install 150 s, discord_act 240 s, discord_voice 90 s.
  • A tool that takes a timeout argument (shell, execute_code): the requested value plus the approval window plus 5 s, capped at MAX_TIMEOUT = 600 s.
  • Any exec-risk tool: at least the approval window plus 5 s, so the loop can never cancel a tool while its own approval card is still on your screen.

Confirm — after the tool returns

The step card closes with an outcome, and the outcome — not a bare success flag — decides both the row you see and the sentence she says.

OutcomeRowSpoken
okthe real output, trimmedthe tool's COMPLETE line
failed! <first line of the error>the tool's FAIL line
refusedone of four sentences, belownothing
pendingthe card is still open, and this row completes when it is answerednothing
interrupted(interrupted)nothing

Each answer comes from the thing that witnessed it, never from the words: pending from the deferred record, refused from whichever gate ended the call without running, failed from the process's own exit code, then ok. Refusal is checked first, because your decision is not an error.

That matters because ok answers a different question — whether the tool handed the model usable output. A shell that exited 2, or was killed at its timeout and reported 124, returns a perfectly good string. Drawn from ok alone, the panel put a green tick over commands that failed, and over an inline refusal, whose "I held off on that one" is also a string.

refused is four endings under one mark, and they share the mark because it answers "did it run". What they must not share is the sentence:

Row
You declinedyou said no — it never ran
The card expiredno answer on the card — it never ran
No card could be drawnthere was no way to ask you — it never ran
You spoke over itthe card went away when you spoke — it never ran

The row used to read "cancelled by user" over a card that had timed out unread — the one thing this panel may never do: report a decision you never made. The audit trail keeps the same distinction, recording expired, dismissed or error rather than user, measured on a typed df -h that expired unread at 181 seconds.

Three of the five outcomes say nothing aloud on purpose. A declined command, an expired card and a barge-in are not failures of hers, so a fail: line would blame the tool for your decision — and refused is four endings that no single canned sentence could cover. Each already returns a first-person sentence she relays in your own language instead.

The row carries the whole output

The line in the panel is trimmed — four lines for shell, one line for most tools. The complete output rides on the same frame behind an expander, capped at the same 16,000 characters the model itself was allowed to read (24,000 for browser__*). When it does clip, it says so.

That is not decoration. Asked four times how many things were in one folder she answered 23, 14+9, 92 and 25 — and the listing that would have settled it reached the screen as 18 of its 443 characters.

The face

Only two expressions are emitted around a tool: focus when it starts, fail when it stumbles.

Success is silent on purpose. The face for a finished turn comes from the audio tag she actually wrote, resolved by the one map every surface reads — one source for voice, browser and terminal instead of three that can disagree. Tools still declare a done face; nothing reads it, and adding an emission there would fight the tag a moment later.


What the model is told when something does not run

Every refusal path hands the model an explicit sentence, because a silent empty result reads as a success. The wording is blunt for a reason: a tool that returned nothing used to be described to you as done.

  • Withheld from this turn: "You do not have '…' in this conversation, so NOTHING ran and there is no result. … Do not describe what it would have returned, and do not tell them it worked."
  • Family switched off in Settings: "'…' is switched OFF in the user's settings, so nothing ran … Tell them plainly that those tools are turned off."
  • Unknown name: "There is no tool called '…' — it does not exist, or its server just disconnected."
  • Exact repeat that succeeded: "You ALREADY called this exact tool with these arguments and it is DONE … Do NOT call it again and do NOT say it failed."
  • Exact repeat that never ran: "it ended at the user's card, or was refused before it started … do NOT say it is done."
  • Per-tool cap hit: "THIS CALL DID NOT RUN … Do NOT keep preparing or re-checking, and do NOT tell the user this one is done."

When a turn is cut

If the turn is cancelled — you barge in, or the process stops — the loop's finally still runs. Every open step row is closed with outcome: "interrupted" and the text (interrupted), the peek frame is cleared, and the working chip is turned off unless a background job is still running. A row nobody closes spins on your screen for the rest of the session.

(interrupted) is display text only. The state of a cut step travels as outcome on the frame, never as that string.