Start1 of 7
What Kotoba is
On this page
Kotoba is an AI companion that runs on your own machine. She has an anime face that lip-syncs and changes expression, real-time two-way voice, and a full agent underneath: she searches the web, reads pages, remembers across sessions, manages a folder of files, and runs shell commands and Python on your computer behind an approval gate.
There is no account, no hosted service and nothing to sign into. You install a Python package, give it an API key for a model provider, and it talks to that provider from your machine.
Three front doors, one engine
| Door | What it is | Command |
|---|---|---|
| Browser | The companion with the Live2D face and the voice call | kotoba serve |
| Terminal | The same companion, writing instead of speaking, with a pixel-art portrait | kotoba |
| Discord | A bot that reads and answers in the servers you invite her to | kotoba discord |
They are not three products. They share one SQLite database, one memory folder and one file library,
so something you tell her in the terminal is there in the browser afterwards. The terminal runs the
engine in-process — it imports no web framework, which is why kotoba --once "…" works on an install
that has neither FastAPI nor Node.
A fourth surface exists but is not a front door: POST /v1/chat/completions on the backend, an
OpenAI-shaped endpoint used by the optional ElevenLabs agent voice mode.
What runs where
- The model is somebody else's. OpenAI or xAI today, or any endpoint that speaks the OpenAI Responses API. Your turns go to them.
- Speech is ElevenLabs. Your microphone audio goes out to them for transcription, and her words go out to them to be spoken. See Requirements.
- Everything else is local. The agentic loop, the tools, the database, her memory, her files and
your encrypted keys are all on your machine. The backend binds
127.0.0.1and nothing has to reach in from outside.
The one exception is the optional agent voice mode, where ElevenLabs' platform runs the call and has
to be able to reach your backend at a public URL. It is off by default.
She runs commands on your machine
The default execution backend is KOTOBA_SANDBOX=local, and that means what it says: her shell and
her Python run on your host, as your user, in her working directory. An approval card stands in front
of that.
Be precise about what the card covers, because this is the thing people assume wrongly.
Exactly two tools consult the command gate: shell and execute_code. Nothing else asks before
it acts. Her file tools are bounded by a jail — every path is validated to stay inside her working
folder — not by a card. write_file overwrites a file in that folder with no prompt at all.
Within those two tools, on the default local backend, this is what runs unasked and what asks
(measured against the gate, not read off a description):
| Runs unasked | Asks first |
|---|---|
ls, ls -la notes | rm notes.md, mv a b |
cat notes.md, grep -r foo . | cat /etc/passwd, cat ~/.ssh/id_rsa |
find . -name '*.md', mkdir out | find . -delete, tail -f log.txt |
python script.py, npm install, curl …, sudo ls |
Two rules explain the table. A read-shaped command has to come off a fixed allowlist, and every one
of its path arguments has to resolve inside her working directory — which is why cat notes.md is
free and cat /etc/passwd is not. And a flag that turns a read into something else (find -delete,
tail -f, sort -o) puts the command back on the card.
When a card does appear you can answer once, allow that command family, or allow that one exact line.
Commands the gate labels destructive — rm -rf is recursive-delete, sudo is
privilege-escalation, curl … | sh is pipe-to-shell — are offered neither of the two "always"
answers, at any width.
Four other tools draw an approval card of their own: mcp_install, mcp_find, discord_act and
discord_apply_plan. The tools an MCP server then exposes are not carded — they register as
ordinary tools with a network risk and never reach the gate. They are recorded in the audit log.
KOTOBA_SANDBOX=docker runs every command in a throwaway container instead, and none removes the
shell and code tools entirely. The full picture is in the security section of these docs.
What it is not
- Not a hosted product. There is no sign-up and no cloud tier.
- No Live2D model ships with it. See Requirements.
- No local speech engine yet. Voice needs an ElevenLabs account.
- No mobile layout. The companion shell has no width breakpoints; browsers also refuse a microphone
on a plain-HTTP origin that is not
localhost, so reaching her over HTTP from a phone on your network gets a page that cannot hear you. She says so rather than going quiet.
Next: Requirements.