Running the tests

There are two suites and one typecheck. All three were run while writing this page, and the numbers below are what they printed.

The four commands

pytest api/tests -q                                       # backend
npx tsc --noEmit --noUnusedLocals --noUnusedParameters     # typecheck
npm test                                                  # frontend
python scripts/build_web.py                               # needs Node, and no root .env

The pull request template turns these into four checkboxes, two of them conditional: the backend suite always; the frontend suite and typecheck if anything under app/, components/ or lib/ moved; the web build if anything the packaged UI ships was touched; and no key, token, personal path or real name in the diff.

The first three commands were run while writing this page. The fourth was not — it rewrites the packaged build — and is covered on the build pipeline page.

The Python suite

461 files under api/tests/, holding about 7,560 tests — the number moves with every fix. That is a long run and a heavy one; while working, run the file you are changing instead. The full suite was not run while writing this page, so no timing is quoted for it.

pytest api/tests/test_registry.py -q          # one file
pytest api/tests -q -k approval               # by name

The suite is organised by the thing it pins rather than by module. Filenames are sentences — test_a_withheld_tool_is_refused_at_the_door.py, test_a_card_needs_a_surface_that_paints_it.py — and the module docstring says what broke and why the test exists. Follow that when you add one: a name that states the property is easier to search than test_utils_3.

Each test has a 120-second timeout (pytest.ini_options.timeout in api/pyproject.toml). The reason is written beside it: a guard that hangs is a cancelled job on a runner and a green summary that never returns locally, which is not a red test.

api/tests/conftest.py also fails any test that leaves a non-daemon thread running, naming the test that leaked it rather than the next one downstream.

Two config files, on purpose

api/pyproject.toml [tool.pytest.ini_options] is authoritative. But pytest only reads it when api/ is the rootdir, so a contributor who clones and types pytest at the top would get no config at all. pytest.ini at the repository root repeats the same three lines so that running from either directory behaves the same. If the two ever disagree, api/pyproject.toml is the one that is right.

The network marker, and the second lock

A few tests fetch and then run somebody else's package: an MCP server off the npm registry (api/tests/test_mcp.py), a container image off a public registry (api/tests/test_sandbox_docker.py). Eight tests in total.

Nothing you clone reaches the internet because you typed pytest. Two independent locks say so:

1. addopts = -m 'not network' deselects them by default.
2. KOTOBA_ALLOW_NETWORK — because a -m typed on the command line replaces the first line rather than adding to it. api/tests/conftest.py skips every network test unless that variable is 1, true or yes.

Measured here, with the variable unset:

$ pytest api/tests -q
7543/7551 tests collected (8 deselected)

$ pytest api/tests -m network -q
8 skipped, 7543 deselected

Selecting them explicitly still skips them. To actually run them, opt in twice:

KOTOBA_ALLOW_NETWORK=1 pytest api/tests -m network

Not run while writing this page — by design, since that is the command that leaves the machine.

Mark a new test network whenever it downloads or executes code this repository did not write.

The frontend suite

npm test runs Node's own test runner over tests/*.test.mjs and lib/__tests__/*.test.mjs — 55 files. Node strips the TypeScript types and imports the real lib/ modules, so there is no build step and no bundler.

Measured: 325 tests, 325 passing, 1.3 s.

The two directories split by subject: lib/__tests__/ covers the integration layer (the gate, the voice socket, playback, attachments), and tests/ covers everything else (the avatar, expressions, the transcript, the settings panel, the notices).

One of them is a house rule rather than a unit test: tests/no-emoji-in-the-interface.test.mjs reads every .ts/.tsx file under components/, app/ and lib/ and fails on any emoji codepoint. The interface's own marks — ✓ ✕ ⚠ ✦ ♥ ★ — are typography and are allowed; the pictographic planes are not.

The typecheck

npx tsc --noEmit --noUnusedLocals --noUnusedParameters

Stricter than npm run build: an unused local or an unused parameter fails it. Measured: exit 0, no output. npm run typecheck is the same flags.

Adding a test to a fix

The rule the pull request template states, in its own words: a test that fails without the change and passes with it is the strongest form of this; "it should work" is the weakest.

Watch it fail before you fix the code. A guard that has never been red is not a guard — see the house rules.