Visual memory — what she chose to keep

Visual memory is the durable half. Where a session capture is scratch that dies with the process, a keepsake is a picture she decided was worth keeping, filed under who or what it is about, with a note. It survives restarts and it is recallable in a conversation months later.

Think of it as the picture twin of her text memory. memory_write saves a fact about someone; remember_image saves a photo of them under the same name, and the two line up.

Where it lives

~/.kotoba/visual-memory/
├── index.json            the catalog
└── images/
    ├── 4f2a91c0b3.png
    └── e0c560b9d2.jpg

KOTOBA_VISUAL_MEMORY_DIR moves the whole directory. The images are copies, taken at save time, so a keepsake keeps working after you delete the original from Files.

One entry in index.json looks like this, and this is the entire record:

{
  "id": "e0c560b9d2",
  "file": "e0c560b9d2.png",
  "about": "Alex Doe",
  "note": "profile photo, glasses",
  "kind": "person",
  "source": "attachment:alex.png",
  "ts": 1788937011.547209
}

id is the only handle anything outside the store may hold. Nothing that shows a keepsake takes a path from its caller: the filename is read out of the index, reduced to its basename and re-checked against the images directory before a byte is opened.

Saving one — remember_image

She calls it with about (required), source (required), and optionally note and kind. kind is one of person, self, product, post, place, scene, other; anything else becomes other.

source decides where the bytes come from:

The URL path exists so that a profile photo is kept as the original file rather than as a screenshot of the page it was on. It strips a CDN's resize-and-blur transform from the query, tries that first, and keeps whichever of the two responses is larger. The fetch is SSRF-checked, follows at most 5 redirects with each hop re-checked, and refuses anything over 12,000,000 bytes or that does not look like an image.

What it refuses, and how honestly

The store answers None for several different reasons and only one of them means the image is already safe. So the tool asks two follow-up questions — is there a duplicate? and are its bytes still on disk? — before it says anything:

The distinction is not pedantry. Reporting a save by searching for the entity name — true whenever a second photo of someone already kept is being saved — announced a rejected image as safely stored.

An index row whose image file has vanished is not a duplicate. It is replaced by the new save, because otherwise a good picture would be refused forever by a row nothing could reap.

Deduplication and the cap

The dedup key is about + note + kind, with about compared without accents or case, so Fulano Pérez and fulano perez are one person. A different note is a different keepsake, which is how one entity holds several pictures.

The store is bounded on purpose, unlike the file library: every entry here is a copy of something else, and the index is re-read on every turn.

about and note are collapsed to a single bounded line because the model writes them, sometimes while reading a page somebody else controls, and about is interpolated into a block of the prompt on every later turn. A newline there would let page text open its own section.

Bringing one back — recall_image

recall_image(query) searches the keepsakes and re-opens the matches as images, so she answers from the picture and not from her own old note.

Search is keyword overlap over about, note and kind, accent- and case-insensitive, with a bonus for a direct substring match on about. It returns the best 8; at most 3 are actually attached as images. Anything past that, and anything whose file is missing, is listed separately as a note with nothing behind it, under a line telling her never to describe those as if she were looking at them.

If nothing came back as an image at all, the result says so plainly and instructs her to say she cannot bring it up right now. The instruction is written from what was attached, never from what was found — an entry handed over as "analyze the image" with no image attached invites exactly one completion, and it is a lie.

What you see

Recall has two audiences and used to serve one: the images went to the model while the screen stayed empty and she said "here he is". Now each attached image also rides the events channel as a recalled_image frame carrying its id only — never the bytes, which would flood the same queue that carries approval cards.

In the browser, the transcript renders an <img> pointing at GET /api/visual-memory/{id}, with the alt text set to what the keepsake is about. That route:

• takes an id, never a path. Traversal attempts and unknown ids are 404, indistinguishably;
• sends X-Content-Type-Options: nosniff and a raster media type, so a stored SVG comes back as application/octet-stream rather than as a scriptable document on your origin;
• sits behind the same gate as every other /api/* route, and accepts the token in the query string because an <img> cannot set a header.

A miss shows nothing. No frame is emitted, so the screen never promises a picture the endpoint would 404 on.

The terminal does not draw recalled images; it shows a status line while the tool runs.

What she is told she has

If the store is not empty, a block goes into every turn — every turn of hers; a delegated helper gets neither this block nor the captures one — naming the entities and how many pictures each one has, the top 12 by count:

VISUAL MEMORY (durable, yours) — images you've chosen to remember, by who/what they're about:
Alex Doe (2), the user (1). Use recall_image(query) to bring one back into view …

Entities are grouped on the accent-folded name, keeping the first spelling seen, so one person with two spellings is one line and not two. The block lists entities, not filenames; filenames belong to the captures block, and that is why view_capture takes a file and recall_image takes a query.

Next: The three tools.