--- title: "Vectra index recovery: separate state, cleanup, and rebuilding" description: Diagnose semantic-search failures across note status, Vectra files, and cleanup tasks, then choose a single-note or full-index recovery path. image: "https://zengliangyi.com/og/blog/en/vectra-index-recovery.png" --- An empty or failing search is not automatically an index-corruption incident. ChatCrystal keeps note status, SQLite embedding rows, Vectra files, and deletion cleanup tasks as separate pieces. Identify the inconsistent layer before choosing a repair scope. This article follows the [vector index](https://github.com/ZengLiangYi/ChatCrystal/blob/764276814e5c5a33ecbfeae63754f74b9261a3a8/server/src/services/vector-index.ts), [embedding flow](https://github.com/ZengLiangYi/ChatCrystal/blob/764276814e5c5a33ecbfeae63754f74b9261a3a8/server/src/services/embedding.ts), and [database initialization](https://github.com/ZengLiangYi/ChatCrystal/blob/764276814e5c5a33ecbfeae63754f74b9261a3a8/server/src/db/index.ts) at [`7642768`](https://github.com/ZengLiangYi/ChatCrystal/tree/764276814e5c5a33ecbfeae63754f74b9261a3a8). The full-recovery example moves the index aside. Stop every writer, including desktop applications sharing the directory, and back up the whole data directory first. Replace `` in commands with an actual ID. ## Record the symptom and status An empty result can mean there are no notes, notes are still `pending`, vector generation failed, the index directory is missing, or the query uses a model incompatible with stored vectors. `crystal status` only helps establish aggregate counts. Inspect a note’s `embedding_status` through the API: ```bash curl 'http://localhost:3721/api/notes?limit=100' curl 'http://localhost:3721/api/notes/' ``` The list is time-ordered and capped at 100 items; note objects include the status field. `done` is database state and does not establish index completeness: initialization marks notes with existing embedding rows as `done` without checking their Vectra items. `pending`, `failed`, and `syncing` need further inspection. Do not treat `GET /api/notes?embedding_status=pending` as a filter: this route does not implement that query parameter. ## Understand the database and index boundary On first access, `getIndex()` creates a `LocalIndex` under `vectra-index`; it does not scan SQLite’s embedding table and reconstruct every vector. The database and file index can therefore be out of sync. A note rebuild first requests vectors from the model, inserts new items in a Vectra update, swaps the SQLite embedding rows and marks the note `syncing`, deletes old items, commits the update, and finally marks the note `done`. On an error it tries `cancelUpdate()`; if the database has already been swapped, it saves `failed` for a later retry. A forced process termination can still leave an intermediate state that needs inspection. Deleting a note creates a pending row in `vector_cleanup_tasks`. Each semantic search preflight tries at most 25 pending cleanup tasks; a failed cleanup remains pending for a later retry, while search continues. That is a retry path for deletion work, not a general repair for arbitrary corrupted files. ## Try a low-risk single-note rebuild first When one note is affected, keep the index in place and call the synchronous entry point, then inspect the result and status: ```bash curl --request POST \ 'http://localhost:3721/api/notes//embed' curl 'http://localhost:3721/api/notes/' ``` This usually generates vectors, but there is a fast confirmation path: when a note is `syncing` and all its database vector IDs are already committed in Vectra, it is marked `done` without requesting the model again. The endpoint is therefore not an unconditional forced rebuild, and it does not rewrite the summary. When generation runs, the model, endpoint, and vector dimensions must be usable. If the provider or embedding model changed, first follow the [embedding configuration guide](/en/blog/embedding-model-configuration) and confirm the clearing and rebuild scope. ## Decide whether to clear the whole index Only after confirming that the file index is unreadable and the database notes are still usable sources should you perform a full rebuild. This example uses the default CLI directory and creates a unique backup location with `mktemp` to avoid overwriting or nesting inside an existing backup: ```bash crystal serve stop && recovery_backup = $( mktemp -d " $HOME /.chatcrystal/vectra-recovery.XXXXXX" ) && mv " $HOME /.chatcrystal/data/vectra-index" " $recovery_backup /vectra-index" && crystal serve --daemon && curl --request POST 'http://localhost:3721/api/notes//embed' ``` Packaged applications resolve their data directory through runtime paths; confirm the actual directory from configuration and logs before applying this example. List affected notes using `GET /api/notes?limit=100&offset=0` and subsequent pages. Repeat the single-note rebuild for every affected ID, checking each response until all targets are covered. `clearEmbeddingIndex()` deletes the index and clears the in-process cache without marking existing `done` notes as pending. `POST /api/embeddings/batch` selects only `pending`, `failed`, and `syncing`, while a restart can mark notes with embedding rows as `done`. Moving only the index aside and calling the batch endpoint can therefore return `queued: 0`; explicit per-note rebuilding avoids omitting those notes. ## Accept the recovery with repeatable checks Record three kinds of evidence: - The target notes reach `done`, and the queue’s failure count no longer grows. - Run a known-answer query, a paraphrase, and a no-answer query; record inclusion, order, and false matches. - Restart the server and query again to verify that the rebuilt index can be reopened. Readable files only establish that the index can be accessed, not that retrieval is useful. If results remain empty, inspect model settings, fields submitted for embedding, and query conditions; see the [semantic search workflow](/en/blog/semantic-search-workflow). Share: ## Related Posts [## AI Conversation Formats: JSONL Events vs SQLite Snapshots Compare 5 AI adapter formats: separate JSONL events from SQLite keys, and inspect indexed, orphan, and unknown-version chat transcripts in memory stores. 曾亮懿 October 7, 2026 4 min read #ChatCrystal #Node.js #SQLite](/en/blog/ai-conversation-format-divergence) [## How Claude Code, Cursor, Copilot, and Trae Store Chats on Disk Compare Claude Code, Cursor, Copilot, and Trae storage paths. Distinguish raw copies from exports, and verify migration retention with test manifests. 曾亮懿 October 7, 2026 4 min read #ChatCrystal #Node.js #SQLite](/en/blog/ai-conversation-management-options) [## Five AI conversation parsers: retained fields and loss boundaries Build an acceptance matrix from Claude Code, Codex, Cursor, Trae, and Copilot parsers, checking tool flags, thinking, order, and snapshot coverage. 曾亮懿 October 7, 2026 4 min read #ChatCrystal #TypeScript #Node.js](/en/blog/ai-conversation-parser-boundaries) ## From writing to code Explore ChatCrystal, developer tools and daily learning notes on my GitHub. Questions about the implementation are welcome. [Visit my GitHub](https://github.com/ZengLiangYi) ```json { "@context": "https://schema.org", "@type": "WebSite", "name": "ZengLiangYi Blog", "alternateName": [ "Zeng Liangyi", "ZengLiangYi", "zengliangyi.com" ], "url": "https://zengliangyi.com", "description": "曾亮懿的个人网站。记录 AI Native 探索、开发者工具与全栈产品实践。" } { "@context": "https://schema.org", "@type": "BlogPosting", "headline": "Vectra index recovery: separate state, cleanup, and rebuilding", "description": "Diagnose semantic-search failures across note status, Vectra files, and cleanup tasks, then choose a single-note or full-index recovery path.", "url": "https://zengliangyi.com/en/blog/vectra-index-recovery/", "image": "https://zengliangyi.com/images/blog/vectra-index-recovery.svg", "datePublished": "2026-10-03T00:00:00.000Z", "dateModified": "2026-10-03T00:00:00.000Z", "author": { "@type": "Person", "name": "曾亮懿", "url": "https://zengliangyi.com/en/about" }, "publisher": { "@type": "Organization", "name": "ZengLiangYi Blog", "logo": { "@type": "ImageObject", "url": "https://zengliangyi.com/favicon-96x96.png" } }, "mainEntityOfPage": { "@type": "WebPage", "@id": "https://zengliangyi.com/en/blog/vectra-index-recovery/" } } { "@context": "https://schema.org", "@type": "BreadcrumbList", "itemListElement": [ { "@type": "ListItem", "position": 1, "name": "Home", "item": "https://zengliangyi.com/en" }, { "@type": "ListItem", "position": 2, "name": "Blog", "item": "https://zengliangyi.com/en/blog" }, { "@type": "ListItem", "position": 3, "name": "Vectra index recovery: separate state, cleanup, and rebuilding", "item": "https://zengliangyi.com/en/blog/vectra-index-recovery/" } ] } ```