--- title: "Vectra Vector Search: Metadata, Updates, and Local Cache" description: Inspect Vectra 0.15.0 query and storage paths with temporary indexes, observing external metadata, files left after cancellation, and instance-cache boundaries. image: "https://zengliangyi.com/og/blog/en/vectra-search-internals.png" --- The [fixed-vector exercise](/en/blog/vectra-local-search/) verifies scores, filters, and reopening. This article examines Vectra 0.15.0 storage and update internals: after canceling an update, which states disappear and which files can remain? Those boundaries affect ChatCrystal’s recovery policy. ## The current query implementation does not use the described HNSW graph ChatCrystal’s [lockfile](https://github.com/ZengLiangYi/ChatCrystal/blob/764276814e5c5a33ecbfeae63754f74b9261a3a8/pnpm-lock.yaml) pins Vectra 0.15.0. The [upstream repository](https://github.com/Stevenic/vectra) provides implementation and documentation. These observations come from src/LocalIndex.ts, ItemSelector.ts, and default storage shipped in that npm version, without inferring other versions’ behavior. LocalIndex.queryItems loads the committed items array, applies the supplied metadata filter, traverses candidates to calculate cosine similarity, retains leading results in a bounded min-heap of at most K, and sorts them. Candidate traversal and heap selection are approximately O(N log K). This path does not use the original article’s binary graph/ HNSW hierarchy, m, or efConstruction parameters. Similarity measures vector direction without proving semantic equivalence; this implementation returns NaN for a zero norm. Model identity, vector space, dimensions, and zero vectors need separate checks. This article has no measurement supporting guaranteed millisecond queries over thousands of items. Requiring no separate database service does not make the npm package dependency-free. ## Default JSON layout depends on metadata configuration Default JsonCodec serializes version, metadata_config, and items into index.json. Items include vectors, norms, and metadata, without always creating the described items/ and graph/ directories. With indexed metadata keys configured, the main index retains only those keys. When other keys exist, it writes a UUID-named metadata file and records metadataFile on the item. Queries filter using metadata in the main index before loading full external metadata for selected results in parallel. A field in the returned result was not necessarily available for the earlier filter. ChatCrystal’s [getIndex](https://github.com/ZengLiangYi/ChatCrystal/blob/764276814e5c5a33ecbfeae63754f74b9261a3a8/server/src/services/vector-index.ts) calls default createIndex without the exercise’s indexed configuration. The exercise exposes that library branch. ## Run a canceled update that leaves a metadata file Install `npm install --ignore-scripts --save-exact vectra@0.15.0` in an isolated directory, save lesson.mjs, and run with Node 24+. All files are in a fresh temporary directory cleaned in finally. No Embedding API or actual ChatCrystal data is used. ```js import assert from 'node:assert/strict' ; import { mkdtemp, readFile, readdir, rm } from 'node:fs/promises' ; import { tmpdir } from 'node:os' ; import { join } from 'node:path' ; import { LocalIndex } from 'vectra' ; const folder = await mkdtemp ( join ( tmpdir (), 'vectra-internals-lesson-' )); try { const index = new LocalIndex (folder); await index. createIndex ({ version: 1 , metadata_config: { indexed: [ 'noteId' ] } }); await index. insertItem ({ id: 'kept' , vector: [ 1 , 0 ], metadata: { noteId: 1 , title: 'kept title' }, }); const stored = JSON . parse ( await readFile ( join (folder, 'index.json' ), 'utf8' )); assert. equal (stored.items. length , 1 ); assert. deepEqual (stored.items[ 0 ].metadata, { noteId: 1 }); assert. ok (stored.items[ 0 ].metadataFile); const result = await index. queryItems ([ 1 , 0 ], 'unused without BM25' , 1 , { noteId: 1 }); assert. equal (result[ 0 ].item.metadata.title, 'kept title' ); assert. equal (result[ 0 ].score, 1 ); const before = new Set ( await readdir (folder)); await index. beginUpdate (); const staged = await index. insertItem ({ id: 'canceled' , vector: [ 0 , 1 ], metadata: { noteId: 2 , title: 'staged title' }, }); assert. ok (staged.metadataFile); assert. ok (( await readdir (folder)). includes (staged.metadataFile)); index. cancelUpdate (); assert. equal ( await index. getItem ( 'canceled' ), undefined ); assert. equal (( await new LocalIndex (folder). getIndexStats ()).items, 1 ); const leftovers = ( await readdir (folder)). filter (( name ) => ! before. has (name)); assert. deepEqual (leftovers, [staged.metadataFile]); const writer = new LocalIndex (folder); await writer. insertItem ({ id: 'later' , vector: [ 0 , 1 ], metadata: { noteId: 3 } }); assert. equal (( await index. getIndexStats ()).items, 1 ); assert. equal (( await new LocalIndex (folder). getIndexStats ()).items, 2 ); console. log ( 'metadata files, cancel boundaries, and instance cache pass' ); } finally { await rm (folder, { recursive: true , force: true }); } ``` The committed item stores only noteId in the main file; queryItems restores title in its result. The staged item’s external metadata file is written before endUpdate. After cancelUpdate, neither the committed snapshot nor a reopened index sees canceled, while that file remains. Expect an empty leftovers array and the assertion fails; restore it to pass. After another instance writes later, the old instance’s cache still reports one item while a new instance reads two. Expect two from the old cache and that assertion fails too. This does not reproduce real product concurrency, damaged files, or a crash. Instance cache and disk state are observed separately. ## An update snapshot is not a disk transaction BeginUpdate loads current data and copies the items array. Item objects remain shared references, so upsert replaces objects to avoid mutating the committed snapshot in place. A repeated beginUpdate throws. CancelUpdate clears the staged snapshot without uniformly deleting already written metadata files. EndUpdate passes staged data to storage.upsertFile, then changes the committed memory snapshot. Version 0.15.0’s default LocalFileStorage uses fs.writeFile directly, without a transaction spanning every related file. Custom storage/codec implementations can differ. Method names do not establish crash guarantees across all backends. ## How ChatCrystal limits visibility of cross-store mismatches [GenerateEmbeddings](https://github.com/ZengLiangYi/ChatCrystal/blob/764276814e5c5a33ecbfeae63754f74b9261a3a8/server/src/services/embedding.ts) prepares all vectors before opening an index update. A SQL transaction replaces embeddings mappings and marks syncing; it saves the database, deletes old vectors, ends the index update, then marks done and saves again. Failure attempts to cancel an unfinished index update; after a SQL swap, it marks failed. SQLite and Vectra do not become one atomic transaction. MaterializeDirectSearchHits accepts only notes with database embedding_status=done and keeps the highest-scoring chunk per noteId. Insufficient candidates can expand the retrieval count; lexical matching and score filtering also apply. Relation expansion selects neighbors with confidence≥0.5 and discounts the direct score by 0.7 times confidence. An already added neighbor is not upgraded by a later higher-scoring path, and the final count can exceed direct topK. These results are processed by the application, beyond raw Vectra scores. When reading [index recovery](/en/blog/vectra-index-recovery/), record the main index, metadata files, SQL mappings, and note state separately. Rebuild decisions require those actual states and failure samples, without assuming cancelUpdate rolls back every disk write. Share: ## Related Posts [## AI Conversation Formats: JSONL Events vs SQLite Snapshots Compare 5 AI adapter formats: separate JSONL events from SQLite keys, and inspect indexed, orphan, and unknown-version chat transcripts in memory stores. 曾亮懿 October 7, 2026 4 min read #ChatCrystal #Node.js #SQLite](/en/blog/ai-conversation-format-divergence) [## How Claude Code, Cursor, Copilot, and Trae Store Chats on Disk Compare Claude Code, Cursor, Copilot, and Trae storage paths. Distinguish raw copies from exports, and verify migration retention with test manifests. 曾亮懿 October 7, 2026 4 min read #ChatCrystal #Node.js #SQLite](/en/blog/ai-conversation-management-options) [## sql.js WASM Persistence: Memory, Exports & Nested Savepoints Trace ChatCrystal’s database lifecycle with sql.js 1.14.2 fixtures, separating SQL commits, exports, file writes, foreign-key restoration, and nested rollback. 曾亮懿 October 7, 2026 5 min read #ChatCrystal #Node.js #SQLite](/en/blog/sqljs-wasm-persistence) ## From writing to code Explore ChatCrystal, developer tools and daily learning notes on my GitHub. Questions about the implementation are welcome. [Visit my GitHub](https://github.com/ZengLiangYi) ```json { "@context": "https://schema.org", "@type": "WebSite", "name": "ZengLiangYi Blog", "alternateName": [ "Zeng Liangyi", "ZengLiangYi", "zengliangyi.com" ], "url": "https://zengliangyi.com", "description": "曾亮懿的个人网站。记录 AI Native 探索、开发者工具与全栈产品实践。" } { "@context": "https://schema.org", "@type": "BlogPosting", "headline": "Vectra Vector Search: Metadata, Updates, and Local Cache", "description": "Inspect Vectra 0.15.0 query and storage paths with temporary indexes, observing external metadata, files left after cancellation, and instance-cache boundaries.", "url": "https://zengliangyi.com/en/blog/vectra-search-internals/", "image": "https://zengliangyi.com/images/blog/vectra-search-internals.svg", "datePublished": "2026-10-07T00:00:00.000Z", "dateModified": "2026-10-23T00:00:00.000Z", "author": { "@type": "Person", "name": "曾亮懿", "url": "https://zengliangyi.com/en/about" }, "publisher": { "@type": "Organization", "name": "ZengLiangYi Blog", "logo": { "@type": "ImageObject", "url": "https://zengliangyi.com/favicon-96x96.png" } }, "mainEntityOfPage": { "@type": "WebPage", "@id": "https://zengliangyi.com/en/blog/vectra-search-internals/" } } { "@context": "https://schema.org", "@type": "BreadcrumbList", "itemListElement": [ { "@type": "ListItem", "position": 1, "name": "Home", "item": "https://zengliangyi.com/en" }, { "@type": "ListItem", "position": 2, "name": "Blog", "item": "https://zengliangyi.com/en/blog" }, { "@type": "ListItem", "position": 3, "name": "Vectra Vector Search: Metadata, Updates, and Local Cache", "item": "https://zengliangyi.com/en/blog/vectra-search-internals/" } ] } ```