--- title: "Semantic Search: Candidate Deduplication and Graph Context" description: Trace query vectors, candidate growth, lexical boosts, and relation expansion in ChatCrystal, distinguishing list filtering from semantic retrieval. image: "https://zengliangyi.com/og/blog/en/semantic-search-workflow.png" --- Five returned notes do not mean five vector candidates, and their scores are not necessarily unadjusted cosine similarities. Explain missed results and ordering by separating application stages from API fields. This article follows the [ChatCrystal embedding service](https://github.com/ZengLiangYi/ChatCrystal/blob/764276814e5c5a33ecbfeae63754f74b9261a3a8/server/src/services/embedding.ts) at [`7642768`](https://github.com/ZengLiangYi/ChatCrystal/tree/764276814e5c5a33ecbfeae63754f74b9261a3a8). Constants describe that version’s rules, not universally optimal retrieval parameters. ## Two entrances serve different questions `/api/notes?search=...` applies SQL LIKE to titles and summaries, useful for checking existing terms. `/api/search?q=...` embeds a query and retrieves note candidates. LIKE also has wildcard semantics; arbitrary input is not always literal matching. CLI examples on the default local connection: ```bash crystal --json notes list --search SQLite --limit 5 crystal --json search "SQLite concurrent writes" --limit 5 ``` The first JSON response is a list object containing `items`; the second is a candidate array. The JSON branch forwards the limit instead of using the TTY search interface’s own fixed count. Eligible results can still be fewer than the upper bound. ## From note text to candidates Notes are embedded from selected fields; queries use the current embedding configuration. Both need matching settings. Follow the [embedding configuration guide](/en/blog/embedding-model-configuration) when changing models. Paragraph grouping produces multiple chunks, so a note can have several vectors. [Vectra v0.15.0 LocalIndex](https://github.com/Stevenic/vectra/blob/v0.15.0/src/LocalIndex.ts) scores candidates and retains top K. This path does not use HNSW or LSH approximate candidate lookup. Evaluate cost and quality against actual library size and models; a local index does not establish zero cost or fixed response time. ## Database checks, deduplication, and ranking The application reads database chunks corresponding to candidates and requires note status `done`. An index entry alone does not establish an eligible result. Multiple chunks are deduplicated by note, keeping the higher score. Term processing includes ASCII technical words and Han-character bigrams. A candidate containing every extracted term receives a score floor of `0.82`; otherwise candidates below `0.58` are filtered. Boosting occurs after materializing vector candidates. It is not a separate keyword fallback scanning the entire library. If checks, deduplication, or filtering leave too few notes, the code progressively grows candidate count, bounded by available index count or stopping conditions. This can add candidates without guaranteeing the requested result count or inclusion of the semantically correct note. ## Compare relation expansion through the API Use `--data-urlencode` instead of manually assembling queries with special characters: ```bash curl --get http://localhost:3721/api/search \ --data-urlencode 'q=SQLite concurrent writes' \ --data-urlencode 'limit=5' ``` Optionally enable relation expansion: ```bash curl --get http://localhost:3721/api/search \ --data-urlencode 'q=SQLite concurrent writes' \ --data-urlencode 'limit=5' \ --data-urlencode 'expand=true' ``` Current expansion follows one-hop relations from direct hits, requiring confidence at least `0.5`. Linked scores use `direct score × 0.7 × confidence`. This is not multihop path search. Notes already in the result set are skipped, so it does not choose the strongest path for every linked result. Expansion can produce more notes than the direct-hit limit. ## Interpret response fields correctly The [HTTP route](https://github.com/ZengLiangYi/ChatCrystal/blob/764276814e5c5a33ecbfeae63754f74b9261a3a8/server/src/routes/notes.ts) wraps the array in `data`, returning `note_id`, `conversation_id`, title, project, score, tags, and `via_relation`. The last field marks a relation type, not a complete path or proof of correct source content. CLI arrays are unwrapped and do not require `.data[]`. Read the body for `note_id`, then check the original conversation and current task. Compare the two entrances and expansion switch using known-answer, paraphrase, exact-term, and no-answer queries. Record false matches instead of showing only successful examples. See [debugging-solution recovery](/en/blog/recover-debugging-notes) for reuse and [RAG and knowledge graphs](/en/blog/rag-and-knowledge-graphs) for evidence boundaries. Share: ## Related Posts [## Recover a debugging solution: retrieve, read, then verify reuse Use a constructed debugging scenario to check notes, vectors, and search results, then compare the original conversation with the current project. 曾亮懿 October 3, 2026 4 min read #ChatCrystal #Semantic search #Debugging](/en/blog/recover-debugging-notes) [## Semantic Search and Context: Finding Hidden Code Dependencies Combine vector search with one-hop graph relation expansion in code refactoring, keeping similarity scores, candidates, and sources verifiable on disk. 曾亮懿 October 3, 2026 3 min read #ChatCrystal #Semantic search #Knowledge graph](/en/blog/semantic-search-relation-context) [## From RAG to Knowledge Graphs: Preserving Context in Retrieval Compare keyword search, vectors, and knowledge graphs. Trace one-hop candidate expansion in ChatCrystal and identify boundaries relationships cannot prove. 曾亮懿 June 5, 2026 5 min read #ChatCrystal #RAG #Knowledge graphs #Retrieval](/en/blog/rag-and-knowledge-graphs) ## From writing to code Explore ChatCrystal, developer tools and daily learning notes on my GitHub. Questions about the implementation are welcome. [Visit my GitHub](https://github.com/ZengLiangYi) ```json { "@context": "https://schema.org", "@type": "WebSite", "name": "ZengLiangYi Blog", "alternateName": [ "Zeng Liangyi", "ZengLiangYi", "zengliangyi.com" ], "url": "https://zengliangyi.com", "description": "曾亮懿的个人网站。记录 AI Native 探索、开发者工具与全栈产品实践。" } { "@context": "https://schema.org", "@type": "BlogPosting", "headline": "Semantic Search: Candidate Deduplication and Graph Context", "description": "Trace query vectors, candidate growth, lexical boosts, and relation expansion in ChatCrystal, distinguishing list filtering from semantic retrieval.", "url": "https://zengliangyi.com/en/blog/semantic-search-workflow/", "image": "https://zengliangyi.com/images/blog/semantic-search-workflow.svg", "datePublished": "2026-10-03T00:00:00.000Z", "dateModified": "2026-10-23T00:00:00.000Z", "author": { "@type": "Person", "name": "曾亮懿", "url": "https://zengliangyi.com/en/about" }, "publisher": { "@type": "Organization", "name": "ZengLiangYi Blog", "logo": { "@type": "ImageObject", "url": "https://zengliangyi.com/favicon-96x96.png" } }, "mainEntityOfPage": { "@type": "WebPage", "@id": "https://zengliangyi.com/en/blog/semantic-search-workflow/" } } { "@context": "https://schema.org", "@type": "BreadcrumbList", "itemListElement": [ { "@type": "ListItem", "position": 1, "name": "Home", "item": "https://zengliangyi.com/en" }, { "@type": "ListItem", "position": 2, "name": "Blog", "item": "https://zengliangyi.com/en/blog" }, { "@type": "ListItem", "position": 3, "name": "Semantic Search: Candidate Deduplication and Graph Context", "item": "https://zengliangyi.com/en/blog/semantic-search-workflow/" } ] } ```