--- title: "Testing Strategy: Counterexamples for SQL, Functions & APIs" description: Check ChatCrystal’s Node test entry points, practice dependency isolation, SQL rollback, and Fastify inject offline, and state what those tests do not cover. image: "https://zengliangyi.com/og/blog/en/testing-strategy.png" --- A test earns its value by detecting a specific regression, rather than adding a green tick. Conversation knowledge stores cross parsing, SQL, vector indexes, HTTP, and model calls. Organize fixtures around failure boundaries. This article checks ChatCrystal’s actual test entry points, then practices functions, dependency isolation, rollback, and route checks in one independent teaching file. ## Current testing has four entry paths The [root scripts](https://github.com/ZengLiangYi/ChatCrystal/blob/764276814e5c5a33ecbfeae63754f74b9261a3a8/package.json) run server, client, electron, and audit-policy sequentially for pnpm test. That is not one `src/**/*.test.ts` glob. Server tests run node:test through tsx; client and Electron have their own test directories. Audit-policy uses a Node command with file isolation disabled. | Path | Current command or mechanism | Observable scope | | --- | --- | --- | | server | `tsx --test src/**/*.test.ts` | Service, database, parser, and route contracts | | client | `tsx --test client/test/*.test.ts` | API wrappers, static source, language, and interface conventions | | electron | `tsx --test electron/test/*.test.ts` | Desktop state, navigation, updates, and wiring | | audit-policy | `node --test --test-isolation=none` | Policy decisions over audit results | node:test and assert are built in, but the test chain also depends on tsx, sql.js, Fastify, and other packages. The entire suite is not dependency-free. Client source assertions are not real browser interaction tests either. The project’s engine requirement is Node 24. A testing pyramid helps consider cost, but no evidence establishes a mandatory unit-test percentage for this project. Prioritize destructive data failures, state transitions, and external dependency failures, then add checks across boundaries based on regression evidence. ## Find counterexamples worth preserving in existing tests [embedding.test.ts](https://github.com/ZengLiangYi/ChatCrystal/blob/764276814e5c5a33ecbfeae63754f74b9261a3a8/server/src/services/embedding.test.ts) checks structured text fields and malformed JSON, as well as Vectra staged and committed state. “Does not throw” alone is insufficient: assert what survives and what stays out of index text. The chunking value 500 is a soft threshold; a long paragraph can exceed it. See [embedding service design](/en/blog/embedding-service-design/). [transaction.test.ts](https://github.com/ZengLiangYi/ChatCrystal/blob/764276814e5c5a33ecbfeae63754f74b9261a3a8/server/src/db/transaction.test.ts) checks commits, rollbacks, and nested savepoints separately. A transaction function’s name does not establish that failures leave no partial data. Route tests use [Fastify inject](https://fastify.dev/docs/latest/Guides/Testing/) to register routes and inspect responses without listening on a port. This does not cover real networks, reverse proxies, or Electron lifecycles. Some service functions accept replaceable dependencies to avoid models and real indexes. The project also directly replaces console.log or globalThis.fetch and restores them in finally, as in [registry.test.ts](https://github.com/ZengLiangYi/ChatCrystal/blob/764276814e5c5a33ecbfeae63754f74b9261a3a8/server/src/parser/registry.test.ts) and [client API tests](https://github.com/ZengLiangYi/ChatCrystal/blob/764276814e5c5a33ecbfeae63754f74b9261a3a8/client/test/api.test.ts). Thus “all dependencies are injected, with no global replacement” is not current behavior. Isolation and restoration matter more than absolute claims about mock styles. ## Write a complete offline teaching test Install pinned dependencies in a new directory, save lesson.test.mjs, and run with Node 24+. These four tests use **teaching functions, a teaching table, and a teaching route, rather than actual ChatCrystal services**. They access no real model, personal database, imported file, or listening port. ```sh npm init -y npm install --ignore-scripts --no-audit --no-fund sql.js@1.14.2 fastify@5.12.3 node --test lesson.test.mjs ``` ```js import assert from 'node:assert/strict' ; import test from 'node:test' ; import Fastify from 'fastify' ; import initSqlJs from 'sql.js' ; const normalize = ( text ) => text. trim (). replace ( / \s + / g , ' ' ); async function summarizeIfAccepted ( accepted , summarize ) { if ( ! accepted) return null ; return summarize (); } test ( 'normalization preserves content and handles whitespace' , () => { assert. equal ( normalize ( ' first \n second ' ), 'first second' ); assert. equal ( normalize ( '' ), '' ); }); test ( 'rejected input never reaches the model dependency' , async () => { const result = await summarizeIfAccepted ( false , async () => { throw new Error ( 'unexpected model call' ); }); assert. equal (result, null ); assert. equal ( await summarizeIfAccepted ( true , async () => 'fixture' ), 'fixture' ); }); test ( 'a failed SQL transaction leaves no inserted row' , async () => { const SQL = await initSqlJs (); const db = new SQL . Database (); try { db. run ( 'CREATE TABLE items (value TEXT)' ); assert. throws (() => { db. run ( 'BEGIN' ); try { db. run ( "INSERT INTO items VALUES ('temporary')" ); throw new Error ( 'fixture failure' ); } catch (error) { db. run ( 'ROLLBACK' ); throw error; } }, / fixture failure / ); assert. equal (db. exec ( 'SELECT COUNT(*) FROM items' )[ 0 ].values[ 0 ][ 0 ], 0 ); } finally { db. close (); } }); test ( 'inject checks one route without listening on a port' , async () => { const app = Fastify (); app. get ( '/lesson' , async () => ({ success: true , data: [ 'fixture' ] })); try { const response = await app. inject ({ method: 'GET' , url: '/lesson' }); assert. equal (response.statusCode, 200 ); assert. deepEqual (response. json ().data, [ 'fixture' ]); assert. equal (( await app. inject ({ method: 'GET' , url: '/missing' })).statusCode, 404 ); } finally { await app. close (); } }); ``` The file should produce four passing tests. Remove ROLLBACK and the failed transaction leaves temporary behind, making the zero-row assertion fail. Restore it and the tests pass again. Alternatively, make the rejected branch call summarize and verify that the unexpected dependency call fails the test. Exercising real failure paths establishes whether assertions check the intended behavior. This minimal SQL test does not implement ChatCrystal’s nested transactions or verify disk persistence. Test actual withTransaction using existing project tests and additional failure fixtures. A rollback in one database also cannot establish consistency across vectors, files, and SQL. ## Isolate environments and fixtures, then clean up Some modules read DATA_DIR when imported. Actual [import service tests](https://github.com/ZengLiangYi/ChatCrystal/blob/764276814e5c5a33ecbfeae63754f74b9261a3a8/server/src/services/import.test.ts) set a temporary directory before dynamic imports. [Tags route tests](https://github.com/ZengLiangYi/ChatCrystal/blob/764276814e5c5a33ecbfeae63754f74b9261a3a8/server/src/routes/tags.test.ts) clean up connections and the temporary directory. Using a default path and deleting tables crosses the isolation boundary. Fixtures should specify IDs, times, and roles, avoiding dependence on execution order. Restore registry, environment, global fetch, and timer changes. File isolation differs from shared state inside a file. WASM lookup also needs to match the dependency layout: the project’s [sql.test-helper.ts](https://github.com/ZengLiangYi/ChatCrystal/blob/764276814e5c5a33ecbfeae63754f74b9261a3a8/server/src/test-utils/sql.test-helper.ts) uses require.resolve rather than assuming a fixed relative node_modules path. ## State what CI success proves The release workflow runs pnpm test before building. Passing existing tests establishes only that covered fixtures did not fail. It cannot rule out unknown regressions or replace packaged startup and interaction checks. Model quality requires separate reproducible samples; a successful mock does not establish model quality. This article runs only the independent four-test lesson and its counterexamples. It does not run the entire application suite or produce timing benchmarks. For an actual change, execute project checks around affected boundaries and record remaining network, file, model, and desktop coverage limits. Share: ## Related Posts [## Five AI conversation parsers: retained fields and loss boundaries Build an acceptance matrix from Claude Code, Codex, Cursor, Trae, and Copilot parsers, checking tool flags, thinking, order, and snapshot coverage. 曾亮懿 October 7, 2026 4 min read #ChatCrystal #TypeScript #Node.js](/en/blog/ai-conversation-parser-boundaries) [## AI SDK Provider Abstraction: Capabilities and Protocols Test 7 ChatCrystal providers and AI SDK factories with offline mocks for Responses, Chat Completions, embeddings, and structured output rejection. 曾亮懿 October 7, 2026 4 min read #ChatCrystal #Node.js #TypeScript](/en/blog/ai-sdk-provider-abstraction) [## SourceAdapter internals: identity, content hashes, and reimport Trace ChatCrystal’s registry, importer, and remote payload to verify identity, canonical hashes, metadata shortcuts, and parser-version boundaries. 曾亮懿 October 7, 2026 4 min read #ChatCrystal #TypeScript #Node.js](/en/blog/source-adapter-internals) ## From writing to code Explore ChatCrystal, developer tools and daily learning notes on my GitHub. Questions about the implementation are welcome. [Visit my GitHub](https://github.com/ZengLiangYi) ```json { "@context": "https://schema.org", "@type": "WebSite", "name": "ZengLiangYi Blog", "alternateName": [ "Zeng Liangyi", "ZengLiangYi", "zengliangyi.com" ], "url": "https://zengliangyi.com", "description": "曾亮懿的个人网站。记录 AI Native 探索、开发者工具与全栈产品实践。" } { "@context": "https://schema.org", "@type": "BlogPosting", "headline": "Testing Strategy: Counterexamples for SQL, Functions & APIs", "description": "Check ChatCrystal’s Node test entry points, practice dependency isolation, SQL rollback, and Fastify inject offline, and state what those tests do not cover.", "url": "https://zengliangyi.com/en/blog/testing-strategy/", "image": "https://zengliangyi.com/images/blog/testing-strategy.svg", "datePublished": "2026-10-06T00:00:00.000Z", "dateModified": "2026-10-23T00:00:00.000Z", "author": { "@type": "Person", "name": "曾亮懿", "url": "https://zengliangyi.com/en/about" }, "publisher": { "@type": "Organization", "name": "ZengLiangYi Blog", "logo": { "@type": "ImageObject", "url": "https://zengliangyi.com/favicon-96x96.png" } }, "mainEntityOfPage": { "@type": "WebPage", "@id": "https://zengliangyi.com/en/blog/testing-strategy/" } } { "@context": "https://schema.org", "@type": "BreadcrumbList", "itemListElement": [ { "@type": "ListItem", "position": 1, "name": "Home", "item": "https://zengliangyi.com/en" }, { "@type": "ListItem", "position": 2, "name": "Blog", "item": "https://zengliangyi.com/en/blog" }, { "@type": "ListItem", "position": 3, "name": "Testing Strategy: Counterexamples for SQL, Functions & APIs", "item": "https://zengliangyi.com/en/blog/testing-strategy/" } ] } ```