AI tools can store conversations as JSONL, event streams, or SQLite key-value data. A shared import pipeline needs clear responsibilities for discovering data, deciding whether it changed, and understanding its format. This article reads ChatCrystal’s SourceAdapter interface, then uses fixtures to verify the control flow that avoids parsing unchanged candidates.
Share output contracts while retaining format differences
adapter.ts defines name, displayName, optional parserVersion, and three asynchronous methods. The core excerpt below depends on shared types and cannot compile alone.
export interface SourceAdapter {
readonly name: string;
readonly displayName: string;
readonly parserVersion?: string;
detect(): Promise<SourceInfo | null>;
scan(): Promise<ConversationMeta[]>;
parse(meta: ConversationMeta): Promise<ParsedConversation>;
}detect returns source information or null; scan returns candidate metadata such as identity, path, size, and modification time; parse produces a normalized conversation and messages. Shared output lets the import service reuse persistence logic, but a TypeScript interface does not validate external JSON at runtime.
Do not assume every detect implementation performs one directory-existence check. The Claude Code adapter enumerates project directories and JSONL files to count conversations; scan also skips files below 100 bytes. Read a method’s responsibility separately from the I/O its implementation performs.
Registration does not establish enablement
registry.ts stores adapters in a Map, warning and overwriting on duplicate names. The current module entry registers five built-in sources: Claude Code, Codex, Cursor, Trae, and Copilot. Registration happens statically at module import; it is not an interface for uploading arbitrary plugins at runtime.
detectAllSources discovers all registered sources concurrently. importAll first filters by enabledSources, then discovers and scans sequentially. These paths differ. Adding an adapter to the Map also requires checking enablement; otherwise the import path may never call it.
Verify selective parsing with a fixture adapter
Save this as lesson.mjs and run with Node 24+ without dependencies. It is a teaching control flow, modeling only enablement, discovery, metadata comparison, and parsing counts. It is not the complete importAll and has no database, content hash, or replacement policy.
import assert from 'node:assert/strict';
const calls = [];
const registry = new Map();
registry.set('fixture', {
name: 'fixture',
async detect() {
calls.push('detect');
return { name: 'fixture' };
},
async scan() {
calls.push('scan');
return [
{ id: 'unchanged', fileSize: 120, fileMtime: 't1' },
{ id: 'changed', fileSize: 240, fileMtime: 't2' },
];
},
async parse(meta) {
calls.push('parse:' + meta.id);
return { id: meta.id };
},
});
const previous = new Map([['unchanged', { fileSize: 120, fileMtime: 't1' }]]);
async function collect(enabled) {
const results = [];
for (const adapter of registry.values()) {
if (!enabled.includes(adapter.name)) continue;
if (!(await adapter.detect())) continue;
for (const meta of await adapter.scan()) {
const old = previous.get(meta.id);
if (old?.fileSize === meta.fileSize && old?.fileMtime === meta.fileMtime) continue;
results.push(await adapter.parse(meta));
}
}
return results;
}
assert.deepEqual(await collect([]), []);
assert.deepEqual(calls, []);
assert.deepEqual(await collect(['fixture']), [{ id: 'changed' }]);
assert.deepEqual(calls, ['detect', 'scan', 'parse:changed']);
console.log('enabled source and parse boundaries pass');Change 120 to 121 in previous and the previously unchanged candidate is parsed, failing the result assertion. Restore it and the program passes. This verifies the metadata shortcut, without establishing that identical size and timestamp always mean identical content.
The actual importer compares content too
The real importAll first skips unchanged size/time pairs, then computes a normalized content hash after parsing. An existing conversation whose file metadata changed but content hash stayed equal receives a metadata update. Changed content follows the replacement policy. Conversations with fewer than two parsed messages are skipped.
Failures have different scopes too. Discovery and scanning sit outside the per-candidate try/catch: a thrown detect or scan error terminates that importAll call. Candidate parsing sits inside the guard, so a thrown parse error increments the error count and allows later candidates to continue. Per-candidate recovery does not establish source-wide fault isolation.
The fast path still has a limit: unchanged size and timestamp prevent reparsing, so this is not a content check on every import. parserVersion records the parsing contract; its optional interface field does not establish that changing the version forces reimport automatically. See bulk import measurement for persistence and replacement details.
Check the whole path when adding a source
A reliable integration needs reproducible format samples, identity rules, ordering and noise handling, and tests of missing and damaged data. Implementing and registering three methods is only part of the work. Also check the configuration defaults and migration, path settings, interface entry points, remote transfer, and watchers for recognition of the new name.
The current configuration restores built-in sources when loading persisted settings. That does not support a promise that disablement survives every restart scenario. Copying another tool’s default directory does not establish compatibility either. Check detect, scan, and parse separately with synthetic or authorized samples, then check persistence and failures through the shared importer.
The interface reduces the influence of format differences on the main pipeline. It does not automatically eliminate identity collisions, omitted configuration, or parsing errors. Integration evidence should come from samples and counterexamples through the complete import path rather than interface type checking alone.