SourceAdapter internals: identity, content hashes, and reimport

After sharing three methods, a new source can still fail at identity, change detection, or remote transfer. The SourceAdapter introduction explains responsibilities; this article follows the pinned registry, importer, and remote payload to examine how a candidate reaches persistence.

Shared output does not share every source assumption

adapter.ts declares name, displayName, optional parserVersion, and asynchronous detect, scan, and parse. Scan returns id, source, filePath, fileSize, fileMtime, and projectDir; parse returns a conversation and normalized messages. The shared types define their fields. The interface itself does not validate external JSON.

“Scan only stats files, without reading content” does not describe every adapter. Cursor scanning reads workspace databases to find conversations and combines them with the global database. Trae decodes the session list in ItemTable. Metadata is the output contract, not evidence of the scan’s I/O cost.

Registration, discovery, and caching have separate boundaries

registry.ts uses a Map, warning and overwriting on duplicate names. index.ts statically registers five built-in sources. There is no protocol for uploading plugins dynamically.

The registry’s detectAllSources discovers sources concurrently with Promise.all. One rejected discovery rejects that aggregate call; it does not collect each source’s failure outcome. ImportAll instead filters enabledSources, calls detect and scan sequentially, and collects all candidates before parsing them individually. Its detect and scan calls are outside the per-candidate try/catch.

The Cursor adapter caches workspace scan results, expiring five seconds after scanning completes. This can reduce repeated work between adjacent detect and scan calls. It is not a conversation-content cache and does not cache an in-flight Promise. The TTL establishes neither concurrent request deduplication nor real-time data freshness.

Metadata, content, and parser versions answer different questions

ImportAll first queries file size, modification time, and content hash for an existing id/source. Matching size and time cause a skip before parse runs. Merely increasing parserVersion therefore does not guarantee reparsing, and matching metadata is not proof of identical content.

After parsing, results with fewer than two messages are skipped. An existing conversation with an equal content hash takes the metadata-update path; a changed hash takes replacement; a new conversation takes insertion. These database changes use SQL transactions, followed by saveDatabase at the end of the batch. SQL rollback, database-file persistence, and vector cleanup completion still need separate acceptance checks.

The hash input in importPayload.ts consists of the source and ordered message fields: index, type, role, content, hasToolUse, hasCode, and thinking. Message IDs, timestamps, conversation titles, and parserVersion are excluded. This is the implementation’s comparison policy, not detection of every change in the raw file.

Verify identity and hashes with offline samples

Save as lesson.mjs and run with Node 24+, without dependencies. This reproduces the hash field selection and remote-message naming rule. It has no database, file scanner, or complete remote-import validator.

import assert from 'node:assert/strict';
import { createHash } from 'node:crypto';

function contentHash(parsed) {
  const canonical = {
    source: parsed.source,
    messages: parsed.messages.map((message, index) => ({
      index,
      type: message.type,
      role: message.role,
      content: message.content,
      hasToolUse: message.hasToolUse,
      hasCode: message.hasCode,
      thinking: message.thinking,
    })),
  };
  return createHash('sha256').update(JSON.stringify(canonical)).digest('hex');
}

function namespace(source, id) {
  return id.startsWith(source + ':') ? id : source + ':' + id;
}

function remoteMessages(conversationId, messages) {
  const ids = new Map(messages.map((m) => [m.id, conversationId + ':' + m.id]));
  return messages.map((m) => ({
    ...m,
    id: ids.get(m.id),
    parentUuid: m.parentUuid
      ? (ids.get(m.parentUuid) ?? conversationId + ':' + m.parentUuid)
      : null,
  }));
}

const parsed = {
  source: 'codex',
  messages: [
    {
      id: 'm1',
      parentUuid: null,
      type: 'user',
      role: 'user',
      content: 'question',
      hasToolUse: false,
      hasCode: false,
      thinking: null,
      timestamp: '2026-01-01T00:00:00Z',
    },
    {
      id: 'm2',
      parentUuid: 'm1',
      type: 'assistant',
      role: 'assistant',
      content: 'answer',
      hasToolUse: true,
      hasCode: false,
      thinking: null,
      timestamp: '2026-01-01T00:00:01Z',
    },
  ],
};
const changedIds = structuredClone(parsed);
changedIds.messages[0].id = 'different';
changedIds.messages[0].timestamp = '2026-02-01T00:00:00Z';
assert.equal(contentHash(changedIds), contentHash(parsed));
const changedThinking = structuredClone(parsed);
changedThinking.messages[1].thinking = 'fixture reasoning';
assert.notEqual(contentHash(changedThinking), contentHash(parsed));
assert.notEqual(
  contentHash({ ...parsed, messages: [...parsed.messages].reverse() }),
  contentHash(parsed)
);
const id = namespace('codex', '7');
assert.equal(namespace('codex', id), id);
assert.notEqual(namespace('cursor', '7'), id);
const normalized = remoteMessages(id, parsed.messages);
assert.equal(normalized[1].parentUuid, normalized[0].id);
assert.equal(contentHash({ ...parsed, messages: normalized }), contentHash(parsed));
console.log('identity and canonical hash boundaries pass');

Delete thinking from canonical and the changed-sample assertion fails; restore it to pass. Then replace parentUuid with the raw parent ID and the relationship assertion fails. These counterexamples cover field selection and identity rewriting separately.

Remote naming does not guarantee collision-free arbitrary inputs

The remote payload preserves sourceConversationId and generates a source-prefixed conversationId. Message IDs and nonempty parentUuid values also receive the conversation prefix. An already matching source prefix remains unchanged, so 7 and codex:7 map to the same result. This rule is idempotent, without establishing a one-to-one mapping for every arbitrary raw ID.

Local importAll and remote normalizeParsedConversationForRemote are different entry points. Do not describe remote namespacing as a step performed by every local import. A new source also needs checks of the shared ChatCrystalSource union, SUPPORTED_IMPORT_SOURCES, configuration defaults, remote payload validation, and watcher/UI paths. Existing configuration loading also restores missing built-in sources.

Treat reimport as a set of acceptance scenarios

Check content changes with unchanged size and time, time-only changes, parser-version-only changes, identical raw IDs across sources, duplicate registration, discovery or scan rejection, and individual parsing rejection separately. The per-candidate catch can record parsing failures and continue, but error logging itself depends on a working database. It does not establish unconditional fault tolerance.

A source integration is complete when discovery, enablement, identity, parsing, change policy, and persistence are covered. Implementing three methods and registering them are only part of that work. Continue with multi-format parsing for concrete formats and loss boundaries.