Five AI conversation parsers: retained fields and loss boundaries

Successful parsing does not establish complete preservation. Five sources can output ParsedConversation while selecting text, tool information, thinking, and order differently. Multi-format parsing covers inputs and database locations; this article focuses on output acceptance, without treating shared fields as a lossless format conversion.

Start with an acceptance matrix for five sources

These observations describe the pinned built-in adapters, not the storage specification of future tool versions.

Source	Input location and text selection	Boundary to observe during acceptance
Claude Code	Project JSONL; user/assistant messages with uuid; strings or content blocks	System messages and known deltas are discarded; tool_result is excluded from text; thinking is separate and tools set a flag
Codex	Rollout JSONL; selected event_msg and response_item branches	User/developer response_items are skipped; full text is compared only with the latest assistant text; calls mark the preceding assistant
Cursor	Workspace ItemTable session index; global cursorDiskKV bubble prefixes and orphan sessions	Type=1 means user, others become assistant; empty assistants are skipped first; new _v only warns; timestamp sorting follows
Trae	ItemTable memento/icube-ai-agent-storage; sessionId selects data.list entry	turnIndex sorting; direct text and nested agent content have precedence; timestamps can be synthetic
Copilot	Entire legacy .json; literal first-line snapshot in .jsonl	Later patches are not replayed; response blocks map separately to text, thinking, and tool flags

Cursor and Trae do not share a SQL-value schema, JSON shape, or session lookup. Shared fields reduce downstream branching without replacing source-specific sample tests.

Tool flags, thinking, and empty messages differ

Claude Code tool blocks set hasToolUse without preserving the complete tool input/result trace. A message with only that flag can survive without text or thinking. Tags such as system-reminder are removed from text. This extraction policy does not prove the removed information has no value for every use.

Cursor skips empty assistant text before extracting thinking or tool flags, so a thinking-only assistant bubble can be lost too. HasCode can come from code fences or codeBlocks/suggestedCodeBlocks arrays, rather than text alone.

Trae prefers direct content; otherwise it combines nested content such as proposal and finish with thinking. Final retention requires text or thinking; a tool flag alone is insufficient. Copilot likewise adds an assistant message only when text or thinking has a value. Claude Code’s tool-only behavior does not apply to every source.

Codex deduplication and ordering have defined scopes

The Codex parser extracts user input from event_msg/user_message and assistant text from agent_message. Response_item/message combines only assistant output_text. If the full text equals lastAssistantMsg.content, that response_item is skipped. There is no deduplication set for every event or all historical text.

Function_call/custom_tool_call sets hasToolUse on the latest assistant. Without one, it does not create a tool-call message. Unhandled branches such as reasoning do not populate thinking; this does not establish that all raw reasoning is absent or encrypted. Missing timestamps use the current time, and final sorting does not reconstruct every event’s causal relationship.

Trae first sorts by turnIndex, then keeps output timestamps monotonic. Those times can be constructed ordering markers, unsuitable for measuring actual reading or model-response duration. Preserve raw event position or source samples when checking order; normalized timestamp alone is insufficient.

Test snapshots and response blocks with a Copilot sample

Save as lesson.mjs and run directly with Node 24+. This teaching extractor covers only first-line wrapping, selected response blocks, and assistant retention. It is not the complete Copilot adapter: there is no file scanning, user-message handling, timestamp processing, database, or runtime schema check.

import assert from 'node:assert/strict';

function firstSnapshot(lines) {
  try {
    const snapshot = JSON.parse(lines.split('\n')[0]);
    if (snapshot.kind !== undefined && snapshot.kind !== 0) return null;
    return snapshot.v ?? snapshot;
  } catch {
    return null;
  }
}

function assistant(response) {
  const texts = [];
  const thoughts = [];
  let hasToolUse = false;
  for (const item of response) {
    if (item.kind === 'thinking' && item.value) thoughts.push(item.value);
    else if ((item.kind === 'text' || item.kind === undefined) && item.value)
      texts.push(item.value);
    else if (item.kind === 'toolInvocationSerialized' || item.kind === 'prepareToolInvocation')
      hasToolUse = true;
  }
  const content = texts.join('\n').trim();
  const thinking = thoughts.length ? thoughts.join('\n') : null;
  return content || thinking ? { content, thinking, hasToolUse } : null;
}

const snapshot = {
  kind: 0,
  v: {
    requests: [
      {
        response: [
          { kind: 'text', value: 'first answer' },
          { kind: 'thinking', value: 'fixture thought' },
          { kind: 'prepareToolInvocation' },
          { kind: 'inlineReference', value: 'not response text' },
        ],
      },
    ],
  },
};
const patch = {
  kind: 1,
  v: { requests: [{ response: [{ kind: 'text', value: 'later answer' }] }] },
};
const lines = JSON.stringify(snapshot) + '\n' + JSON.stringify(patch);
const selected = firstSnapshot(lines);
assert.notEqual(selected, null);
assert.deepEqual(assistant(selected.requests[0].response), {
  content: 'first answer',
  thinking: 'fixture thought',
  hasToolUse: true,
});
assert.equal(firstSnapshot('broken\n' + JSON.stringify(snapshot)), null);
assert.equal(firstSnapshot(JSON.stringify(patch)), null);
assert.equal(assistant([{ kind: 'prepareToolInvocation' }]), null);
assert.deepEqual(assistant([{ kind: 'thinking', value: 'thought only' }]), {
  content: '',
  thinking: 'thought only',
  hasToolUse: false,
});
console.log('snapshot and selected response boundaries pass');

Set hasToolUse to false in its assignment and the mixed-response assertion fails. Select the last line rather than the first and the initial-snapshot assertion fails too. Restoring the code passes. This verifies the choices for these samples, without proving later patches never affect conversation content.

Record missing content as well as message counts

Prepare normal text, tool-only, thinking-only, mixed blocks, empty responses, repeated text, missing timestamps, malformed JSON, and unknown-version samples for each source. Record expected text, role, tool flags, thinking, order, identity, and reasons for omission. Claude/Codex can skip malformed JSON lines; a damaged first line in Copilot can instead produce an empty conversation. These are not one universal recovery policy.

Parsed output also reaches the import pipeline, including its fewer-than-two-message filter and content comparison. Distinguish a parser discarding a message, an importer skipping a conversation, and a database remaining unchanged. A structurally valid output JSON establishes its shape, without proving source compatibility or completeness.