Building an Embedding Service: Text, Chunks, and Commits

The text an embedding service accepts, its chunking rules, and the point where it replaces the old index affect retrieval and failure recovery. This article separates those responsibilities in ChatCrystal’s pinned implementation, then verifies chunk and vector boundaries without calling a model.

Decide which text becomes searchable

This article follows embedding.ts. The service deduplicates and joins the title, summary, key conclusions, tags, and code-snippet descriptions. For ordinary imported-conversation notes, that does not put the raw conversation and every code body into the vector input.

Two memory sources, agent-writeback and manual-note, have additional rules: they include code bodies and available fields for root causes, resolutions, pitfalls, reusable patterns, decisions, error signatures, and file paths. Each code body has consecutive whitespace collapsed before being limited to 1000 JavaScript character units. This distinction affects which notes can be retrieved by code content. Code bodies cannot be treated as universally excluded from vector input.

Write down the actual input before choosing a model. Switching embedding models cannot directly recover information removed from that input. See embedding configuration for provider capabilities and settings.

500 is a merging target, not a hard limit

The chunker uses CHUNK_SIZE = 500, splits at blank lines, and tries to merge adjacent paragraphs. JavaScript .length counts UTF-16 code units rather than tokens. It does not guarantee that every chunk stays below 500: a 700-unit paragraph without blank lines remains one chunk. There is no overlap window either.

The following standalone JavaScript version preserves the current algorithm’s decision order while simplifying variable names. Save it as lesson.mjs in a temporary directory and run with Node 24+. It needs no dependencies and accesses no knowledge-base data.

import assert from 'node:assert/strict';

function chunkText(text) {
  const size = 500;
  if (text.length <= size) return [text];
  const chunks = [];
  let current = '';
  for (const paragraph of text.split(/\n\n+/)) {
    if (current.length + paragraph.length + 2 > size && current.length > 0) {
      chunks.push(current.trim());
      current = paragraph;
    } else {
      current += (current ? '\n\n' : '') + paragraph;
    }
  }
  if (current.trim()) chunks.push(current.trim());
  return chunks;
}

assert.deepEqual(
  chunkText('a'.repeat(700)).map((s) => s.length),
  [700]
);
assert.deepEqual(
  chunkText('a'.repeat(300) + '\n\n' + 'b'.repeat(300)).map((s) => s.length),
  [300, 300]
);
assert.equal('😀'.length, 2);
console.log('chunk boundaries pass');

Change the first expectation to [500, 200] and the program fails; restore [700] and it passes. This establishes that the implementation does not hard-split long paragraphs. It does not establish compliance with any model’s token limit. A strict limit would need model-specific counting, long-paragraph splitting, and corresponding counterexamples; these are proposed improvements.

Generate every vector before starting the update

The service calls AI SDK embed sequentially and prepares every chunk’s result before opening a Vectra update. If a model call fails beforehand, this replacement does not begin. The factories in providers.ts create models through embeddingModel(model); generation and embedding settings are separate.

When designing a service, you can add an explicit vector contract before committing. Append the following to the same file. It is a teaching design, not an existing application validator. embedOne is an injected fake model and commit is a callback.

async function stageVectors(texts, embedOne, commit) {
  const vectors = [];
  let dimension;
  for (const text of texts) {
    const vector = await embedOne(text);
    const norm = Array.isArray(vector) ? Math.hypot(...vector) : NaN;
    if (
      !Array.isArray(vector) ||
      vector.length === 0 ||
      !vector.every(Number.isFinite) ||
      !Number.isFinite(norm) ||
      norm === 0
    ) {
      throw new Error('invalid vector');
    }
    dimension ??= vector.length;
    if (vector.length !== dimension) throw new Error('dimension mismatch');
    vectors.push(vector);
  }
  await commit(vectors);
}

let commits = 0;
await assert.rejects(
  stageVectors(
    ['first', 'second'],
    async (text) => (text === 'first' ? [1, 0] : [1, 0, 0]),
    async () => {
      commits++;
    }
  ),
  /dimension mismatch/
);
assert.equal(commits, 0);
await assert.rejects(
  stageVectors(
    ['zero'],
    async () => [0, 0],
    async () => {
      commits++;
    }
  ),
  /invalid vector/
);
await stageVectors(
  ['valid'],
  async () => [1, 0],
  async () => {
    commits++;
  }
);
assert.equal(commits, 1);
console.log('staging boundaries pass');

These fixed two- and three-dimensional vectors verify callback timing and validation order; they represent no language semantics. The lesson has not defined empty input, model-identity records, concurrent writes, or transactions across files. A real service needs those contracts too.

Matching dimensions are insufficient

Two models with the same output dimension can still have incompatible vector spaces. The current implementation reads the configured model but does not store a complete model fingerprint in each index record for compatibility validation. When switching models, treat old-index compatibility as a separate concern rather than comparing array lengths alone.

The actual commit crosses SQLite state and a Vectra file: a database transaction replaces chunk mappings and marks the note syncing, the database is saved, old vectors are deleted, the Vectra update ends, and the note is marked done and saved again. There is no atomic transaction spanning both stores. See index recovery for failure cleanup and recovery. Preparing before committing describes only the model-call stage’s position.

Continue with the fixed-vector local-search exercise to observe persistence, filters, and scores, or read the official embed reference. Check input coverage, chunk boundaries, vector validity, model compatibility, and commit failures separately; one successful search does not establish the whole pipeline’s behavior.