---
title: "Local-First and Cloud AI: Understanding Data Boundaries"
description: Trace storage, embeddings, summaries, and agent recall in ChatCrystal to identify which data boundaries local-first architecture actually controls.
image: "https://zengliangyi.com/og/blog/en/local-first-ai-data-boundaries.png"
---

Keeping conversations on your computer is one step toward controlling your data. But if summarization calls a remote model, or retrieved notes enter a cloud agent’s context, some content still leaves that computer. Privacy in an AI knowledge base starts with the processes and addresses involved in each operation.

[ChatCrystal](/en/projects/chatcrystal) stores notes and vector indexes in the application’s data directory while allowing model providers to be configured separately. Storage and inference can therefore live in different places. This article was revised against commit [`7642768`](https://github.com/ZengLiangYi/ChatCrystal/tree/764276814e5c5a33ecbfeae63754f74b9261a3a8). It examines those data flows without treating local-first design as a guarantee that nothing leaves the machine.

## Separate three locations

**Where records live, where models run, and who receives the results are three questions.** A desktop window on your computer does not answer the latter two. Files in an application server’s data directory are not necessarily on every client’s computer either.

| Stage | Content involved | Location to inspect |

| --- | --- | --- |

| Import and storage | Conversations, notes, metadata | Import process, application server, data directory |

| Embedding | Note chunks and search queries | Actual embedding endpoint |

| Summarization | Conversation content submitted to the model | Actual summarization endpoint |

| Relationship discovery | New-note information and candidate summaries | Language model used for relationship discovery |

| Agent recall | Search results and notes fetched on demand | MCP client and its model provider |

This table describes inspection points, not the results of a traffic audit. Request construction in the relevant version determines the exact fields sent. Recheck it when changing models, endpoints, or application-server location.

## A local index may still use remote embeddings

Vector search involves two moments: indexing converts the note text assembled for retrieval into vectors, and querying converts the current question into a vector before comparing it with the index. The [embedding implementation](https://github.com/ZengLiangYi/ChatCrystal/blob/764276814e5c5a33ecbfeae63754f74b9261a3a8/server/src/services/embedding.ts) calls the configured model in both cases.

Index text combines fields such as the title, summary, key conclusions, tags, and code descriptions. Task memories also include portions of code, causes, resolutions, error signatures, file paths, and other details before chunking. Inspect those fields when assessing transfers: the payload can contain more than a summary, and it is not a chunk-by-chunk embedding of the entire original conversation.

When that model runs remotely, index-text chunks or query text go to its endpoint. Storing the returned vectors in a local Vectra index does not undo the earlier transfer. Nor should vectors automatically be treated as anonymized data. Index metadata also retains fields such as titles and project names, so manage index backups and sharing according to the source content.

Conversely, retrieval can avoid a remote model service when the embedding model, required files, and application server are ready on the local machine. Verify those conditions. A provider named Ollama does not prove that its configured address is local.

## Inspect summarization and relationship discovery separately

ChatCrystal configures language models separately from embeddings, allowing combinations such as local embeddings with remote summarization. The tradeoff is explicit: query vectors can be calculated locally, while content submitted for remote summarization still leaves the machine.

Relationship discovery also involves more than a database lookup. The [current implementation](https://github.com/ZengLiangYi/ChatCrystal/blob/764276814e5c5a33ecbfeae63754f74b9261a3a8/server/src/services/relations.ts) finds candidate notes, then constructs a language-model prompt from the new note’s title, summary, conclusions, and candidate information. Checking only the request behind the summarization button misses this path.

Continue tracing the recipient of the result. After a local MCP server returns notes, its client may include them in a remote model’s context. That introduces another transfer boundary; the knowledge base’s storage design cannot make guarantees for its client.

## Verify offline behavior operation by operation

Opening an application and having every feature work offline are different acceptance criteria. Prepare known data, disconnect external networking, and check:

- Whether existing notes and relationships remain readable.

- Whether a new semantic query works with the embedding model ready.

- Whether new local conversations can be imported and parsed.

- Whether the language model required for new summaries and relationships also runs locally.

- Whether the agent client itself requires a remote model to continue.

Record the error and endpoint when an operation fails. Missing model files, a stopped model service, and an unreachable application server require different responses; they should not all be described as lost memory.

## Measure cost and speed in the same workload

Local execution reduces dependence on external inference services, but hardware, model downloads, power, upgrades, and recovery still cost something. Cloud services reduce some infrastructure work while introducing network, usage-limit, and availability constraints.

Local execution does not establish lower latency, and an untested minimum GPU-memory figure is not useful. Model size, quantization, input length, concurrency, and available hardware all affect the result. For a personal knowledge base, evaluate real questions and measure initial loading, warm-query latency, summary quality, and failure recovery.

Training use, retention, and human access are matters of service policy and account configuration. They cannot be inferred from the word cloud. Before adopting a service, check the current terms for the specific product, plan, and endpoint. This article makes no blanket commitment on a provider’s behalf.

## Turn the choice into inspectable configuration

The local-first capabilities I value are identifiable storage, exportable content, configurable model endpoints, and recoverable failures. Files on a computer still need access controls, backups, and disk protection. Sync folders and backup destinations are also part of the data flow.

Start by listing the application-server address, both model endpoints, the data directory, and the agent client. Use a test note without sensitive content to observe the complete workflow. Whenever deployment changes, confirm again what goes where.

Continue with [local vector database selection](/en/blog/local-vector-database-selection) for storage tradeoffs, or [agent knowledge-base design](/en/blog/agent-knowledge-memory-loop) for how results enter an agent’s context.

Share:

##  Related Posts

[##  Building an Embedding Service: Text, Chunks, and Commits

Read ChatCrystal’s actual embedding pipeline, separate text selection, soft chunking, vector validation, and commits, and verify boundaries offline.          曾亮懿       October 6, 2026      4 min read

#ChatCrystal

#AI

#Node.js](/en/blog/embedding-service-design)    [##  Structured JSON Prompts: Field Design & Verifiable Evidence

Use ChatCrystal’s summaries and relations to define fields, visible evidence, and counterexamples, then distinguish shape and evidence checks with offline Zod.          曾亮懿       October 6, 2026      4 min read

#ChatCrystal

#AI

#Node.js](/en/blog/structured-json-prompts)    [##  Vectra local search: fixed vectors, filters, and persistence

Run a deterministic Vectra 0.15.0 exercise in a temporary directory to check cosine scores, metadata filters, reopening, and canceled updates.          曾亮懿       October 6, 2026      4 min read

#ChatCrystal

#AI

#Node.js](/en/blog/vectra-local-search)

##  From writing to code

Explore ChatCrystal, developer tools and daily learning notes on my GitHub. Questions about the implementation are welcome.

[Visit my GitHub](https://github.com/ZengLiangYi)

```json
{
  "@context": "https://schema.org",
  "@type": "WebSite",
  "name": "ZengLiangYi Blog",
  "alternateName": [
    "Zeng Liangyi",
    "ZengLiangYi",
    "zengliangyi.com"
  ],
  "url": "https://zengliangyi.com",
  "description": "曾亮懿的个人网站。记录 AI Native 探索、开发者工具与全栈产品实践。"
}
{
  "@context": "https://schema.org",
  "@type": "BlogPosting",
  "headline": "Local-First and Cloud AI: Understanding Data Boundaries",
  "description": "Trace storage, embeddings, summaries, and agent recall in ChatCrystal to identify which data boundaries local-first architecture actually controls.",
  "url": "https://zengliangyi.com/en/blog/local-first-ai-data-boundaries/",
  "image": "https://zengliangyi.com/images/blog/local-first-ai-data-boundaries.svg",
  "datePublished": "2026-06-04T00:30:00.000Z",
  "dateModified": "2026-10-23T00:00:00.000Z",
  "author": {
    "@type": "Person",
    "name": "曾亮懿",
    "url": "https://zengliangyi.com/en/about"
  },
  "publisher": {
    "@type": "Organization",
    "name": "ZengLiangYi Blog",
    "logo": {
      "@type": "ImageObject",
      "url": "https://zengliangyi.com/favicon-96x96.png"
    }
  },
  "mainEntityOfPage": {
    "@type": "WebPage",
    "@id": "https://zengliangyi.com/en/blog/local-first-ai-data-boundaries/"
  }
}
{
  "@context": "https://schema.org",
  "@type": "BreadcrumbList",
  "itemListElement": [
    {
      "@type": "ListItem",
      "position": 1,
      "name": "Home",
      "item": "https://zengliangyi.com/en"
    },
    {
      "@type": "ListItem",
      "position": 2,
      "name": "Blog",
      "item": "https://zengliangyi.com/en/blog"
    },
    {
      "@type": "ListItem",
      "position": 3,
      "name": "Local-First and Cloud AI: Understanding Data Boundaries",
      "item": "https://zengliangyi.com/en/blog/local-first-ai-data-boundaries/"
    }
  ]
}
```