---
title: "LLMs and embeddings: separate tasks, even with one provider"
description: Distinguish text generation from vector generation, then verify ChatCrystal model settings, connection tests, and index rebuilding boundaries.
image: "https://zengliangyi.com/og/blog/en/llm-and-embedding-separation.png"
---

Two settings labeled “model” do not necessarily accept the same name. Note generation needs a language model to produce content; semantic search needs an embedding interface to produce numeric vectors. Diagnose the task and return type before checking provider and model names.

This article checks configuration and provider implementations in [ChatCrystal](/en/projects/chatcrystal) at commit [`7642768`](https://github.com/ZengLiangYi/ChatCrystal/tree/764276814e5c5a33ecbfeae63754f74b9261a3a8). It does not rank model quality across providers.

## One input, two outputs

- Text generation  takes a debugging conversation and produces a title, summary, and conclusions for note generation and relation discovery.

- Embedding  takes searchable note text or a query and produces a numeric vector for note indexing and retrieval of similar candidates.

[OpenAI’s embedding guide](https://developers.openai.com/api/docs/guides/embeddings) shows a dedicated vector request and response. Generated text cannot be inserted directly as a vector. A model name containing GPT, Claude, or Qwen does not establish its supported tasks; inspect the actual model and interface capabilities.

Generation interfaces are not universally `/v1/chat/completions`. Providers can use Responses or their own protocols; [ChatCrystal’s adapters](https://github.com/ZengLiangYi/ChatCrystal/blob/764276814e5c5a33ecbfeae63754f74b9261a3a8/server/src/services/providers.ts) select the call path. Separate configuration follows the task contract, rather than an absolute claim about every model architecture.

## The application has two configurations

[Configuration code](https://github.com/ZengLiangYi/ChatCrystal/blob/764276814e5c5a33ecbfeae63754f74b9261a3a8/server/src/config.ts) stores `llm` and `embedding` separately, each with `provider`, `baseURL`, `apiKey`, and `model`. Environment variables provide defaults; existing model settings in `config.json` override the corresponding defaults. After changing environment variables, check what the service actually loaded.



```bash
crystal   config   get  
  crystal   config   test
```



`get` displays model names, endpoints, and whether keys are set, without showing their contents. `test` checks both paths. LLM success does not make an embedding failure irrelevant.

## A local example of separate settings

This example selects two different tasks; it is not a recommendation of the fastest or most accurate models:



```bash
crystal   config   set   llm.provider   ollama  
  crystal   config   set   llm.baseURL   http://localhost:11434  
  crystal   config   set   llm.model   qwen2.5:7b  
  crystal   config   set   embedding.provider   ollama  
  crystal   config   set   embedding.baseURL   http://localhost:11434  
  crystal   config   set   embedding.model   nomic-embed-text
```



The models must already be downloaded and the service running. The current Ollama adapter appends `/v1` to the configured address, so use the service root here without adding another `/v1`. [Ollama’s compatibility documentation](https://docs.ollama.com/api/openai-compatibility) describes generation and embedding endpoints. Compatibility with an interface does not establish support for all its features.

The current ChatCrystal Anthropic and OrcaRouter entries do not implement embedding model creation. Select another entry supporting that task for retrieval. This is a boundary of this application’s version, not a claim that a vendor will never offer vector services.

Local embeddings can also be combined with a remote language model. Vector computation for notes and queries can then stay local, while material submitted to remote generation still leaves the machine; see [data-flow privacy boundaries](/en/blog/local-first-ai-data-boundaries).

## Interpret connection tests carefully

The [test route](https://github.com/ZengLiangYi/ChatCrystal/blob/764276814e5c5a33ecbfeae63754f74b9261a3a8/server/src/routes/config.ts) generates a short response, then embeds a test string. It catches errors separately and reports each result through `connected`. HTTP success does not mean both connections succeeded. If CLI output contains a failed result, process exit status alone is also insufficient evidence of success.

| Symptom | First checks |

| --- | --- |

| Refused connection or timeout | Running service, reachable endpoint, which process owns the address |

| 404 or missing model | Correct path, available model ID, support for the requested task |

| Authorization failure | Valid key in the relevant configuration, account access |

| Short test works; summary fails | Actual input length, structured output capability, resources, returned error |

An error code is a clue. A 404 does not automatically prove that a generation model was selected for embeddings. Preserve the full error and relevant settings, and investigate each call path separately.

## Changing embeddings affects existing indexes

Equal dimensions alone do not make old and new vectors compatible. Changing models, versions, or input preparation can change the representation space. Note vectors and query vectors need matching settings, followed by retrieval evaluation.

ChatCrystal returns confirmation requirements and warnings for some embedding configuration changes. Confirming a provider or model change clears existing vectors and the index and marks notes `pending`; it does not automatically enqueue rebuilding. Back up and understand those warnings before choosing `--confirm`. A successful update does not mean rebuilding has started.

Once the new model configuration passes its test, local mode can explicitly submit pending notes through the [batch embedding endpoint](https://github.com/ZengLiangYi/ChatCrystal/blob/764276814e5c5a33ecbfeae63754f74b9261a3a8/server/src/routes/notes.ts):



```bash
curl   --request   POST   http://localhost:3721/api/embeddings/batch
```



The returned `queued` count describes queued tasks, not completed work. Read the task snapshot and select an actual note ID from the list for inspection:



```bash
curl   http://localhost:3721/api/queue/status  
  crystal   --json   notes   list
```





```text
crystal --json notes get <note-id>
```



The queue response contains status under `data`; note-detail JSON directly includes `embedding_status`. Once a note is `done`, test a known-answer query and one that should not match. `pending` or `syncing` means unfinished work; investigate `failed` with the task error. An empty queue alone does not establish that every note succeeded. Other setting changes, such as `baseURL`, also require checking whether existing vectors still match new queries.

For a first installation, follow the [single-note workflow](/en/blog/chatcrystal-getting-started). Test a query with a known answer and another that should not match, so both success and failure paths are observable.

Share:

##  Related Posts

[##  Building an Embedding Service: Text, Chunks, and Commits

Read ChatCrystal’s actual embedding pipeline, separate text selection, soft chunking, vector validation, and commits, and verify boundaries offline.          曾亮懿       October 6, 2026      4 min read

#ChatCrystal

#AI

#Node.js](/en/blog/embedding-service-design)    [##  Structured JSON Prompts: Field Design & Verifiable Evidence

Use ChatCrystal’s summaries and relations to define fields, visible evidence, and counterexamples, then distinguish shape and evidence checks with offline Zod.          曾亮懿       October 6, 2026      4 min read

#ChatCrystal

#AI

#Node.js](/en/blog/structured-json-prompts)    [##  Vectra local search: fixed vectors, filters, and persistence

Run a deterministic Vectra 0.15.0 exercise in a temporary directory to check cosine scores, metadata filters, reopening, and canceled updates.          曾亮懿       October 6, 2026      4 min read

#ChatCrystal

#AI

#Node.js](/en/blog/vectra-local-search)

##  From writing to code

Explore ChatCrystal, developer tools and daily learning notes on my GitHub. Questions about the implementation are welcome.

[Visit my GitHub](https://github.com/ZengLiangYi)

```json
{
  "@context": "https://schema.org",
  "@type": "WebSite",
  "name": "ZengLiangYi Blog",
  "alternateName": [
    "Zeng Liangyi",
    "ZengLiangYi",
    "zengliangyi.com"
  ],
  "url": "https://zengliangyi.com",
  "description": "曾亮懿的个人网站。记录 AI Native 探索、开发者工具与全栈产品实践。"
}
{
  "@context": "https://schema.org",
  "@type": "BlogPosting",
  "headline": "LLMs and embeddings: separate tasks, even with one provider",
  "description": "Distinguish text generation from vector generation, then verify ChatCrystal model settings, connection tests, and index rebuilding boundaries.",
  "url": "https://zengliangyi.com/en/blog/llm-and-embedding-separation/",
  "image": "https://zengliangyi.com/images/blog/llm-and-embedding-separation.svg",
  "datePublished": "2026-10-03T00:00:00.000Z",
  "dateModified": "2026-10-03T00:00:00.000Z",
  "author": {
    "@type": "Person",
    "name": "曾亮懿",
    "url": "https://zengliangyi.com/en/about"
  },
  "publisher": {
    "@type": "Organization",
    "name": "ZengLiangYi Blog",
    "logo": {
      "@type": "ImageObject",
      "url": "https://zengliangyi.com/favicon-96x96.png"
    }
  },
  "mainEntityOfPage": {
    "@type": "WebPage",
    "@id": "https://zengliangyi.com/en/blog/llm-and-embedding-separation/"
  }
}
{
  "@context": "https://schema.org",
  "@type": "BreadcrumbList",
  "itemListElement": [
    {
      "@type": "ListItem",
      "position": 1,
      "name": "Home",
      "item": "https://zengliangyi.com/en"
    },
    {
      "@type": "ListItem",
      "position": 2,
      "name": "Blog",
      "item": "https://zengliangyi.com/en/blog"
    },
    {
      "@type": "ListItem",
      "position": 3,
      "name": "LLMs and embeddings: separate tasks, even with one provider",
      "item": "https://zengliangyi.com/en/blog/llm-and-embedding-separation/"
    }
  ]
}
```