Ollama Local Workflow: Generation, Vectors, and Retrieval

Running models locally can reduce material sent to remote services, but first establish that the selected model is local. Ollama also supports cloud models; a localhost service address alone does not prove every inference happens on the machine.

This article explains how to verify one note through the Ollama adapter in ChatCrystal at commit 7642768. It offers no unmeasured hardware speeds, model rankings, or promise of zero cost. Local operation still uses download bandwidth, storage, memory, and electricity.

Install and identify the local service

Follow the official Ollama quickstart for your system and select a local model. Check:

ollama --version
ollama list

In a terminal environment without a running service, start ollama serve. If the port is occupied, identify its owner and inspect the actual API response. An occupied port alone does not establish a working Ollama service.

The ChatCrystal 0.6.1 npm entry also requires Node.js 24 or later. This is an application requirement, not an Ollama installation requirement. See the getting-started guide for installation and desktop-interface differences.

Prepare models for separate tasks

These names are examples from the application’s fixed-version defaults, not a quality ranking:

ollama pull qwen2.5:7b
ollama pull nomic-embed-text
ollama list

The first serves generation; the second serves embeddings. Download size and runtime resources depend on version, quantization, context, and concurrency. Disk capacity for files does not establish sufficient inference memory. Test short input before actual summary lengths. If resources are insufficient, choose a model suited to the machine and recheck its task capabilities.

Test generation and vectors independently

The official compatibility reference describes compatible endpoints and their supported scope. Test generation through the path used by ChatCrystal’s adapter:

curl --fail-with-body http://localhost:11434/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"qwen2.5:7b","messages":[{"role":"user","content":"Explain note indexing in one sentence."}],"stream":false}'

Test vectors separately:

curl --fail-with-body http://localhost:11434/v1/embeddings \
  -H 'Content-Type: application/json' \
  -d '{"model":"nomic-embed-text","input":"Note indexing test"}'

Inspect HTTP status and response shape: generation should contain assistant text; embeddings should contain a numeric vector inside data. Generation success does not establish vector availability, and a compatible endpoint does not give every model all features. These commands perform local inference; this article reports no measured performance from them.

Connect ChatCrystal

With the default local connection:

crystal config set llm.provider ollama
crystal config set llm.baseURL http://localhost:11434
crystal config set llm.model qwen2.5:7b
crystal config set embedding.provider ollama
crystal config set embedding.baseURL http://localhost:11434
crystal config set embedding.model nomic-embed-text
crystal config get
crystal config test

The application adapter appends /v1 to the Ollama address, so supply the service root here. If vectors already exist, some configuration changes require confirmation and reindexing. Follow the model configuration and rebuilding steps; a successful save does not establish a completed rebuild.

config test reports the two connection paths separately. Passing short tests can still be followed by failure on long summaries or structured output. Retain the actual error, input length, and model information for diagnosis.

Verify one real note

Import a known conversation, select a record outside the demo data with reusable experience, generate its note, inspect the content, and read embedding_status from note JSON. The experience gate can legitimately filter some conversations; vector creation can fail independently after a successful summary.

Once status is done, test a query that should find the note and another with no relevant answer. See recovering a debugging solution. Then increase input and concurrency, recording load time, response time, failure rate, and peak resources. Distinguish cold loading from steady inference.

Verify offline behavior and data boundaries

Once required local models are downloaded, no cloud model is selected, and dependent services are available, test the full workflow without network access. A successful short local chat does not establish that import, summaries, relation discovery, and queries all work offline.

Inspect the actual LLM and embedding configurations together. Combining local and remote backends can still send material along the remote path. The application database and vector index also require care; see local-first data boundaries.