Core Concepts

Models

Engram is model-agnostic. Any model that runs in Ollama can power Engram's chat, agents, and code analysis. Switch by changing one environment variable — no code changes required.

Required Models

Two models must be present before starting Engram. The setup script pulls them automatically.

llama3.1:latest

The default inference model. Powers all chat, agent reasoning, code review, and email drafts.

8B params · 4.7 GB · 128k context

nomic-embed-text:latest

The embedding model. Converts all text to 768-dimensional vectors for Qdrant storage and semantic search.

137M params · 274 MB · embeddings only

bash

ollama pull llama3.1:latest && ollama pull nomic-embed-text:latest

Switching Models

Switch models two ways: select one from the model dropdown in the dashboard chat bar (takes effect immediately, per-session), or set OLLAMA_MODEL in your .env file as the system-wide default. The embedding model is always nomic-embed-text — do not change it unless you re-embed all existing memories.

bash

# .env
OLLAMA_MODEL=mistral:latest   # switch to Mistral

# Then restart
./scripts/start.sh

Re-embedding not required

Changing the inference model does not require re-embedding your memories. The embedding model is separate. Only change EMBED_MODEL if you want a different embedding space — which will invalidate all existing vectors.

Supported Models

Any Ollama-compatible model will work. The following are tested with Engram v1.7.

Model	Params	Disk	Context	Best for
llama3.1:latestDefault	8B	4.7 GB	128k	General chat, agents, code review
mistral:latest	7B	4.1 GB	32k	Fast responses, structured output
gemma2:latest	9B	5.5 GB	8k	Summarisation, document Q&A
deepseek-r1:latest	7B	4.7 GB	64k	Reasoning-heavy tasks
nomic-embed-text:latestRequired	137M	274 MB	8192 tokens	Embeddings only — required for Qdrant

Performance Tips

Use quantised models for speed

Models like llama3.1:8b-instruct-q4_K_M use 4-bit quantisation — roughly half the RAM at ~10% quality loss. Good for interactive use on 8 GB machines.

Enable GPU acceleration

On Apple Silicon, Ollama uses Metal automatically. On Linux with an NVIDIA GPU, install nvidia-container-toolkit and add --gpus all to your Docker run args.

Pre-warm the model

On first boot, Ollama loads model weights into memory. This can take 10–30 seconds. Subsequent calls are fast. The setup script sends a warm-up ping automatically.