Local Models

Crush can auto-discovers models from local providers. Add a custom provider with type set to llamacpp, omlx, lmstudio, litellm, or ollama and leave out the models list. Crush will populate the model list automatically.

# Piece of cake.provider add ollama \  --name Ollama \  --type ollama \  --base-url "http://localhost:11434/v1/"

For llama.cpp (llama-server), point at the server's base URL:

provider add llamacpp \  --name "llama.cpp" \  --type llamacpp \  --base-url "http://localhost:2222"

Manual Model Configuration

You can still list models explicitly. User-defined models always take precedence over discovered ones, and any fields you set won't be overwritten by auto-discovery. Auto discovery will run if the model list is empty for any openai-compat provider or if you pass "discover_models": true it will merge the found models with your hand configured ones.

# crushrcprovider add ollama \  --name Ollama \  --type ollama \  --base-url "http://localhost:11434/v1/" \  --discover-models true model add ollama/qwen3:30b \  --name "Qwen 3 30B" \  --context-window 256000 \  --default-max-tokens 20000

The --discover-models true flag merges discovered models with the one above; your explicit model fields win on conflicts.