llama-model: manage GGUF models and llama-server without the Ollama daemon

by

Mascote do LinuxPro retirando um modelo da estante de arquivos GGUF, com a lhama observando

If you run AI models locally with llama.cpp / llama-server, you know the pain: downloading GGUF files by hand, editing configs, restarting services, hoping the model starts. The llama-model solves this — a CLI written in Go, a single static binary, that manages your model collection and the llama-server service without needing the Ollama daemon. And it is open for testing: we want users!

Content hide
1 What it does
2 Why this is interesting
3 Installing (Linux amd64)
4 Getting started in 2 minutes
5 We want you to test it! 🧪

What it does

Command Function
list Lists models on disk (name, size, quantization)
pull <modelo> Downloads directly from the Ollama registry — no daemon, with sha256 and resume support
set <modelo> Switches the llama-server model, restarts, validates /health and performs a rollback if it fails
rm <modelo> Removes a model while preserving shared blobs
gpu list / gpu watch NVIDIA GPU telemetry (temperature, power, clocks, throttling)
proxy Frontend with the Anthropic protocol to use Claude Code with llama-server

Why this is interesting

  • No daemon: the pull speaks HTTPS directly to the Ollama registry. Blobs verified by sha256 and downloads that are resumable — if the connection drops, run it again and pick up where it left off.
  • set with a safety net: edits /etc/default/llama-server, restarts, and waits for /health. If the model does not start, restores the previous config automatically. It also adjusts OpenCode and Hermes configs if they exist (with .bak of each file).
  • Telemetry that survives a crash: the gpu watch records temperature/power peaks, decodes throttling reasons, and, if the card drops off the bus, prints the last good reading alongside the kernel's Xid messages — what you need to distinguish thermal issues from power issues.
  • Claude Code + local model: the proxy adapts the Anthropic protocol for llama-server (templates such as Qwen's reject certain system messages), with working streaming.

Installing (Linux amd64)

curl -sL https://github.com/jniltinho/llama-model/releases/latest/download/llama-model_$(curl -s https://api.github.com/repos/jniltinho/llama-model/releases/latest | grep -oP '"tag_name": "vK[^"]+')_linux_amd64.tar.gz | tar xz
sudo install -m0755 llama-model /usr/local/bin/

Or build from source (Go 1.26+):

git clone https://github.com/jniltinho/llama-model
cd llama-model && make build && sudo make install

Getting started in 2 minutes

llama-model list                    # o que já está no disco
sudo llama-model pull qwen3.8:27b   # baixa do ollama.com/library
sudo llama-model set qwen3.8:27b    # troca, reinicia e valida
llama-model gpu watch               # telemetria ao vivo das GPUs

And to connect Claude Code to your local model:

llama-model proxy --listen 127.0.0.1:11436 --upstream http://127.0.0.1:11435
export ANTHROPIC_BASE_URL=http://127.0.0.1:11436

We want you to test it! 🧪

The project is open source (MIT), with CI and releases on GitHub, and it is at the stage where feedback from real users is worth its weight in gold:

Running local AI on Linux keeps getting better — and good operational tooling is what separates a toy from a real server. Ready to try it?