Install and get started (KoboldCPP)
KoboldCPP is based on llama.cpp with a light UI. It is common in roleplay communities and can expose an OpenAI-compatible HTTP API for MiniTavern.
Install
- Open GitHub Releases and download the build for your OS (Windows often has a standalone
.exe; other systems follow compile/release notes). - On first run, pick a backend (CUDA / Metal / Vulkan / CPU, etc.) for your hardware.
- Prepare a GGUF model file (see Recommended models).
Load a model
- In KoboldCPP pick the model file and start.
- Confirm generation works in its built-in chat/UI before exposing the API.
- Note context length, GPU layers, etc.; defaults that run are fine at first.