Self-host vLLM, with the operator surface you actually wanted.

vLLM Warden is the operator console for serving large language models from a single GPU host. It manages model lifecycle, exposes an OpenAI-compatible API at /v1, schedules across multiple GPUs, and gives you per-consumer API tokens with rate limits — all from one self-hosted appliance, no external control plane.

Open console View on GitHub

OpenAI-compatible /v1

Drop-in for any client that already talks to OpenAI.

Multi-GPU scheduling

Let vLLM Warden pick the right GPU for each model.

API tokens with rate limits

Per-consumer keys, TPS caps, usage telemetry.

Single-node appliance

One Docker compose stack, no external control plane.