Home / Tools
Open Source

Tools We Built
and Run Every Day

The lab runs on one GPU workstation. These are the parts of its software we publish in the open, with their source and their measurements. Each one is documented where it lives.

Model serving

MIT Python · vLLM · systemd

Servedeck: One Control Plane for Local vLLM

Runs each local vLLM model as its own systemd user unit, serves one OpenAI-compatible /v1 endpoint that answers to the name main, and shows live KV-cache use and throughput. Restarting it never touches a running model.
Write-up NVFP4 · MoE · CPU offload

GLM-5.3-Flash on One RTX PRO 6000

A 320B-parameter mixture-of-experts model (18B active) in NVFP4, served from one workstation card by keeping its routed experts in pinned host RAM. The best result we know of on a single RTX PRO 6000, with the full method.

The workstation

Source Linux · systemd · NVIDIA

hostops: Root-Level Services for One Workstation

A UPS-aware GPU power cap and a power ledger: the root-level services of the lab's one workstation.