Tuesday, November 3, 2026 Today’s Paper
Log in Join free
Vol. I · No. 35 Texas Edition
Tuesday, Nov 3
Texas first. Then the world.
Always free Updated 3 minutes ago
Read. React. Discuss.
Tech

Show HN: LlamaForge – a browser control panel for llama.cpp's own router

By dadwritestech 1 min read 5 views 0 comments
Show HN: LlamaForge – a browser control panel for llama.cpp's own router
Image: Hacker News (Newest)

I run local models on two mismatched GPUs and got tired of hand-editing llama.cpp's models.ini and re-reading --help after every release. LlamaForge is a browser panel over llama.cpp's own llama-server router. It runs no inference itself.A few choices people here may care about:- The backend is pure Python stdlib (ThreadingHTTPServer, no pip install). The frontend is vanilla JS with no build step.- The settings UI is generated from llama-server --help at runtime, so a flag added upstream shows up without a LlamaForge release (200+ on current builds).- Auto-Forge measures instead of guessing: it tries layer placements, KV cache types and speculative drafters in a sandboxed router, checks that a drafter gives the same answers, and writes the winner to models.ini with an undo. On my 5080 + 5060 Ti it took Qwen3.8-27B from 26.8 tok/s on code to 126. (The old setup sat mostly on the slower card; the 5080 alone without a drafter does 52.)- It can't track usage per client, because clients talk to llama.cpp directly. That's on purpose.Early preview. Windows + NVIDIA is the well-trodden path. Linux and macOS pass CI but have had little real-hardware use. It's heavily AI-assisted, and…

Hacker News (Newest) Original story · github.com
Read the full story

Discussion (0)

No comments yet. Start the conversation!