Sovereign local LLM stack for a software team
- Challenge
- A software team needed AI coding assistance without sending proprietary source code to a cloud service.
- What was built
- Tuned llama.cpp for full GPU layer offload — first on an 8 GB GPU, then on a 100 GB-VRAM machine — connected an open-source coding CLI to the local model, and exposed retrieval over the internal repositories as an MCP server. Colleagues were coached through adoption.
- Outcome
- An AI coding CLI available to everyone in the organisation, with every line of code and data staying in-house.
Where this applies
- Generative AI for teams bound by confidentiality or data-residency rules
- AI pair-programming and knowledge search over internal code and documentation
- A predictable-cost alternative to per-seat cloud AI subscriptions