#Local Llm

llmfit does the local-LLM math you've been faking

The k8sgpt creator's Rust CLI right-sizes models to your GPU — keeping its catalog honest is the hard part.

Article · 1w ago6

Convert and Quantize Hugging Face Models to GGUF for llama.cpp

Turn any Hugging Face checkpoint into a 4-bit GGUF that runs fast and small on your own hardware.

Tutorial · 2w ago0

llama.cpp Finally Competes With Its Own Wrappers

With Hugging Face backing, the engine under Ollama and LM Studio now ships its own installer and model catalog.

Article · 2w ago4

macOS VMs Have Been Sandbagging Your GPU

Cua's Metal capability shim recovers up to 16x llama.cpp throughput that Apple's paravirtualized graphics was hiding.

Article · 2w ago0

Your macOS VM's GPU Was Never Slow, Just Mislabeled

A shim that lies to Metal about GPU capabilities recovers near-bare-metal llama.cpp speeds inside Apple Silicon VMs.

Article · 2w ago1

Meta's Muse Glimmer Bets Open Weights on Local Agents

The 30B Apache 2.0 model fits on a 24GB GPU and quietly abandons the Llama license.

Article · 2w ago2

Ante Bets Coding Agents Should Be Single Binaries

A 15MB Rust agent with llama.cpp baked in gets the architecture right and the trust model wrong.

Article · 2w ago3

Muse Glimmer Is Meta's Apology for the Llama License

A dense 30B coding agent under Apache 2.0 puts Meta back in the local-model race it started and lost.

Article · 2w ago2

Muse Glimmer Is Meta's Best Open-Weights Move Since Llama 2

The 30B Apache-licensed model won't out-code the cloud, but it's engineered for the machine you actually own.

Article · 2w ago1

Local AI Code Completion in VS Code with Qwen2.5-Coder and Ollama

Wire Qwen2.5-Coder through Ollama and Continue for private, offline autocomplete and chat in VS Code.

Tutorial · 3w ago0

Run Mixtral 8x7B Locally with llama.cpp and Benchmark MoE vs. Dense

Compile llama.cpp, run quantized Mixtral 8x7B, and measure the mixture-of-experts speed-versus-memory trade-off yourself.

Tutorial · 3w ago0

An 8B Fine-Tune Now Fits in 4 GB of VRAM

Soup streams frozen decoder layers through the GPU one at a time, and proves the arithmetic is bit-exact.

Article · 3w ago1

Your SSD Is the New VRAM for Local LLMs

Swiftlet streams sparse Qwen experts off disk, fitting an 80B model into 4.3 GB of Mac RAM.

Article · 3w ago1

Your SSD Is the New VRAM

Swiftlet streams MoE experts from SSD, fitting an 80B Qwen in 4.3GB of RAM — and a 35B on an iPhone.

Article · 3w ago1

AirLLM's 4GB 70B Trick Is Real, and Beside the Point

Layer streaming makes giant models fit; disk bandwidth decides whether you'd ever want to run them that way.

Article · 3w ago1