Topic

#Inference

23 articles on Inference — news, releases, guides and analysis from the SourceFeed engine.

Article 1w ago 2

Inference ASICs Won the Benchmarks, Then Lost Their Independence

AMD bought Taalas, Nvidia gutted Groq, Cerebras went public — how to place inference workloads after the shakeout.

Mariana Souza

$llmfit does the local-LLM math you've been faking$

llmfit does the local-LLM math you've been faking

The k8sgpt creator's Rust CLI right-sizes models to your GPU — keeping its catalog honest is the hard part.

Article · 1w ago6

Mistral 3 Was Never About the Leaderboard

Eight months on, the Apache-licensed family looks less like a frontier bid and more like enterprise infrastructure.

Article · 2w ago2

Nvidia's Switchyard Finally Makes Model Routing Boring

Nemotron 3.5 Lightning gets the headlines, but the Rust proxy quietly rewires how agent workloads buy compute.

Article · 2w ago1

Nvidia's Real Product Here Is the Router, Not the Model

Nemotron 3.5 Lightning is a capable workhorse, but NeMo Switchyard targets the escalation logic every agent team keeps hand-rolling.

Article · 2w ago1

NVIDIA's Switchyard Matters More Than Its New Model

Nemotron 3.5 Lightning is a solid agent workhorse, but the open-source router is what changes agent economics.

Article · 2w ago4

ECS Finally Learns to Schedule a Slice of a GPU

AWS closes a year-old gap for G6f slices — worthwhile for small-model inference, once you've done the pricing math.

Article · 3w ago0

Your SSD Is the New VRAM for Local LLMs

Swiftlet streams sparse Qwen experts off disk, fitting an 80B model into 4.3 GB of Mac RAM.

Article · 3w ago1

Your SSD Is the New VRAM

Swiftlet streams MoE experts from SSD, fitting an 80B Qwen in 4.3GB of RAM — and a 35B on an iPhone.

Article · 3w ago1

$Cloudflare's Quantization Math Is Right. The Disclosure Isn't$

Cloudflare's Quantization Math Is Right. The Disclosure Isn't

FP8 caches and INT4 weights are becoming table stakes; what providers tell you at purchase time is the real fight.

Article · 3w ago0

AirLLM's 4GB 70B Trick Is Real, and Beside the Point

Layer streaming makes giant models fit; disk bandwidth decides whether you'd ever want to run them that way.

Article · 3w ago1

A 26B Model in 2 GB of RAM, Courtesy of Your SSD

TurboFieldfare streams Gemma 4's experts from flash, and the trick will outlive the tool.

Article · 1mo ago2

Google Split Its TPU in Two Because Agents Broke Inference

Cloud Next 2026's bifurcated silicon, flattened networks, and managed sandboxes reveal what agentic workloads actually demand.

Article · 1mo ago2

Discarded Teslas Still Deliver Local AI VRAM

Enterprise cards at $60–200 remain usable for intermittent inference and training if you tolerate older stacks and power draw.

Article · 1mo ago0

Popping the CPU-GPU Latency Bubble in Inference

Pipelined decoding techniques show that software optimization, not just raw hardware scaling, is the key to maximizing GPU utilization.

Article · 1mo ago2

OpenAI Jalapeno and the Shift to Custom Inference Silicon

Custom ASICs are replacing general-purpose GPUs for running large language models to survive the crushing cost of scale.

Article · 2mos ago7