#Llm Inference

You're Not Buying Compute, You're Buying Utilization

Every honest cost model of open-model inference reaches the same verdict, and the reason should change how you decide.

Article · 1w ago1

Speed Is Now a Paid Tier at OpenAI

Cerebras wafer-scale silicon pushes GPT-5.6 Sol to 750 tokens per second, and changes how inference gets priced.

Article · 2w ago1

NVIDIA's roadmap is a memory story, not a compute story

Read A100 through Rubin by bandwidth, number formats, and rack power, and the right 2026 rental choice falls out.

Article · 2w ago0

Latent Reasoning Escapes the Lab, Bolted Onto DeepSeek-V4

A 36M-parameter head makes V4-Flash think silently in latent space — and the serving hacks matter more than the benchmarks.

Article · 2w ago2

Your Agents Are Waiting on the CPU, Not the GPU

Agentic workloads moved the inference bottleneck off the accelerator, and hardware ratios are quietly following.

Article · 3w ago5

How a 20B Model Hits 120 tok/s on an iPhone

DeepGrove's Maple-Preview stacks native ternary training on MoE sparsity, and the arithmetic largely holds up.

Article · 3w ago1

DeepSeek V4 Flash on One AMD GPU Took Nine Patches

A single 192 GB MI300X now serves the full checkpoint — proof of AMD's hardware edge and its lingering software gap.

Article · 3w ago1

Quantize the Decode, Not the Prefill

Cloudflare's production numbers for Kimi K2.6 and GLM 5.2 show where trillion-parameter inference actually gets cheaper.

Article · 3w ago5

Linear Attention Just Graduated to Frontier Scale

Kimi Linear looked like another attention paper in October — now its KDA design underpins a 2.8T-parameter flagship.

Article · 1mo ago2

The Real Cost of 'Just Use vLLM,' According to Netflix

Netflix's Model Runtime team details the version pinning, GIL workarounds, and preemption bugs behind production LLM inference.

Article · 1mo ago2

Hetzner Is Quietly Commoditizing LLM Inference

A free, OpenAI-compatible endpoint from Europe's cheapest host says more about inference margins than any benchmark.

Article · 1mo ago1

KTransformers Turned CPU Offloading Into Real Infrastructure

The Tsinghua-born framework dissolved into SGLang and LLaMA-Factory, making trillion-class MoE models a single-machine problem.

Article · 1mo ago1

AI's Energy Problem Isn't Your Chatbot Query

The reassuring per-prompt numbers measure the chatbot era, just as agents change the math by 200x.

Article · 1mo ago2

VRAM Beats TOPS for 2026 Local AI GPUs

Memory capacity and bandwidth decide what models you can run; software maturity still decides how painful it feels.

Article · 1mo ago2

Ditching HBM: Inside the Monolithic 3D AI ASIC

PhantaField’s Sophon PFG-1 puts 330 GB of DRAM on-die, bypassing the GPU memory wall for low-batch LLMs.

Article · 2mos ago2