The prefill/decode split, the 4-bit sweet spot and the quality cliff below it, and why reflexively reaching for an H100 is often a multi-thousand-dollar mistake.
Read PostKV caching rethought with LMCache and 14x faster TTFT, the 4-stage production routing pipeline, and Fable-as-Advisor at 92% quality for 63% of the cost.
Read PostGPU utilization is a costly liar. Bypassing the Prometheus tax, fractional GPU allocation, deployment-aware routing, and the scale-to-zero mandate.
Read PostHardcoding every query to a flagship model is a 40–85% budget leak. Classifier-based routing, latency budgets, meta-models, and self-learning bandit routers.
Read PostTarget all linear layers, not a higher rank. QLoRA's neutralized quantization tax, DPO's beta parameter, GRPO for reasoning, and why LoRA guards factuality.
Read PostModel routing that cuts costs 50-60%, the 4-layer agent engineering stack, and self-improving harnesses lifting small models 33-60% — plus Sonnet 5 ships.
Read PostThe 18x gap between total and active parameters, the three walls of MoE scaling, and how parallel folding and sync-free execution turn them into doorways.
Read PostPrompt engineering gave way to loop engineering. The Ralph Loop, circuit breakers, RLVR replacing RLHF, and alignment by unlearning.
Read PostSpeculative decoding at 4x speedups, the RAG taxonomy every engineer needs, and Karpathy's agentic engineering framework — plus GPT-5.6 under government review.
Read Post
