This week’s newsletters converged on a theme I wasn’t expecting: a lot of “it works, but why” pieces. Not “here’s a new model,” but “here’s why the thing you already run behaves the way it does” — why your multivector collection is slow, why your agent’s token bill is high, why a skill helps when the underlying model didn’t change. Five deep-dives below, a busy news week (OpenRouter got acquired, chip spending kept compounding, Anthropic explained its watermark), and a few tools worth your afternoon.

1. Ollama vs vLLM vs SGLang: Picking an Engine Is an Architecture Decision, Not a Preference

Last week I went deep on the scheduler behind vLLM’s throughput advantage — continuous batching and PagedAttention. This week’s ByteByteGo issue steps back a level: which of the three engines should you even be running, given what’s actually calling it.

The three map cleanly to three request shapes. Ollama takes one local user hitting an OpenAI-compatible API, queues requests FIFO, and runs a pre-quantized GGUF model — it’s built for local dev and laptop-scale hardware, not concurrency. vLLM is for many users hitting the server at once: continuous batching slots new requests into an already-running batch instead of making them wait, and PagedAttention manages the KV cache so GPU memory doesn’t get wasted on padding. SGLang targets a different shape entirely — agents and multi-turn chats where successive prompts overlap heavily. Its prefix-aware scheduler routes requests through RadixAttention, a radix tree that reuses every shared prefix instead of recomputing it.

The takeaway that actually changes what I’d reach for: if you’re serving a chat app with independent, unrelated requests, vLLM’s batching is the right lever. If you’re running an agent loop where every turn re-sends most of the previous context, SGLang’s prefix cache is solving a problem vLLM’s PagedAttention doesn’t specifically target. “Best open-weight engine” depends on whether your workload looks like a queue of strangers or one long conversation with itself.

2. Everyone Is Using Multivectors Wrong

This is the best “one line, three months late” piece I’ve read in a while. HiDevs’ write-up starts from a production symptom — an ingestion job that used to finish overnight now runs into the afternoon, a search cluster sized for 8GB getting OOM-killed at 16GB — and traces it back to a single unset config value on a ColBERT/ColPali/late-interaction collection in Qdrant.

The root cause: late-interaction models don’t compress a document into one vector, they keep one vector per token. A single sentence embeds to 33–48 vectors with ColBERT; a ColPali PDF page embeds to roughly 1,030. So “we indexed 20,000 pages” quietly means 20.6 million vectors, and a default Qdrant collection builds an HNSW graph over every single one of them — at a cost that scales with the square of vectors per document (Qdrant’s own math: ~49 million comparisons just to insert one 700-vector page). None of that graph ever gets used, because the standard architecture is prefetch-then-rerank: a cheap dense-vector search returns ~200 candidates, and only those get rescored with MaxSim over the stored multivectors. Qdrant’s own docs say the quiet part out loud — rescoring doesn’t use the HNSW index at all.

The fix is hnsw_config=HnswConfigDiff(m=0) on the multivector field, leaving HNSW switched on for whatever field actually serves your first-stage retrieval. Graph memory for that field drops to zero, insert time collapses, and nothing about search quality changes because that graph was never queried. It doesn’t fix vector size or MaxSim cost — those are separate levers (quantization, float16, on-disk memory tiers) — and if your multivector field is your only retrieval path at scale, you need to index a cheaper derived representation instead (mean-pooled ColPali patches, or FastEmbed’s MUVERA transform), not just flip m=0 and hope. Worth checking indexed_vectors_count on your own collections this week — it should sit at zero for any field where you’ve disabled the graph, and it won’t show anything meaningful until the collection actually clears its indexing threshold, so don’t trust a small local test that shows no difference.

3. GraphRAG: How AI Answers Questions Hidden Across Many Documents

ByteByteGo’s framing here is the clearest explanation I’ve seen of when GraphRAG actually earns its cost. Microsoft’s own docs split queries into two kinds: local queries, where the answer resembles the question and lives in one place (“which service owns the payments retry logic”) — standard vector RAG handles these fine. Global queries require reasoning across the whole corpus (“which failure causes recur most often across all our postmortems”) — and similarity search fails here for a specific reason: the query vector for “recur most often” lands near documents using the words “recurring” or “frequent,” which is a vocabulary coincidence, not the underlying pattern. Microsoft tested whether bigger context windows just paper over this (8K vs 64K tokens of retrieved context) — the gap in comprehensiveness and diversity stayed open regardless.

GraphRAG’s fix is to build a knowledge graph — entities and typed relationships extracted from your documents — then run hierarchical clustering (Leiden) to produce community summaries at multiple levels of resolution, generated at index time, before anyone asks a question. A global query runs map-reduce over those pre-written summaries instead of grepping vectors. The catch is where the cost goes: two LLM passes over your whole corpus at index time, with entity-description merging alone eating roughly 75% of that cost per Microsoft’s own numbers. Their follow-up, LazyGraphRAG, cuts indexing cost to 0.1% of full GraphRAG by deferring the LLM work to query time and using plain NLP for extraction instead — with comparable quality on global queries and over 700x cheaper per-query cost.

The number that matters most if you’re deciding whether to build this: LinkedIn’s SIGIR 2024 paper reported a 77.6% improvement in mean reciprocal rank and a 28.6% drop in median resolution time after moving their support-ticket retrieval to a knowledge-graph structure. And a caveat worth remembering before you sell this internally as a hallucination fix — Microsoft’s own evaluation found GraphRAG scored similarly to baseline RAG on faithfulness. Its actual edge is comprehensiveness and sourcing, not being more truthful per claim. If most of what your users ask is “which service owns X,” you don’t need this. If you’re regularly getting “what pattern shows up across everything,” that’s the actual signal to build it.

4. How Semantic Code Navigation Cuts Agent Token Costs by up to 36%

Daily Dose opens with two real numbers: Microsoft reportedly cancelled Claude Code for 5,000 engineers this year after token costs climbed to $500–$2,000 per engineer per month, and Uber’s CTO said the company burned its entire 2026 AI coding budget in four months. Both stories got framed around the bill. The piece’s actual argument is that nobody asked what the agent was doing with the tokens — and the answer, most of the time, is finding the place to change, not writing the change itself.

Text search breaks down in three specific ways once a codebase gets real: a name shows up at hundreds of irrelevant locations and the agent has to read each one to rule it out (costs tokens); two things share a name but mean different things — an overloaded method, a shadowed variable — and telling them apart requires knowing what each actually refers to (costs tokens); or the code you need shares no vocabulary with what you searched for at all — an interface implemented without being named nearby, a callback invoked without its name appearing (doesn’t cost tokens, but it’s the dangerous one: a missed rename fails loudly at build time, but a missed structural connection in a behavior change ships silently and surfaces as an unrelated bug later).

The fix the article walks through treats the codebase as a graph instead of text — classes, methods, fields as nodes, calls/implements/extends as edges, each carrying an exact file and line — so an agent can ask “find every place that implements this interface” and get exact locations back instead of grepping and reading. It’s rebuildable without a compiler, so it stays current on code that doesn’t currently compile, which is the normal state of code mid-edit. In a controlled test against real merged open-source commits (10 runs per side, every run required to actually pass build and tests), cost fell in all six tested tasks across four languages, from 5% on a simple change up to 36% on a Java interface change — with the biggest wins concentrated exactly where text search struggles most: changes that have to land identically across many related call sites.

My takeaway: this is a different lever from the “cheaper model doesn’t mean a cheaper turn” point from a couple weeks back. That was about which model you call; this is about how the agent finds context before it calls it at all — and a token dashboard won’t show you the difference between the two.

5. What 8,100 Trials Reveal About Why AI Agent Skills Work — and Crash

AlphaSignal’s Sunday deep-dive covers a new academic study (8,135 trial records across Terminal-Bench 2.0 and SkillsBench) that tests an assumption most of us have never actually checked: that a skill helps an agent because it teaches it something. It doesn’t. The study compared raw execution, raw workflow-memory logs, and a distilled, standardized skill using identical prior experience, and found the distilled version beat raw logs by 6.06 percentage points — and that procedural anchoring (showing the agent the steps) accounted for 65.7% of successful skill cases, versus just 4.5% for explicit knowledge injection. A skill’s job is to keep an agent from getting derailed mid-task, not to hand it facts it didn’t know.

Two findings changed how I’d build a skill catalog. First, annotation matters more than volume: distilling a skill from a mixed batch of successful and failed trajectories with the outcome labels visible got 74.6% success on later tasks; the identical batch with labels stripped dropped to 40.0% — the agent couldn’t tell signal from noise and baked the noise into the skill. Second, retrieval precision and task success decouple at scale in a genuinely counterintuitive way: growing a skill pool from 5 to 100 options dropped retrieval precision from 29.6% to 3.3%, but downstream task success held steady around 36–39%, because even a “wrong” retrieved skill usually shares enough procedural overlap to still help. The real failure mode at scale isn’t picking the wrong skill — it’s “semantic confusability,” where near-identical skills collapse together in embedding space. The paper’s proposed fix is a two-level router: an LLM first buckets the task by domain, then a strict trigger condition (an exact error string, say) selects inside that bucket — rather than one flat vector search across the whole catalog.

The week in AI news

Models & products

  • Claude Opus 5 reportedly succeeded on 14 of 15 drug-binder targets in a benchmark — roughly 2x the industry-average success rate — with prompts and data open-sourced on Hugging Face.
  • Z.ai’s GLM-5.3 added 1M-token context and posted a 50% coding gain with no architecture changes, per AlphaSignal; its ExploitBench score roughly doubled to 24.4%, which the newsletter says came as an unplanned side effect.
  • Cerebras’ CS-4 runs three wafer-scale chips at a claimed 129.6 PB/s of bandwidth and 10x the efficiency of the CS-3.
  • Claude Code shipped a /design command that generates side-by-side artboards and outputs real code, no Figma round-trip.
  • Meituan’s LongCat-Video-Avatar 1.5 (MIT-licensed) turns a photo plus audio into lip-synced video, running on a 40GB GPU at about 44 seconds of compute per second of output.
  • Anthropic explained its text watermarking scheme this week (a keyed function narrows which next-token candidates are “valid” at each step; detection just counts how often generated text matches that pattern) — the same week Superhuman and The Rundown AI both separately covered Dario Amodei publicly answering critics. Three different newsletters on Anthropic transparency in one week is its own signal.

Infrastructure & money

  • OpenRouter is being acquired by Stripe — a16z and AI Secret both covered it, framing model-routing infrastructure as becoming payments-adjacent.
  • Chip and data-center spending kept compounding: Nvidia backed a $105B Ohio data-center deal, Google locked in a $12.2B chip deal, and Micron announced a $10B AI memory lab, all inside one week per The AI Report; AI Secret separately flagged Google choosing Nvidia over its own TPUs for at least one workload.
  • OpenAI reportedly lost $12.3B in a single quarter, per Daily Bite.
  • Cursor is reportedly building a GitHub competitor — flagged by both Superhuman Code and The Rundown AI on consecutive days.

Policy & labor

  • The AI Report ran “the US is pushing allies to pick an AI side” twice this week (Aug 19 and again Aug 23) — a story that’s clearly still developing, not a one-off.
  • The FDA is reportedly working on both competency tests for AI chatbots and revised testing rules for medical AI, per The AI Report and MyClaw.
  • a16z’s charts of the week flagged data-center construction as a driver of blue-collar hiring; separately, The Median reported that data/AI-sector hiring surged in H1 2026 after a 2024–2025 plateau, based on an analysis of two million job posts.

Agents & robotics

  • Superhuman reported Claude got its own dedicated learning hub, and separately that Unitree’s humanoid robot broke Usain Bolt’s speed record.
  • MyClaw reported Salesforce says its AI agent usage tripled, and that Reddit is turning posts into AI-generated videos.

Tools & reads worth a look

  • LMCache — pulls KV cache management out of the inference engine’s own process so cache I/O (GPU/CPU/disk transfers) stops blocking attention compute on the GPU. Daily Dose cited 14x faster time-to-first-token and 4x faster decoding on H200s running Qwen3-235B with 50 concurrent users. Works with vLLM, SGLang, and TensorRT-LLM, and it’s open-source.
  • ByteByteGo’s list of the 12 most-starred “agent skill” repos on GitHub (as of August 2026) — worth a skim right after the AlphaSignal piece above. graphify turns a codebase into a knowledge graph for agent navigation, which is the same underlying idea as the semantic-code-nav piece this week, just packaged as a drop-in skill.
  • FastEmbed’s MUVERA postprocessing module — mentioned in the multivectors piece above; if you’re stuck trying to get late-interaction retrieval behind plain HNSW without pooling patches by hand, this is the other established way to do it.
  • “I gave my job to Claude for a day” (Build with AI Club) — a practitioner’s account of handing research and judgment calls to Claude for a full day. A good gut-check read against the agent-skills study above: where does procedural anchoring actually hold up when the task isn’t a benchmark?

This digest is a set of notes synthesized from newsletters I subscribe to, not original reporting — credit to the people who wrote the source material: Daily Dose of Data Science, ByteByteGo, HiDevs, AlphaSignal, a16z, AI Secret, The AI Report, Daily Bite, The Rundown AI, Superhuman AI, Superhuman Code, MyClaw, The Median, and Build with AI Club. Go subscribe if any of this was useful — they did the reporting, I just took notes.