Another dense week in AI. Here’s what I pulled from the newsletters — filtered for signal over noise, with my own read on what actually matters for engineers.
Deep Dives
1. Speculative Decoding Is Crossing the 4x Barrier
Source: Daily Dose of DS — “Speculation Is All You Need!” (Jun 24)
Standard speculative decoding has been stuck at 2–3x speedup for a while, and the ceiling was always the drafter — the small autoregressive model that proposes tokens one at a time for the large target model to verify. Modal’s DFlash changes the game.
How it works:
- Normal speculative decoding: small drafter proposes K tokens sequentially → large target verifies them all in one forward pass → keep everything up to the first mismatch, discard the rest.
- The drafter’s sequential generation is the bottleneck.
What DFlash changes:
- Replaces the autoregressive drafter with a block diffusion model that denoises an entire block of tokens in one parallel pass.
- The drafter also ingests hidden states from multiple layers of the target model rather than the raw token sequence — so it starts from the target’s internal view of context. This is the key insight: the drafter isn’t guessing cold, it’s working from the target’s own representations.
- Result: the drafting cost stays flat regardless of block size, and the draft tokens align with the target more often.
Why acceptance length is everything:
Decode is memory-bound, not compute-bound. A 100B+ model spends most of each step moving weights from GPU memory into compute units. Reading those weights costs the same whether you’re verifying 1 token or 20. So every additional token accepted per verification pass is nearly free throughput — which is why the acceptance length maps almost linearly to speedup:
| Acceptance length | Speedup |
|---|---|
| 2 | 1.86x |
| 4 | 3.57x |
| 8 | 5.62x |
Modal trained their DFlash drafters on the target’s own output distributions and real production traffic, pushing acceptance length from ~3 to 9+. Running Qwen 3.5 122B-A10B on a B200 hit 1,000+ tokens/sec vs 250 without speculation.
My take: If you’re self-hosting inference at scale, training a custom drafter on your production traffic is now the highest-leverage thing you can do for throughput. DFlash models are on HuggingFace and already integrated with vLLM, SGLang, and Transformers.
2. RAG vs Graph RAG vs Agentic RAG — The Actual Trade-offs
Source: ByteByteGo — “EP220: RAG vs Graph RAG vs Agentic RAG” (Jun 27)
ByteByteGo put out a clean taxonomy I keep wanting to reference. Here’s the compressed version:
Standard RAG
- Query → embedding → vector DB → top-K chunks → LLM
- Fast and cheap, but if the wrong chunk is retrieved, nothing catches it
- Best for: document Q&A where speed matters and the answer is self-contained
Graph RAG
- Query classified: specific → local search (vector DB + knowledge graph traversal); broad → global search (pre-built community reports, no graph traversal)
- Expensive to build, slow to update. The graph is the investment.
- Best for: structured knowledge domains — legal, compliance, biomedical, anything where relationships between entities matter
Agentic RAG
- Reasoning agent reads query → breaks into sub-questions → picks sources → multi-source retrieval → another agent checks sufficiency → re-retrieves if needed → synthesizes answer
- More capable, self-correcting, slower, harder to debug
- Best for: multi-step reasoning where a single retrieval pass can’t possibly answer the question
Standard RAG is where most teams should start. Agentic RAG is where you end up when standard RAG keeps failing on the hard cases.
3. Karpathy’s Agentic Engineering Framework, Now With Tooling
Source: Daily Dose of DS — “Karpathy’s Agentic Engineering Finally Has Proper Tooling” (Jun 28)
Karpathy defined agentic engineering at Sequoia Ascent 2026 as three core skills: spec design, eval loops, and security oversight. The gap until recently was that practicing it meant juggling an editor, terminal, browser, cloud console, and a separate eval framework.
Google’s Agents CLI closes that loop. One install injects seven skills into your coding agent covering ADK patterns, project scaffolding, eval setup with LLM-as-judge scoring, deployment targets, and observability. The same skills land across Antigravity, Claude Code, Cursor, and Codex simultaneously.
The 6-step lifecycle:
uvx google-agents-cli setup— injects ADK skills into your coding agent- Build — describe the agent in natural language; the coding agent scaffolds the full project
- Test locally — ADK Web UI on localhost; verify retrieval accuracy and refusals
- Eval (the most-skipped step) — generate test scenarios by category, run LLM-as-judge scoring
- Deploy to Agent Runtime — Cloud Trace enabled by default
- Register to Gemini Enterprise — makes the agent discoverable org-wide with IAM access controls
The eval data point buried in here: 89% of teams running agents have observability, but only 52% have evals. That’s the gap Karpathy’s been flagging. In their demo run, citation accuracy was 1.00 across 20 eval cases — but the eval also caught a hallucination edge case (the agent appending general knowledge when it should have refused) that was traced to a single instruction line. An eval caught it before deploy; observability alone wouldn’t have.
My take: The eval gap is real. Most teams instrument their agents for production visibility but skip systematic pre-deploy testing. The Agents CLI makes generating a coverage-driven eval suite a single prompt, which removes the friction excuse.
Week in AI News
-
White House limits GPT-5.6 release — The Trump administration asked OpenAI to roll out GPT-5.6 to select government-approved partners before any wider release, citing capabilities on par with Anthropic’s Mythos. Sam Altman told employees this was the path to a general release “a couple of weeks later.” The pattern: as frontier models hit certain capability thresholds, government sign-off is becoming part of the release process. (The Rundown AI, Jun 26)
-
Anthropic accuses Alibaba of the largest known distillation attack — 28.8 million Claude exchanges extracted through ~25,000 fraudulent accounts over 45 days, targeting Claude’s agentic reasoning, coding, and long-horizon task capabilities. Anthropic is calling for antitrust clarity to let AI labs share threat intel and stronger chip export controls. For context, February saw DeepSeek, Moonshot, and MiniMax involved in attacks totaling 16M+ exchanges. (The Rundown AI, Jun 26)
-
OpenAI ships Jalapeño, its first custom inference chip — Co-designed with Broadcom, purpose-built for inference only. GPT-5.5 Instant is now accessible via “chat-latest” in the API. The motivation is hard to miss: $20.9B operating loss concentrates the mind on unit economics. Every major lab is vertically integrating silicon. (Alpha Signal, Jun 25)
-
GLM-5.2 is triggering a “DeepSeek moment” for agentic coding — Another Chinese open model forcing a rethink on closed AI pricing. The autonomous coding dream gets cheaper again. (Alpha Signal, Jun 28)
-
Claude Tag launches in Slack — Anthropic debuted Claude Tag, bringing agentic capabilities into Slack channels. Tag @Claude with a task, it breaks it into stages, works through them using approved tools, and responds when done. Builds context across channels over time. Karpathy called it the “3rd major redesign of LLM UI UX.” Expect this to hurt a few “agentic coworker” startups. (The Rundown AI, Jun 24)
-
SpaceX acquires Cursor for ~$60B — Grok Build ships a joint Cursor-SpaceX model inside the IDE. (Alpha Signal, Jun 23)
Tools & Reads
-
ART (Agent Reinforcement Trainer) by OpenPipe — Open-source RL framework that sidesteps manual reward engineering for multi-step agents. Let the agent attempt tasks multiple times, an LLM judge grades attempts relative to each other, the model learns from what worked. Uses GRPO (same algorithm behind DeepSeek R1). Native support for tool calls, multi-turn conversations, LangGraph/CrewAI/ADK integrations. GitHub (Daily Dose of DS)
-
Sakana Fugu — Auto-routes requests across model pools via a single OpenAI-compatible API endpoint. Swaps on export blocks, no code changes required. SWE-Pro/GPQA-D/ALE-Bench ranked. For teams running multiple models and tired of managing routing logic manually. (Alpha Signal, The Rundown AI)
-
Nous Research Hermes Blank Slate — Agents start with exactly 4 defaults; web, memory, and MCP stay off until explicitly added via hermes tools. Interesting design philosophy: zero-trust defaults for agentic systems. (Alpha Signal, Jun 22)
These are synthesized notes from newsletters I read each week. Content credits: Daily Dose of DS (Avi), ByteByteGo, The Rundown AI, Alpha Signal. Views are my own.