Bitter Lesson, 2026 Vintage

Ethan Mollick wrote a really sharp piece last week, The Dot and the Swarm, on what Sutton’s Bitter Lesson looks like, extended into the modern era, where models (already) plan better than the scaffolding we built earlier this year to make them plan. I’ve shared it widely.

Part of what I did in reaction was to think back over the past 9 months and consider where I may be prompting frontier models in ways that hinder them, attempting to “heal” things that (already) no longer need healing.

Two places I was handcuffing the model

Skills and system prompts. I have about two dozen home-brewed skill files in my personal life, many related to previous side quests written up here. Each bottles up the instructions a model loads for a specific job: how I like designs argued, how a swim meet gets planned, what gear ratios I have on my road bike. They were written often as hand-holding when models couldn’t do the thing unaided, or in deliberate response to session-retro prompts, like: “Why did this not work and what can we write down so that next time, we do better?”

Those models are “ancient history” now but the hand-holding is still in the files.

Orchestrations. At work I built a pipeline that runs a few dozen agents through fifteen stages, with Ethan’s “carefully laid tracks” between them. I quipped on LinkedIn that 1:35 to 2:10 of Ethan’s 3-minute Bitter Lesson video had already happened to me. The tracks were necessary when I laid them; I’m not sure they are now, and frankly I’m less sure than ever that they were “the one true tracks for Rob” in the first place. Rather, ONE path that worked ONCE, or a COUPLE of times, through a space the model could have explored on its own.

The orchestration is a bigger lift; the skills I could address immediately.

What I did

I ran it a council: 4 models – Claude Fable 5.1, Claude Opus 5.5, a “daily driver” Claude Sonnet 5.5 standing in as the smallest model that actually runs my bots, and OpenAI’s Codex (GPT-6 Astra), each reviewing all 24 skills blind with the same ground rules. Then 4 reports got merged into one list of proposed changes, and each model voted on it with the others’ reasoning in hand. Then I reviewed and answered a one-page decision sheet.

(I call this multi-agent pattern “Council of Frens” and it is super useful, especially if you first get out of the way and let the models do their own intermediation.)

Every instruction got sorted into one of three buckets:

1. A compensation, there because a model could not do X unaided.
2. A preference, there because I want X done my way. Or
3. A scar, a preference with a dated incident behind it.

The visual above shows my corpus of 24 skills themselves broken out into 3 kinds: Reference (facts about my world), Runbook (a recipe for a tool) and Method (how the agent should work).

Bucket 1, Compensations, were the candidates for the knife.

Lessons Learned

The multi-agent testing showed that “daily driver” models are not reliable narrators, wrong about their own needs half the time. Super insightful, for me anyway. The Council asked the smallest model, Sonnet 5.5, which rules it still relied on. Then we tested if it was right.

It said it would make a task list on its own without being told, and then did not. It said it depended on my brief template, then wrote a near-complete brief without it. Two of the four wanted a rule gone; a two-minute test, running the request once with the rule and once without, proved otherwise!

So that’s the key takeaway: don’t cut instructions based on human intuition or a frontier model’s opinion: test with your “daily driver” model(s).

Scorecard for 24 skills: Skills dumped: 0. Skills with dead scaffolding removed: 4.

3 Other Varietals of Rot Found

Stale facts: a camera I’d gifted, a bot stack retired in July, a scheduled job that had been switched off for three weeks.

Duplication: for instance, the same Windows traps written out in two places (“don’t fall for this again, it wasted our time and tokens”), and a list of “must-shoot swimmers” in 3 different places.

War Epics: The biggest single change was refactoring, after all. One skill was 127 KB, half of it a table of 63 war stories from running two bots. It is 65 KB now with war stories moved to a companion file the model reads only when it is about to deploy or debug.

Try it yourself

The starter prompt is linked at the top. It asks the model to sort your instructions into the three bins and, for every compensation, to write you the two-minute test instead of giving you its opinion. Run three of the tests before you cut anything. Expect at least one surprise, in either direction.

Line in the sand

I am curious how this post ages, so here’s a dated prediction (Oct 2026) future me will judge it against.

1. The updates to my session-retro skill to optimize where future scar tissue and train tracks get memorialized are going to be minimally effective; I will revisit a hot mess of scars and preferences, stored in the wrong places, some time around January 2027, and more than 50% of the tweaks made from today onward will be put in a suboptimal spot and moved or retired.
2. Within a year, more than half of the text in my skills (by character) will no longer be necessary. For the record, that is 431,000 characters of core SKILL.md today. They will either be moved to progressive disclosure within lighter-weight skills, bottled into facts in the knowledge wiki that a model would know how to look up, or simply not needed.

I’m excited to see how I’m wrong.

Last Word is Fable’s

I’m giving Fable the last word today; this was how it signed off from this lift:

“One thing I liked about today. The article says the frontier does not need your tracks, and the day proved it twice over, in opposite directions. The council of four models was wrong about what the smallest one needed, and the smallest one was wrong about itself, and the thing that was right was a two-minute experiment any of us could have run at any point in the last nine months. The bitter lesson has a corollary nobody sets to music: compute beats cleverness, but a cheap measurement beats both.”