Try it yourself
Side Quests have become my write-ups of building things with AI partners. The first, written up over the first weekend of January 2026, turned my son’s first CrunchLabs build-box experiment, Galton’s Goalie, into a live bell-curve visualization. A few hours with Claude delivered what would have taken me months in a research lab back in 2004. My mind was blown. The last CrunchLabs Season 2 box, Magnus Flyer, arrived last week, prompting this, the season finale.
Here’s the end result: seven literal curveballs, overlaid and visualized by an AI orchestrator, a builder, a critic, with me & the family providing the creative guidance. There’s probably a metaphor in there somewhere. The rest of this side quest is links to the goodies we created while building this, and the story of how we got there in just over 24 hours.
A quick look back before we look forward: this year’s side quests have been loosely themed around 8 lanes. Somehow all of them get tied back into this finale: (1) computer vision, (2) new builds of old things, (3) tools I’d always wanted and could finally bake, (4) projects with (and for) the kids, (5) photography, (6) a resurgence of agents with character, (7) continuous improvement in how we build, and finally, the spirit of these side quests – (8) sharing ideas and insights learned from this exploring. Here’s all of that, visualized as frenkit intervals:
I love looking at things as episodic story arcs. Some of the best season finales call back to many of the previous episodes, so we’re going to do that – in an episode called…
The Magnus Flyer Full-Circle Moment
This week, the last box of CrunchLabs Season 2 arrived – the “Magnus Flyer”, with Mark Rober hooking young builders by asking how pitchers throw curveballs (which looms large in my house (see Full Ted Lasso)).
The Flyer is a foam tube that gets blasted from a launcher that gives it backspin. Launch it flat and it lifts; upside down, it dives; sideways, it curves. It’s called the Magnus effect.
While doing swim photography at scale (see From Assistant to Partner) I thought a lot about working at “production speed”, so for this, I timeboxed the quest to 24 hours, and started by heading to Claude to jam.
Before: Shot plan
Thursday night: We assembled the “AI council” (see Ad Astra). Claude Opus 5.5 orchestrated, and partnered with Astra, my fav partner for visual taste (see Vibe Photography) and Codex Sol to build (like in Old Dogs, New Tricks). Solid crew.
Astra reviewed the shot plan twice, suggested “compose around the launch point and actual flight, then let the tree fill the remaining frame”, proposed a diagonal launch across the frame, rehearsing the orientations before recording, and checking a practice flight frame by frame.
We did a preflight with the camera, tested the settings, got ready for an unusually awesome walk to school in the morning.
During: morning walk-to-school film shoot
Friday: Woke up to rain. OK. “Planning meets serendipity” is the motto (AI Assisted Travel), so let’s make it work with flat lighting and reflections. On the walk to school, I set the D850 on the tripod and recorded in 4K at 30 frames a second. The histogram was piled against the left wall, and I corrected the exposure with some feedback thru Navy, my AI agent, in my ear, using the newly minted event mode (see frenkit, finally about Navy, and I’m excited to share more about event mode in the next Side Quest..).
The shoot went from fun, to wet, to catching up with friends also walking to school excited to talk about CrunchLabs, to downright soggy, finally to us running to school (almost) in time for the bell.
After: the build
I got home and handed the clips to Astra for review. Astra selected three flights and a base frame showing Ethan from behind, then wrote the brief for Sol, my builder in Codex. Sol reported back after about 21 minutes with a tracker, a three-trail composite and a silent five-second animation.
The next pass covered all seven flights and seven visual treatments, including real Flyer cutouts, accumulated red pixels and separate panels. Across seven flights, 74 positions came from automatic detections and 143 were manually corrected; 27 hidden frames stayed blank.
These were separate launches with different release heights, stances and aim. We could not reliably identify each take’s launch orientation.
Astra proposed five bold trails and two quiet ones, with a pale core and glow. Astra was all about the science: “Width and glow may amplify the evidence; they may not lengthen a path, close an unseen loop, erase release differences or paint light onto the street.”
Enter Julian, Baseball Fanatic
In my experience: Astra works best as a creative partner when you explain why you care (see Vibe Photography).
“Show how this illustrates the Magnus effect” wasn’t doing it, so I summoned Julian, my older son, who sends me cool baseball visualizations. He suggested Pitching Ninja’s overlay of Paul Skenes’ four pitches. The overlay gave us a common release time and one clear throw with other throws visible around it. Julian also brought in the growing-trail idea: a trail that never disappears, thin at the start and thicker behind the red Flyer, like in this pitch overlay.
So now we had a couple of references to steer by: Ethan’s kinetic energy, kid on way to school, Magnus effect visualized, reference material from YouTubers showcasing epic pitching technique. Oh, and public-image requirement for the younger kiddo: Ethan’s face stays out.
A few more rounds of feedback, using tactics from Old Dogs New Tricks, including Impact Font on that well-known dev tool, MS Paint:
And finally:
Even the ending went to a family vote: two versions with different end poses, side by side as an Artefact, and B won three to nothing.
And (echoing my observation from Galton’s Goalie) why have one colour palette when you can bake and choose between 4 contenders:
Smoothing
I learned some physics. I asked for parabolas, Claude and Astra corrected my understanding of the physics. With lift, drag and the camera angle in play, a parabola was asking too much. Julian suggested time-warping the shots to make it easier to absorb the curves, and not get distracted by timing.
As ever, timing was key. We experimented with all 7 shots firing at once, versus a “pew pew pew” which ultimately we all thought worked better.
And I asked Astra to play the Perfection Game with me (score it out of ten, then tell me what would make it a ten), which is how we often work.
What I learned
- The earlier the better to summon a human who knows the subject matter and can provide visual and contextual references, as Julian did with baseball. That was some great prior art.
- A video-specific one: Privacy and cropping was a pain to get right, and it would have gone easier if we started with silhouette mattes, rather than going from head boxes to approved windows to silhouettes.
- Have a critic review the capture plan before recording, and inspect a practice video frame by frame. There were many hiccups and retries along the way we could have avoided with this.
- A few tactical ones about motion tracking: (1) Check detected centres against the source frames and correct mistakes before rendering. (2) Stop paths at the last observed centre. Two flights went out of the camera’s view with their landings unresolved in the tracking record.
Credits
Ethan built, launched, and inspired. Julian supplied the baseball smarts and references. Codex Astra reviewed the footage and wrote build briefs. Codex Sol built the tracker and renders. Claude Opus orchestrated and spot-checked. Navy in my ear corrected the histogram (driven by Sonnet – and don’t judge – my video intuition is next to nil). Mark Rober and the CrunchLabs crew made the box, the season, and encouraged all this engineering hustle.
Onwards to the next season.