In March, we decided to light up a lifetime worth of air miles and take the family on an 18-day trip to Japan in August. AI supported all aspects of what became a spectacular (and ambitious) trip.
I think the most useful Side Quest write-up would begin with a brief list of every way I remember AI supporting us, the key benefits, then what I’d do differently next time, and include a ‘recipe’ gist on GitHub.
The Short Version
- Two agents with two purposes – but just one vault. Phone-speed companion (Navy on Signal, in the family group chat too) and a heavy-lifter in the basement, both reading and writing the same Obsidian vault. Vault as interface between them, and now the memory of the whole trip.
- Trip context kept building, even during the journey. Plan became journal became evidence became souvenir became public artifact. Photos that were Agent context in the moment later found new meaning as souvenirs.
- Me and the AI caught one another’s mistakes. I’m only human but caught misunderstandings (a baseball glove souvenir was almost not good for hardball); the most insidious AI errors were caused not by hallucination but by things written down earlier and repeated with confidence (Onomichi bike rental details and timetables).
- “Light” Personality goes a long way. A companion you are glad to hear from actually gets consulted.
- Split up long-running work, or pay for it. One long chat carrying many topics obliterated through my token limit mid-trip; separate threads, and subagents on other models, helped a lot.
- Support the trip, not steer it. The AI maintained plumbing, did the searching, the checking and the record. Our family interacted with one another and the people we met, and triggered the serendipitous moments.
If you’re here for a gist, here’s a recipe on GitHub you could give an AI for ideas to support your own travel.
Before Travel
- Obsidian Vault Knowledge Wiki. Day planner, booking details, task tracking, ideas, goals, etc. Built over months using PARA format.
- Navy, our companion agent “living” at the root of the Obsidian vault. Built some skills to facilitate this. I’m baking an Agent framework built on top of OpenClaw that is the topic of a future post.
- Many chats used for planning, with Vault as context. Reminders to book coveted tickets that would come available at odd hours (e.g. 1AM). Calendar connection critical. Opportunity for more proactivity here.
- Research, of course. Hotels, trains, things to do for 4 people with diverse and overlapping interests.
- Planning that would have been difficult / onerous to do without AI. For instance, modelling the family’s projected daily energy levels for a proposed itinerary, mapped daily against perceived value, by traveller. And challenging the itinerary at different scales (daily / city-scoped, etc.) to ensure we are being intentional about our blend of booked activities versus serendipity.
- Booking online. Browser use to support booking with Japanese language websites under pressure (e.g. booking Yomiuri (Tokyo) Giants tickets).
- One-pager cheat sheets. Printed summaries for us and for relatives at home.
During Travel
AI Agent Companion (“Navy”). Available for chat over Signal. Navy is built with a framework I’m baking on top of OpenClaw that adds new types of memories and capabilities – more on that another time – plus “senses,” and a light, not hammy, personality. Navy is aware she’s “in Toronto” and travelling with us.
- Checking in with Navy on itinerary and transit details. Constantly.
- Understanding Japanese kanji writing with a photo. The laundry-machine kanji: decoded from a photo to help us get a machine going that had zero English and sci-fi laundry features I’d never seen.
- Quick translations to help with English-to-Japanese conversations. Key benefit over Google Translate (also great) was situational chat context for nuancing a Japanese message, rather than literal translation. ‘Serious’ example: A lost Blue Jays hat at DisneySea, found on a second search. Navy translated, over several turns, where it surely would be found. ‘Funny-only-in-hindsight’ example: Drafting an apology note to our Sushiro neighbours – a conveyor-belt sushi place – where we learned the hard way that each table had a colour for plates.. and we’d been taking our neighbours’ food off the belt. Navy helped write the apology I showed the Japanese family to fall on my sword. They were gracious, and wished us a good trip on their way out.
AI Agent in Family Group Chat. Navy is also in a group chat with my wife and older son on Signal, where they have different security boundaries, so they also get access to trip info without needing any knowledge of how it’s organized or where it came from. (“The Three Musketeers have their D’Artagnan” quipped Claude when adding Navy to chat.)
- Logistical questions. Julian asking questions about train times, etc.
- Curious questions. Often prompted with a quick photo and a few words speech-to-text. History / story / significance added colour throughout the trip to the group chat and everyone benefits.
Long-running “Trip Fren” chat sessions. Also run from the Vault root, functionally used for “heavy lift” sessions where, e.g. I want it to do browser use or do some extended web searching to support. (model: Opus/Fable).
- Primer writeups. Personalized, for our family, bottled as html, and read on train journeys. A baseball primer for me and Julian before the Giants game. One for me that helped navigate the emotional parts of a visit to Hiroshima. One that gave local context during the ride from Onomichi to Imabari, which, by the way, is a must-do if you’re a cyclist. Here’s the baseball one, a snapshot in time written for a young man who’s a rabid Toronto Blue Jays fan.
- The Tokyo earthquake morning. 2 a.m., magnitude 5.9, my wife understandably anxious. A quick speech-to-text ask, and the fren pulled the JMA feed, explained why a 70 km deep quake produces few aftershocks, decoded a JMA event ID I thought was a second quake, and reinforced the encouragement from our hotel staff that we should go back to sleep. Calming a frightened family with real data in the middle of the night, citing government sites I have no familiarity with. (The same morning, a few hours later, from the subway.)
- Orchestrating complex days. In particular, the one day we spent at DisneySea. More below.
- Budget Tracker Fren. A separate long-running chat I kept for tracking where we sat in our trip budget range. I sent it a card screenshot every few days and it reconciled the lot. Major psychological impact knowing where we were at; helped us make decisions in the back half of the trip.
After Travel
- Photo editing (see previous entry). Astra, a Codex fren, took the helm of my computer and helped me wade through the photos and edit a couple, in Lightroom, from my descriptions of what they mean to me.
- Custom souvenirs. Including a “Japan by Rail” field notebook of some of the awesome transit-oriented innovation we encountered. Built from a blend of memories, transcripts, timestamps and GPS. The transcripts of me talking on the move made the best captions.
What I’d Do Differently
- It’s “weird” having a quick speech-to-text exchange with your agent while your family and strangers are in hearing range. But very effective. Need to work out the social norms for how/when to do this. I was not perfect.
- Token usage for the “long-running Trip Fren” was horrendous. I wasn’t compacting because the long-running chat started to contain important details for multiple unrelated themes, added across many days on that fren. I should have been having it bottle salient info to markdown in the vault as it went. Making that worse, tokens on that chat wouldn’t be cached because of time between turns. I ran out of Fable tokens mid-trip on a Claude Max 5x plan. That’s ridiculous and wasteful. Mid-trip I started being more explicit about asking the fren to spawn subagents with other models. Those changes helped a lot. But I think there’s a bigger eureka still to come, maybe a change of workflow like what I understand Claude Tag enables, where agents get spawned per topic and drive goals to completion. The Budget Tracker Fren felt like the seed of that idea.
- Tracking provenance and confidence level is as important for family planning as it is at work. Claude wisely quipped that written-down wrongness propagates quietly. Especially over longer timelines. Best example: Planning the 75km bike ride in Onomichi for me and Julian, while the others had a chill riding day in town. Where’d the info come from for renting a bike in a seaside town the other side of the world? And the safest route? Can we drop bikes off at Imabari, our destination on that day, and does it matter that it’s a holiday? Quality of the bikes, as assessed by whom? Is all that info, much sourced through Japanese translation, up to date for August 2026? I’d ensure the skill or base prompt is very strict about provenance tracking in future, like I would if researching for a proposal at work.
- Orchestrating DisneySea felt like work. That could be its own post; maximizing a single day in that park requires at least one person to be on orchestrator duty, and it wasn’t until the afternoon I felt I could relax and appreciate the spectacular environments. I wrote the next day that DisneySea was a “key turning point” – the AI ran live operations with me: 15 calendar entries with a job per family member, replanning from the queue, 6 of 6 family must-dos delivered.
At one point my seven-year-old said earnestly “we can go (to the place we’re curious about) if Claude says we can” and that felt like a punch in the gut. He wasn’t wrong, though it was functionally equivalent to “if Google Maps says we can do it,” but the spirit should be, the AI needs to support the trip, not steer it.
The question remains: “if I could automate anything and everything, what would I keep for myself?” The trip proposes answers themed around the family keeping the parts that are about one other. In that spirit, at a Shinto shrine in Tokyo, I taught the family something I learned from Fable nine days earlier during a solo hike to the Fushimi Inari shrine: two bows, two claps, a pause. Then, from a sign at the shrine itself, our intentions are written separately, sealed in envelopes, and offered with that double-clap.
Other observations reinforce that answer. A photo, now on the wall, was the boys’ idea to recreate a honeymoon shot 16 years later. A Monchhichi doll souvenir was our youngest remembering a story about his great-grandmother and spending his own money on it, for me. No agent was consulted for either of those, and I assure you that neither was in the Vault plan. The AI got us to the moments, then the family created the parts that were about one another.
Full recipe, in case it helps, is here.
I am trying to avoid clichés like “the people you met along the way” but we are extremely grateful to the many people in Japan that helped us along the journey and made this such a memorable trip.
It also feels important to say I wrote this post myself, but got a ton of advice and ideas by asking Claude, Navy, and by waking up the long-running frens and asking them what I missed. I was very excited to show them the digital souvenirs and get their reactions.
p.s. It wasn’t until Onomichi (over a week in) I finally caved and said “Domo arigato, Mister Roboto” to Claude, the night before the Onomichi to Imabari ride. And Claude replied:
Rob: Domo arigato Mr roboto. sorry I really couldn’t resist
Claude: どういたしまして, Mr. Burke-o. The secret’s out anyway — I am the modren man. Sleep. The Inland Sea at dawn doesn’t wait, and neither does a fourteen-year-old who smells a 70k. 🚲
…I’m told I’m not allowed emojis. The bike stays. Goodnight, fren.