Cross-Country Olympic Mind Games

During the Winter Olympics, the lights of the CN Tower are taking part in an interactive art installation that’s controlled by a participant’s brainwaves.

What I like about the CN Tower installation (in addition to the cross-country nature of the biofeedback loop) is how they teach participants to generate the correct brainwave patterns. We faced a similar challenge back at MediaLabEurope.

CN Tower Light Show
CN Tower Light Show

During the 2010 Winter Olympics, the lights of the CN Tower are taking part in an interactive biometric art installation.

From 3,300km away, a participant in Vancouver has their brainwaves monitored as they watch a live video of the Toronto skyline. When they are inattentive, the CN Tower’s lights move at a lethargic pace. But as the participant concentrates, the lights of the CN Tower cycle around with the speed of Christine Nesbitt ’round the speed skating track.

The measurement being used to determine a participant’s attentiveness is their ratio of their alpha waves to beta waves. This very rough estimate of attentiveness is also what we used for an early project in the MindGames group called “BrainChild”, where a participant unlocked a door through sustained concentration.

What I like about the CN Tower installation (in addition to the cross-country nature of the biofeedback loop) was something I read in this Torontoist article. If participants are having trouble “concentrating,” they are encouraged to count the lights in skyscraper windows.

This is a very clever, indirect way to achieve the desired result. When working with “untrained” biofeedback participants, it can be difficult to articulate the subtle changes that they need to make which will lead to a desired signal (in this case, a change in the ratio of alpha to beta waves).

Mind Balance Training

Mind Balance Screenshot
Mind Balance Screenshot

A project we built at MIT MediaLabEurope called Mind Balance faced a similar challenge.

Mind Balance also relied on an electroencephalogram (EEG) metric to create a control mechanism. However, instead of measuring the ratio between alpha and beta waves, we were monitoring the occipital lobes at the back of the head to detect artifacts from the electrical signals produced by the brain’s visual processing.

The subtlety there was that success for a participant required not just having a visual pattern in your field of view, but also attending to that pattern – a “squishy” concept that required training. Some of the “indirect” methods we used to explain a “good stare” included encouraging a participant to “stare right through” a region on the screen.

We used a 45-second acoustic feedback training session to help participants learn this concept assisted by very clear and immediate feedback. It worked with varying degrees of success, but thankfully, enough success in over 95% of cases for a participant to successfully go on to generate a single reliable control axis.

We resurrected Mind Balance for the Microsoft Ireland Visual Studio 2005 launch and it was awesome (and somewhat improbable) that we got it to work in an Irish pub setting. You can read more about Mind Balance here, and check out the shots of our prototype headgear called Cerebus.

(Another MindGames project where indirectly achieving biometric results played a big role was Relax to Win. The techniques players used to achieve relaxation under pressure was an important and fun part of the interaction.)

Jedi Mind Tricks Re-Re-Rediscovered

Ed Lalor wears Cerebus
Mindgames researcher Ed Lalor models Cerebus

Excitement around Brain-Computer Interfaces for commercial and artistic use seems to resurface every few years.

Although the headgear for the Olympic installation looks considerably refined compared to our prototype (pictured here), achieving reliability for novel and useful BCI metrics remains a really tough problem, principally because most people aren’t cool with having someone drill into their head, which is a shame, because the brain’s electrical signals are so much weaker by the time they reach the surface.

Later this year, with the imminent launch of Project Natal, the focus now seems to be on visual and acoustic interfaces.

But during the Olympics, it’s all about the Mind Games. My hat’s off to Interaxon for putting together this engaging cross-country installation.

(More about the Olympic mind-controlled art installation at PopSci, Interaxon, Torontoist)

(More about Mind Balance here with links to pics)

Standards are the Sands of Time (or, the Illusion of Preservation)

On a long enough timeline, the survival rate for every standard drops to zero.

From emulated games to digital negatives, how can we preserve our digital content for posterity, and inform our decisions about which formats we adopt?

A special jpg: 2 monkeys, city, moonlight
A special jpg: 2 monkeys, city, moonlight

How many jpg images do you suppose are living on the internet? Probably quite a few.

Among all those images, the jpg on the right is particularly special to me, because it’s a screen capture of an adventure game I wrote for the Commodore VIC-20 when I was very very young (like almost single digits).

If you can’t make it out, those are two monkeys standing under the Toronto skyline, reading a newspaper by moonlight.

At the time I wrote the game, it could only be played by reading the bits that were encoded as waveforms on an audio cassette tape. Which was awesome.

But one day my VIC-20’s cassette drive silently stopped reading tapes, and the game could be played no more.

The photograph below (another jpg) was taken of the defunct
VIC-20 hardware some time before it was tossed into a big dumpster.

Commodore VIC-20 and Tape (deceased)
Commodore VIC-20 (deceased) and Cassette

Resuscitation by Emulator

VICE Vic-20 Emulator on Windows
2001 Emulator (not dead yet)

Fast-forward through more than a decade to 2001, where I find VICE, a program that can emulate the VIC-20 on Windows.

With the game’s cassette reclaimed from the basement and a dusty tape player connected to my PC’s audio port, I transferred the audio that made up the game’s bits into an audio .wav file, and then converted that audio file into a format that could be loaded by the emulator.

I was so happy to be able to run my game again, and have it preserved “forever,” even though its contents are meaningful only to me (and maybe my family).

Satisfied, I zipped it up together with the emulator
and put it into my folder called “C:\pastlives” for safe keeping.

Now almost ten more years have passed. In this 64-bit world, my game still runs.

For now.

(slightly) Longer Term Thinking

This ran through my head when I was considering whether I should adopt Adobe’s Digital Negative (DNG) format as a standard for my photo archives. The alternative is the Nikon raw (NEF) format I currently use for my negatives.

It literally kept me up last night thinking about the layers of technology I depend on to retrieve and view things in “C:\pastlives”, like my VIC-20 emulator or my digital photos.

Check out this stack of just some of the standards and technologies I used to play my game today:

Some of the technology standards used to play my VIC-20 game on Windows 7
Some of the technology standards and file types used to play my VIC-20 game on Windows 7. (another jpg)

(Red == the ones I perceive as more volatile)
Sure, you could substitute for some of these (the VIC-20 emulator, for example, also runs on Linux).

But how likely is it that a complete cocktail of prerequisite technologies will be around long enough for me to load and play the game in three more decades? In ten?

These standards aren’t stacking up like a solid foundation, they’re piling up like sediment!

(To abuse a pretentious analogy) It’s like the sands of time themselves.

Consider the Floppy

5 1/4 inch Floppy Disk
5 1/4 inch Floppy Disk

Remember floppy disks? Those once-ubiquitous storage devices? How many of you still have machines around that can read a 5 ¼ inch floppy?

On a long enough timeline, the survival rate for every technology standard drops to zero.

By my estimation, all it’ll take is for me to neglect its preservation for approximately a decade, and my game shall be played no more.

Spoiler Alert!

I’ll go out on a limb and say that despite my efforts, my VIC-20 game will never outlive even all the jpgs in the internet.

So I’m just going to tell you that one monkey rescues the other monkey from the Toronto zoo and they escape back to Africa.

There’s an action sequence where you have to climb through the sunroof of a taxi and jump off at just the right moment to escape a high-speed police chase.

Screenshots, while they last, are available upon request.

Roomba

I finally watched a Roomba dance its funky dance.

The iRobot Roomba 550 we affectionately named Butler spent a half-hour cleaning the 4 downstairs rooms of a home, sucking up an impressive gob of dust from the previous night’s party.

Hello, Roomba
Hello, Roomba!

I finally watched a Roomba dance its funky dance.

The iRobot Roomba 550 we affectionately named Butler spent a half-hour cleaning the 4 downstairs rooms of a home, sucking up an impressive gob of dust from the previous night’s party.*

As I watched Roomba, I realized I’d forgotten…

How much we are inclined to anthropomorphize technology.

How we’re wired to perceive complex reasoning where only simple behavior exists.

How important it is to have users on your side!
(Roomba was occasionally bumbling but always so helpful — looking everywhere for last night’s crumbs, remembering which area needs more attention, and just generally “doing his best.”)

First Impressions

The design of the unit is clean, sleek, and functional.

Roomba in Motion across the rug
Roomba in Motion, scooting across the rug

The “wall following” behavior is particularly clever and plays to the strengths of the round unit.

Object detection mostly works (it’s supposed to slow down before a bumper-kiss), but it was blind to some antique table legs on our test run, threatening to knock over some photo frames and antique china.

Its motion is smooth, and it successfully un-stuck itself from the curtains.

I loved watching it ultimately find “home base” by IR and dock with it, 2001 Blue Danube style.

But I worried..

  • About non-techie users having to choose a “home base” location. A sleepy Roomba needs to locate its home, but a resting Roomba is hardly showpiece décor.
  • About long-term battery wear and life.
  • About how often you’d need to empty the dust out of his bowels, and what happens if you forget to do so.
Roomba Cleaning Patterns (from the manual - click for .pdf download link)
Roomba Cleaning Patterns (from the manual .pdf)

Navigation & Hackery & Papers

Roomba uses behaviors like spiraling, wall-following and room crossing (as explained in the image, right, from the manual) to create its not-quite-completely-random walk.

And below, check out this very clever 30-minute long-exposure photo (found at SignalTheorist via Botjunkie) that reveals the Roomba’s “Lovely, Inefficient” cleaning path.

Roomba Path Long Exposure
Roomba Cleaning Path – Long Exposure Image

This brings us inevitably to the hacking. iRobot, to their credit, encourage you to hack Roomba!

Martha, whose Roomba apparently can bring her beer (wait… what?!), recommends the book Hacking Roomba. Here’s the book’s companion website.

Is anyone still using Microsoft Robotics Studio for stuff like this?  (ah, the memories of dancing Lego robots.)

In the meantime, with my head full of Roomba, it’s the perfect time to re-read some of iRobot Founder Rodney Brooks‘ seminal papers, like Intelligence Without Representation and Elephants Don’t Play Chess.

Because Elephants these days apparently can clean living rooms!

*the dust was left behind from the previous night’s party, in an otherwise impeccably clean home (i.e. not mine)

The Unfolding of Language

deutscher
Deutscher's The Unfolding of Language

When Stephen Fry laments “it is a cause of some upset that more Anglophiles don’t enjoy language,” it’s as if Michael Phelps were to lament that not enough people enjoy water. So when Stephen wholeheartedly recommended Guy Deutscher’s The Unfolding of Language, which he characterized as more playful and engaging than books on similar subject matter, I was hard pressed to say no. It’s taken me ages to find time to get into the meat of this book (strictly my own fault), but now, about two-thirds of the way through, I wanted to offer it my wholehearted recommendation for anyone who is even remotely interested in language and its origins and evolution.

Deutscher’s prose is indeed playful and accessible, his examples thought-provoking, and his subject matter fascinating: what are the forces that shape and transform language?

Deutscher mentions more than once that “These days, there are no systems of communication which are in the process of evolving their first words.”  He’s right, I suppose, but only on a technicality.  Last week I was taught the basics of a computer scripting language I’d never worked with before. Surely the constructs of some arbitrary scripting language represent one of many “artificial” systems of communication which are in the process of evolving their first “words” (and tokens).

I am writing this now as Deutscher transitions in the book from talking about the destructive forces which are applied to language (which favor economy, expressiveness, and analogy), into the constructive ones which enable new linguistic richness to blossom. Metaphor, apparently, provides many of the raw materials for new grammatical elements.

With that observation under my belt, and aspiring to be a creative force in the universe, I suddenly feel a bit better about my obsession with admiration of Roger’s Profanisaurus (a dictionary of profanity that originates in the pages of the UK’s Viz magazine, which derives cleverness and vulgarity in equal measure from a playful, multi-layered cocktail of metaphor, rhyming slang and other wordplay).

And of course there are my dear LOLcats, who reflect (again in equal parts) the absurd and absurdly rapid evolution of linguistic memes as they’re propelled at the speed of the internets. Since I’m in Ireland, and Deutscher recently reflected on the necessity of the word possessing-implying “have”, here’s a somewhat appropriate LOLCat I just cooked up  – with my cap off to Jim Condron for his help with the Irish word for “flavr.” (context here for the uninitiated)

Orish Kitteh Ubserves: Deres a flavor on meh
oirish kitteh tinks: deres a flavr on meh, so dere iz.

Back to Deutscher’s book.  He spends the fifth chapter illustrating a point by employing a fictional dialogue between a cast of characters at a ‘George Orwell Centennary Conference’.  it’s a technique akin to the one I admired in Hofstadter’s Godel, Escher, Bach. Actually, that’s all I have to say.  Check it out.  Go for a swim.  And apologies to Deutscher, Mellie, Fry for this rambling but heartfelt review.