TechDays 2010: Real-World Patterns for Cloud Computing

Tech Days 2010If it’s Autumn, it must be time for TechDays again here in Canada. This year  I was Microsoft Canada’s Content Lead for the “Real-World Patterns for Cloud Computing” session.

The Toronto flavour of the event has been taking place yesterday and today.

Compute, Store, Scale

The Cloud Computing presentation focused on scenarios involving where we use the cloud for compute, storage, and scaling (of both compute power and storage). It looked at a real-world case study using Windows Azure Worker Roles, and multiple Azure Storage account (leveraging Blobs and Queues – Azure Table Storage and SQL Azure are other stories for other presentations).

Demo Path

"Real World Patterns for Cloud Computing" Demo Path (click to enlarge)
"Real World Patterns for Cloud Computing" Demo Path (click to embiggen)

Here’s the basic demo path (as pictured above):

1. Efficiently upload content to the cloud

2. After upload, add message(s) to Azure Queue indicating to a Worker Role there’s content to process

3. Use a Worker Role to read messages from the Queue and process the uploaded content

4. Show how to scale the whole thing – both storage (if it exceeds 100TB, the limit of an Azure Storage Account), and compute (if there is too much content to process by a single Worker Role).

The focus of the talk was on incorporating good practices (and avoiding gotchas) throughout this process. We started with some simple code that uploaded content to the cloud and processed it using a worker role, and then modified the solution to incorporate a number of improvements. Then we slammed the whole thing with a heavy load and used the Windows Azure Dynamic Scaling Sample to monitor the growth of the queue and scale up and down accordingly.

(This last part addresses what is certainly one of the most frequently asked questions I get about Azure – does it scale its computing automatically, and if not, how do I do so? The Dynamic Scaling Sample provides one very configurable and suitable solution for automated scaling of Azure computing.)

If you were at TechDays and are interested in the sample code, please contact me and I’ll send a link. You may also be interested in Microsoft Evangelist Wade Wegner’s presentation from TechEd2010, which he delivered with Jerry Schulist from the Tribune Company. I’m indebted to Wade and Jerry for their excellent presentation and their pioneering work with Azure.

Today: More TechDays + streaming PDC 2010

If you’re interested in Azure and cloud computing, you almost certainly will be interested in following what’s going on this week in Redmond at Microsoft PDC 2010. The entire conference will be streamed online – follow that link for a slick schedule and some nice .ics links so you can add talks of interest to your calendar.

[Update, 29 Oct 2010: There were indeed quite a few announcements related to Azure at PDC2010 – check them out!]

Thanks Again

Thanks to everyone who came out – I really enjoyed the day and that was probably the best post-presentation Q&A session I’ve ever had, even if Joey had to “Kanye” me off stage again when question time ran out! My compliments to the MS Canada team – they have the choreography of these large-scale tech events down to an art.

Next stop: Ottawa!

I’m looking forward to presenting this talk again in Ottawa on November 9th for the TechDays Ottawa event.

Legend of the Greasepole gets the Silverlight 4 + Analytics treatment

Legend of the Greasepole Title Screen

Legend of the Greasepole has been ported to Silverlight 4 and reincarnated on

Greasepole is the long-suffering game about multimedia tribute to the inexplicable Engineering traditions at Queen’s University in Canada. Over 50 students contributed to the project back in the day.

There’s a significant AI component to Greasepole – the autonomous “frosh” characters have models for learning and communicating with one another.

A couple of years ago I ported it from C++ to C# and XNA. I abstracted out a series of services (graphics, sound, input, timer, persistence) so that it might ultimately be ported again to a platform like, say, Silverlight or something. Why? I don’t know, maybe I’m a little obsessed with the illusion of preservation.

The Silverlight 2 version was a bit shaky. Silverlight 4’s hardware acceleration and bitmap caching make it pretty solid. It is also awesome to hear from friends that it apparently works on the Mac.

Analyze These… Shenanigans

I also added a little analytics. Although it should probably be said that the Greasepole event largely defies analysis, the game itself does not, and so this is the first time I can let someone poke their head in and see how the froshies are doing all around the world.

Back in the day, the worldwide best time was in excess of a mere 53 minutes. But I had to learn that by way of Sean Murray (class of ’05; wonder where he is now) sending me a screenshot. Now the interwebs will tell us immediately. (Admittedly, it’s not a fair fight against Sean, because the frosh are now permanently in “keener” mode, and the Options screen has been replaced by a dozen trendy Achievements for you to “unlock”).

So get going stalling those frosh, and my question for you is – what statistics would you like to see?

“Number of pints Al ‘Pop Boy’ Burchell has quaffed?”

“Number of hippos fed”?

“Height of human pyramid vs time”?

I am going to enjoy cooking up visualizations for some of those.

(Coding notes: A few new VS2010 things helped with this update: Web.config transformation (rocks), improvements to Web Publish functionality, XAML designer, Entity Framework experience… and more.)

Play Legend of the Greasepole Online Edition.

Cross-Country Olympic Mind Games

During the Winter Olympics, the lights of the CN Tower are taking part in an interactive art installation that’s controlled by a participant’s brainwaves.

What I like about the CN Tower installation (in addition to the cross-country nature of the biofeedback loop) is how they teach participants to generate the correct brainwave patterns. We faced a similar challenge back at MediaLabEurope.

CN Tower Light Show
CN Tower Light Show

During the 2010 Winter Olympics, the lights of the CN Tower are taking part in an interactive biometric art installation.

From 3,300km away, a participant in Vancouver has their brainwaves monitored as they watch a live video of the Toronto skyline. When they are inattentive, the CN Tower’s lights move at a lethargic pace. But as the participant concentrates, the lights of the CN Tower cycle around with the speed of Christine Nesbitt ’round the speed skating track.

The measurement being used to determine a participant’s attentiveness is their ratio of their alpha waves to beta waves. This very rough estimate of attentiveness is also what we used for an early project in the MindGames group called “BrainChild”, where a participant unlocked a door through sustained concentration.

What I like about the CN Tower installation (in addition to the cross-country nature of the biofeedback loop) was something I read in this Torontoist article. If participants are having trouble “concentrating,” they are encouraged to count the lights in skyscraper windows.

This is a very clever, indirect way to achieve the desired result. When working with “untrained” biofeedback participants, it can be difficult to articulate the subtle changes that they need to make which will lead to a desired signal (in this case, a change in the ratio of alpha to beta waves).

Mind Balance Training

Mind Balance Screenshot
Mind Balance Screenshot

A project we built at MIT MediaLabEurope called Mind Balance faced a similar challenge.

Mind Balance also relied on an electroencephalogram (EEG) metric to create a control mechanism. However, instead of measuring the ratio between alpha and beta waves, we were monitoring the occipital lobes at the back of the head to detect artifacts from the electrical signals produced by the brain’s visual processing.

The subtlety there was that success for a participant required not just having a visual pattern in your field of view, but also attending to that pattern – a “squishy” concept that required training. Some of the “indirect” methods we used to explain a “good stare” included encouraging a participant to “stare right through” a region on the screen.

We used a 45-second acoustic feedback training session to help participants learn this concept assisted by very clear and immediate feedback. It worked with varying degrees of success, but thankfully, enough success in over 95% of cases for a participant to successfully go on to generate a single reliable control axis.

We resurrected Mind Balance for the Microsoft Ireland Visual Studio 2005 launch and it was awesome (and somewhat improbable) that we got it to work in an Irish pub setting. You can read more about Mind Balance here, and check out the shots of our prototype headgear called Cerebus.

(Another MindGames project where indirectly achieving biometric results played a big role was Relax to Win. The techniques players used to achieve relaxation under pressure was an important and fun part of the interaction.)

Jedi Mind Tricks Re-Re-Rediscovered

Ed Lalor wears Cerebus
Mindgames researcher Ed Lalor models Cerebus

Excitement around Brain-Computer Interfaces for commercial and artistic use seems to resurface every few years.

Although the headgear for the Olympic installation looks considerably refined compared to our prototype (pictured here), achieving reliability for novel and useful BCI metrics remains a really tough problem, principally because most people aren’t cool with having someone drill into their head, which is a shame, because the brain’s electrical signals are so much weaker by the time they reach the surface.

Later this year, with the imminent launch of Project Natal, the focus now seems to be on visual and acoustic interfaces.

But during the Olympics, it’s all about the Mind Games. My hat’s off to Interaxon for putting together this engaging cross-country installation.

(More about the Olympic mind-controlled art installation at PopSci, Interaxon, Torontoist)

(More about Mind Balance here with links to pics)

Standards are the Sands of Time (or, the Illusion of Preservation)

On a long enough timeline, the survival rate for every standard drops to zero.

From emulated games to digital negatives, how can we preserve our digital content for posterity, and inform our decisions about which formats we adopt?

A special jpg: 2 monkeys, city, moonlight
A special jpg: 2 monkeys, city, moonlight

How many jpg images do you suppose are living on the internet? Probably quite a few.

Among all those images, the jpg on the right is particularly special to me, because it’s a screen capture of an adventure game I wrote for the Commodore VIC-20 when I was very very young (like almost single digits).

If you can’t make it out, those are two monkeys standing under the Toronto skyline, reading a newspaper by moonlight.

At the time I wrote the game, it could only be played by reading the bits that were encoded as waveforms on an audio cassette tape. Which was awesome.

But one day my VIC-20’s cassette drive silently stopped reading tapes, and the game could be played no more.

The photograph below (another jpg) was taken of the defunct
VIC-20 hardware some time before it was tossed into a big dumpster.

Commodore VIC-20 and Tape (deceased)
Commodore VIC-20 (deceased) and Cassette

Resuscitation by Emulator

VICE Vic-20 Emulator on Windows
2001 Emulator (not dead yet)

Fast-forward through more than a decade to 2001, where I find VICE, a program that can emulate the VIC-20 on Windows.

With the game’s cassette reclaimed from the basement and a dusty tape player connected to my PC’s audio port, I transferred the audio that made up the game’s bits into an audio .wav file, and then converted that audio file into a format that could be loaded by the emulator.

I was so happy to be able to run my game again, and have it preserved “forever,” even though its contents are meaningful only to me (and maybe my family).

Satisfied, I zipped it up together with the emulator
and put it into my folder called “C:\pastlives” for safe keeping.

Now almost ten more years have passed. In this 64-bit world, my game still runs.

For now.

(slightly) Longer Term Thinking

This ran through my head when I was considering whether I should adopt Adobe’s Digital Negative (DNG) format as a standard for my photo archives. The alternative is the Nikon raw (NEF) format I currently use for my negatives.

It literally kept me up last night thinking about the layers of technology I depend on to retrieve and view things in “C:\pastlives”, like my VIC-20 emulator or my digital photos.

Check out this stack of just some of the standards and technologies I used to play my game today:

Some of the technology standards used to play my VIC-20 game on Windows 7
Some of the technology standards and file types used to play my VIC-20 game on Windows 7. (another jpg)

(Red == the ones I perceive as more volatile)
Sure, you could substitute for some of these (the VIC-20 emulator, for example, also runs on Linux).

But how likely is it that a complete cocktail of prerequisite technologies will be around long enough for me to load and play the game in three more decades? In ten?

These standards aren’t stacking up like a solid foundation, they’re piling up like sediment!

(To abuse a pretentious analogy) It’s like the sands of time themselves.

Consider the Floppy

5 1/4 inch Floppy Disk
5 1/4 inch Floppy Disk

Remember floppy disks? Those once-ubiquitous storage devices? How many of you still have machines around that can read a 5 ¼ inch floppy?

On a long enough timeline, the survival rate for every technology standard drops to zero.

By my estimation, all it’ll take is for me to neglect its preservation for approximately a decade, and my game shall be played no more.

Spoiler Alert!

I’ll go out on a limb and say that despite my efforts, my VIC-20 game will never outlive even all the jpgs in the internet.

So I’m just going to tell you that one monkey rescues the other monkey from the Toronto zoo and they escape back to Africa.

There’s an action sequence where you have to climb through the sunroof of a taxi and jump off at just the right moment to escape a high-speed police chase.

Screenshots, while they last, are available upon request.