Standards are the Sands of Time (or, the Illusion of Preservation)

On a long enough timeline, the survival rate for every standard drops to zero.

From emulated games to digital negatives, how can we preserve our digital content for posterity, and inform our decisions about which formats we adopt?

A special jpg: 2 monkeys, city, moonlight
A special jpg: 2 monkeys, city, moonlight

How many jpg images do you suppose are living on the internet? Probably quite a few.

Among all those images, the jpg on the right is particularly special to me, because it’s a screen capture of an adventure game I wrote for the Commodore VIC-20 when I was very very young (like almost single digits).

If you can’t make it out, those are two monkeys standing under the Toronto skyline, reading a newspaper by moonlight.

At the time I wrote the game, it could only be played by reading the bits that were encoded as waveforms on an audio cassette tape. Which was awesome.

But one day my VIC-20’s cassette drive silently stopped reading tapes, and the game could be played no more.

The photograph below (another jpg) was taken of the defunct
VIC-20 hardware some time before it was tossed into a big dumpster.

Commodore VIC-20 and Tape (deceased)
Commodore VIC-20 (deceased) and Cassette

Resuscitation by Emulator

VICE Vic-20 Emulator on Windows
2001 Emulator (not dead yet)

Fast-forward through more than a decade to 2001, where I find VICE, a program that can emulate the VIC-20 on Windows.

With the game’s cassette reclaimed from the basement and a dusty tape player connected to my PC’s audio port, I transferred the audio that made up the game’s bits into an audio .wav file, and then converted that audio file into a format that could be loaded by the emulator.

I was so happy to be able to run my game again, and have it preserved “forever,” even though its contents are meaningful only to me (and maybe my family).

Satisfied, I zipped it up together with the emulator
and put it into my folder called “C:\pastlives” for safe keeping.

Now almost ten more years have passed. In this 64-bit world, my game still runs.

For now.

(slightly) Longer Term Thinking

This ran through my head when I was considering whether I should adopt Adobe’s Digital Negative (DNG) format as a standard for my photo archives. The alternative is the Nikon raw (NEF) format I currently use for my negatives.

It literally kept me up last night thinking about the layers of technology I depend on to retrieve and view things in “C:\pastlives”, like my VIC-20 emulator or my digital photos.

Check out this stack of just some of the standards and technologies I used to play my game today:

Some of the technology standards used to play my VIC-20 game on Windows 7
Some of the technology standards and file types used to play my VIC-20 game on Windows 7. (another jpg)

(Red == the ones I perceive as more volatile)
Sure, you could substitute for some of these (the VIC-20 emulator, for example, also runs on Linux).

But how likely is it that a complete cocktail of prerequisite technologies will be around long enough for me to load and play the game in three more decades? In ten?

These standards aren’t stacking up like a solid foundation, they’re piling up like sediment!

(To abuse a pretentious analogy) It’s like the sands of time themselves.

Consider the Floppy

5 1/4 inch Floppy Disk
5 1/4 inch Floppy Disk

Remember floppy disks? Those once-ubiquitous storage devices? How many of you still have machines around that can read a 5 ¼ inch floppy?

On a long enough timeline, the survival rate for every technology standard drops to zero.

By my estimation, all it’ll take is for me to neglect its preservation for approximately a decade, and my game shall be played no more.

Spoiler Alert!

I’ll go out on a limb and say that despite my efforts, my VIC-20 game will never outlive even all the jpgs in the internet.

So I’m just going to tell you that one monkey rescues the other monkey from the Toronto zoo and they escape back to Africa.

There’s an action sequence where you have to climb through the sunroof of a taxi and jump off at just the right moment to escape a high-speed police chase.

Screenshots, while they last, are available upon request.

Nikon Raw (NEF) Codec: fast, free new option

The Fast Picture Viewer codec is free, works on 32- and 64-bit Windows, and has since become my go-to codec for raw (NEF) images on both Windows 7 and Windows Vista.

Nikon Logo

Earlier this year, I wrote about the where, why and how of Nikon Raw (NEF) Codecs for Vista and Windows 7.

In the intervening months, my fellow photographers have introduced me to a shiny new alternative called the Fast Picture Viewer codec, which is free, works on 32- and 64-bit Windows, and has since become my go-to codec on both Windows 7 and Windows Vista.

Fast Picture ViewerWith this codec installed, you get Raw image support in places like Windows File Explorer, Windows Live Photo Gallery, and Windows 7 Media Center. And it’s surprisingly fast.

Here’s a link back to my updated earlier article, which explains the codec story in more detail.

Or you can just go download the Fast Picture Viewer codec.

Microsoft tech Bonus: It was created by a BizSpark startup.

Viewing a directory of Raw (NEF) Files from 64-bit Windows 7
The goal: a directory of Raw (NEF) Image files and their metadata in Windows Explorer. This is 64-bit Windows 7.

Nikon Raw (NEF) Codecs for Vista and Windows 7

Nikon Logo

Two years ago, I mentioned in an article that Nikon’s Raw (NEF) Codec was an important part of my photo-processing pipeline.  After many laptops, photos, and software updates, it still is.

I now work with photos on 32- and 64-bit machines running Windows 7 and Windows Vista.  In the hope it will help other Nikon photographers, here is an update describing some of my more recent experiences working with Raw (NEF) files under Windows.

[Note: I last updated this article May 2012.  tl;dr: download Nikon’s 32- and 64-bit NEF codecs here.]

First – Why would you want a NEF Codec?

The principal reason you’d want a Raw (NEF) Codec is to view Raw Nikon images and metadata from within Windows File Explorer and Windows Photo Gallery, and now Windows Live Photo Gallery as well, which was released as part of Windows Live Essentials.

Viewing a directory of Raw (NEF) Files from 64-bit Windows 7
The goal: a directory of Raw (NEF) Image files and their metadata in Windows Explorer. This is 64-bit Windows 7.

Of course, if you’re shooting Raw images, you’ll probably also want a fully-featured application that can view and edit NEFs (such as Nikon’s Capture NX 2, or Adobe’s Lightroom or Photoshop), in addition to the Codec. But the Codec itself is very useful for viewing, sorting and “triaging” your photos.

The options available to you for NEF Codec solutions will depend on whether you’re running 32- or 64-bit Windows.  Here are the ones I use today.

Nikon LogoRecommended NEF Codec for both 32- and 64-bit Windows:
Nikon’s NEF Codec

(current version: 1.14.0)  (click for download info)

Nikon’s Codec has gone through a number of revisions.  It is currently at version 1.14, and can be downloaded from here.  This codec is free to download. I should note that I couldn’t get it to work on a pre-release version of Windows 8.

Fast Picture Viewer

Alternate option: FastPictureViewer Codec (NOT FREE)

(current version: 3.2) (click for download info)

My impression (not even remotely scientific, since I’ve switched machines and cameras) is that version 1.8 is quite a bit quicker and more robust (stalls less often) than earlier versions.  My improved experience may also have to do with updates to Windows Vista, so I’m not sure.

Nikon Logo

64-bit alternate option: Ardfry’s x64 NEF Codec

(current version: 1.0.0.12) (click for download info)

In addition to the Fast Picture Viewer, another third party, Ardfry Imaging, have released a 64-bit NEF codec for Windows Vista x64.  I have previously worked with it on multiple 64-bit Windows 7 installations, and it worked well for me.  I evaluated the Ardfry Codec beta for its trial period, and decided it was well worth the $19USD they were asking for a registered copy.

But what if I don’t want to pay for (or install) a Codec?

Please note that even if you choose not to install a codec like FPV or Ardfry’s, Nikon’s Capture NX 2.1 works fine under 64-bit Windows (in 32-bit emulation mode), as you can see in the screenshot of 64-bit Windows Task Manager below.

Capture NX 2.1 works fine under 64-bit Windows 7 in 32-bit emulation mode
Capture NX 2.1 works fine under 64-bit Windows 7 in 32-bit emulation mode

That being said, I value being able to see my photos and their metadata within Windows Explorer and Windows Live Photo Gallery, which is not possible without the codec.  So a codec solution like the ones listed above makes sense for me. The Windows 7 installation pictured at the top of this article is sporting the Ardfry codec.

I hope this helps, and am always grateful for tips, advice, and further thoughts on streamlining my photography pipeline.

Nikon’s Raw (NEF) Vista Codec Updated

Nikon Logo

Nikon has released an updated version of their Raw (NEF) Codec for Vista that resolves an issue that caused it to suddenly stop working last week. Apparently the problem had something to do with an expired certificate.

The new codec is still labeled version 1.01. However, if you uninstall the old 1.00 or 1.01 codec, and install this new one, you’ll be back in business.

I mentioned back in January that I have stopped shooting JPG images, in favour of only shooting RAW (NEF) images. I am still using the following photography pipeline:

  • View, sort, and triage NEF images: from Vista’s File Explorer and Windows Photo Gallery.
  • Load into Nikon Capture NX: for post-processing. This program has earned my respect with its ability to store my edit history in the RAW file, and also one ingenious feature (yes, I think ingenious is exactly the word for it) called Color Control Points. However, I have nothing positive to say about Capture NX’s user interface: it desperately needs keyboard shortcuts, and lacks basic navigational functionality like mouse-wheel zoom. I found a great resource for Nikon Capture NX tips and tricks here at Nikonians.org.
  • Batch export to JPG: using Nikon Capture NX’s batch processing
  • Upload: to Flickr.

There are two challenges with this pipeline:

  • First, the Vista codec still isn’t particularly quick, even on a relatively beefy laptop (Vista Overall Experience Index: 3.0; Processor: 4.6, Memory: 4.7, Graphics: 3.6). You can flip through photos quickly enough, but if you want to delete a photo, Vista spins its wheels while the codec renders the high-res image. To work around this, I use the keyboard shortcuts (1, 2, 3, 4, 5) to quickly “rate” all my photos, and then delete all the bad ones in one go.
  • Second, although you can associate metadata (“tags”) with NEFs in Photo Gallery, using the above-mentioned codec, the tags don’t show up anywhere I can find them in Capture NX. Therefore I am not tagging my images on Vista (defeating a lot of its organizational potential) and instead am just tagging the JPGs on Flickr, which, by that point, have become disassociated from the original NEF images. Some day, if I want to sync them, it may be an intractable task (or at least an image processing challenge).

So my Vista-Flickr NEF pipeline experience is good but not great. I am still going to keep shooting RAW (NEF) only, as I am learning how to make subtle and powerful changes to my photos using Capture NX, which I find very valuable. After all, if it’s worth shooting, it’s worth trying to shoot it right!

[February 2009 Update]

I posted some updated thoughts on NEF Codecs for Vista and Windows 7 for 32- and 64-bit installations. Thank you for all the thoughts and discussion here and I hope this continues to be helpful.