Archive Fever

A love letter to the post real-time web

By Matthew Ogle  •  16 December 2010
The archivist produces more archive, and that is why the archive is never closed. It opens out of the future. Jacques Derrida, Archive Fever, 1994

The Now

“When was the last day you can remember not checking Twitter?”

A friend asked me this earlier this year, and like many of us I couldn’t quite recall one. The innocent question ended up prompting my first “no devices” holiday in at least five years.

What surprised me most was how long it took to adjust to life without the warm twitchy blanket of what’s now called “the real-time web”1 — the portable tribes of Twitter, Facebook, Tumblr, and the countless other services always on in our pockets and minds.

Upon reentry to London, those same services were also the hardest to dive back into. While things like email and maps simply jumped back onto screens, instantly natural, returning to the real-time web felt weightier. It required a conscious, single-minded commitment to The Now, which, let’s face it, is Where The Action Is.

Within a few days, though, I was back with my friendly communal hivemind and glad of it. But I’d gained some perspective on how narrow the real-time channels were. Their steely focus on The Now neglected all sorts of other facets of human existence, communication, happiness. I started to wonder if our online experiences had to be this way. I also started to wonder: how did we get here?

The Action

I don’t think it had much, if anything, to do with declining attention spans or other purported societal ills. Instead, I believe the last four years of progress toward the real-timiness we now enjoy was largely driven by the fact that making it required solving lots of difficult (read: exciting) engineering problems. Online makers of things go where the excitement is.

I was partly responsible for this myself at Last.fm; designing and maintaining systems that could handle up to 800 scrobbles a second presented endless challenges and distractions. Any serious product or information architecture considerations — so, what do we do with all these scrobbles? 2 — almost always came second by sheer necessity.

This pattern repeated itself all over the web, perhaps most famously by Twitter, whose early technical failings and ambiguous product direction are legendary (and now largely irrelevant). A profusion of new ways of both storing the data real-time services demanded (like NoSQL) and doing clever things with it (Hadoop, Hive, et al.) sprung into being. Similar innovation occurred on the front end too.

But now I feel a growing sense — both from a technology and a UX standpoint — that displaying a reverse-chronological list of “stuff happening now” is not enough. It’s no longer where the action is. 3

If real-time’s gotten easier and excitement’s dwindling, where’s the next frontier? I think incredible opportunity is hiding in plain sight.

The Memory

In response to the growing popularity of email and the internet in the early 90s, philosopher Jacques Derrida made a useful observation on technology’s relationship to human memory and conceptions of “the archive:”

The technical structure of the archiving archive also determines the structure of the archivable content…archivization produces as much as it records the event.

By providing us with new ways to share what we’re doing right now, the real-time web also captures something we might not have created otherwise: a permanent record of the event. We’ve all been so distracted by The Now that we’ve hardly noticed the beautiful comet tails of personal history trailing in our wake. We’ve all become accidental archivists; our burgeoning digital archives open out of the future.

What were you thinking about on November 23rd, 2009? You probably have no idea, but Twitter might. What was your personal soundtrack to the summer of ’07? Ask Last.fm. Hit up Dopplr to find out how many miles you travelled last year, Foursquare for the Berlin bar that people you know check in to more than any other, or Facebook to see the photos of the last time you hung out with your best friend on the other side of the world.

Without deliberate planning, we have created amazing new tools for remembering. The real-time web might just be the most elaborate and widely-adopted architecture for self-archival ever created.

The Challenge

Except we can’t quite, you know, do much with it yet.

Nearly every “memory retrieval” scenario above (Dopplr excepted) is difficult or impossible to accomplish today without external tools or programming experience. (Or a willingness to hit ‘next’ for three hours.)

To pick on poor Twitter again, using their new site to find something you said in 2009 is an exercise in frustration. Or, in a more realistic use case, it’s similarly hard to find that link you tweeted last week. Or a reply someone wrote last month that’s suddenly useful or delightful. Or even have a nice way to read highlights you missed during a busy day at work.

The problem is ultimately one of attitude. The current philosophy underlying most of the real-time web is that if it’s not recent, it’s not important. This is what we need to change.

Happily, there are many glimmers of light. The past couple years have seen an increasingly beautiful array of mashups, art experiments, utilities, and even a few ahead-of-their-time startups explore the latent potential in personal archives and alternatives to real-time and the relentless reverse-chronological list.

Even Facebook themselves are having a go; the raw nerve their “Photo Memories” feature struck with some users proves there’s plenty of emotion to be tapped in the archive, albeit carefully. The emotional appeal of the past merging into the present is also what drives this recent video promoting Path4.

These experiments are vital, and they suggest that letting us discover and interact with both the near-present and the deeper corners of our own personal archives need not be something relegated to the margins of the early-adopter web. It’s time to integrate archive mentality directly into existing products and services, and in some cases create entire new ones.

Beyond the attitude change that’s already underway, I don’t know everything we’ll need to make this happen. But a few additional requirements I’ve been thinking about include:

The Fever

As people, we are made up of the sum total of all the experiences in our lives and in the lives of those we love. Being able to track what’s happening right now — amplified and revitalized by the real-time web — is important and will undoubtedly remain so. But by recording what’s happening now, we’ve also created rich but neglected personal archives whose potential we’re only beginning to explore.

I believe we, as makers of online services, have an incredible opportunity to ground the things we create in both the present and the past, making them — and thus ourselves — richer, more beautiful, and more human.

But first we need to catch archive fever. 5

  1. Hopefully the use of “web” in “real-time web” isn’t too distracting; these days most services of course live happily inside apps and other places outside the browser. (And it took around four days, for the record.)
  2. We usually ended up doing the same thing our peers at other companies did: throwing them on a page in a reverse chronological list. Hey, job done! Next we’ll add some slick JavaScript! (While this was of its time and not without value, I now realize it was also a cop out.)
  3. This maturation is not without benefits. We’ve gained solid UI patterns which can be reused in interesting ways (as my /dev/fort 5 cohort recently tried to do with Spacelog), and have created a host of new technologies for powering these experiences (in fact, some might say we’ve created too many).
  4. Love the prominent date labels, totally spooked by the too-beautiful actors.
  5. Midway through the work that inspired the title of this piece, Derrida goes long in an attempt to further define “archive fever”:
    We are en mal d’archive: in need of archives. Listening to the French idiom…to be en mal d’archive can mean something else than to suffer from a sickness, from a trouble or from what the noun mal might name. It is to burn with a passion. It is never to rest, interminably, from searching for the archive right where it slips away. It is to run after the archive, even if there’s too much of it… it is to have a compulsive, repetitive, and nostalgic desire for the archive, an irrepressible desire to return to the origin, a homesickness, a nostalgia for the return to the most archaic place of absolute commencement.
    Edited 16 Dec 2010: As Alex Mahan rightly notes, Derrida also links the archivist's compulsion to the Freudian death drive, crediting archive fever not just with the preservation of memory but also with its simultaneous destruction. (Typical!) Our hope is that we can invoke the positive aspects of the archival drive Derrida captures so poetically.