Archiving the Ephemeral:

Deja News and the Ethics of Perfect Memory

1They may have been obscured by the popular media's fascination with the World Wide Web, but for many Net users, Usenet newsgroups still constitute an equally important interactive tool. While Web pages present relatively static information that can be structured through hypertext links and searched using Yahoo! and similar services, newsgroups are fora for open, interactive discussion on virtually any conceivable subject, amongst participants from around the world -- more than any other part of the Net, they are instrumental in the formation of virtual communities by allowing like-minded individuals to get together, exchange their opinions, and organise themselves.

2Having emerged from email mailing-lists, newsgroups are among the oldest uses of computer networks for communication; based around a simple exchange of all new postings among the participants, they offer few of the comforts of more modern technologies, and so it is not surprising that a number of services have begun to provide powerful Web interfaces for newsgroup access. One of the oldest and most reknowned amongst these sites is Deja News. Since its launch in May 1995, Deja News has expanded to cover around 50,000 different newsgroups, which are contained in a memory archive that is several hundred gigabytes in size, according to some reports (Woods, n.pag.); the company itself boasts accesses by more than three million users each month to the hundreds of millions of articles it has archived ("Company Background", n. pag.).

3What makes Deja News attractive to so many users are the many search options the service has added to its newsgroups interface. Deja News visitors can search for any article subjects, keywords, newsgroups, or participants' names they are interested in: if you're looking for the absolutely latest on the White House sex scandal, search for "Clinton + Lewinski" and limit your search to articles posted in the last few days or hours; if you want to know what newsgroups your co-worker is writing to in her lunch break, simply ask Deja News to find all postings by "Annabel Smith". If you can't quite remember the address for the latest Internet camera site, Deja News will.

4 To put such powerful research capabilities at the fingertips of any Web user raises any number of legal as well as ethical questions, however. To begin with, does Deja News even have the right to archive anyone's and everyone's postings to newsgroups? The simple fact that users' articles are posted openly, for all newsgroup participants to see, does not necessarily automatically imply that the article may be made available to a greater public elsewhere -- by analogy, if you have a casual conversation with a group of people, you aren't usually expecting to see your words in the history books the next day (but analogies between the Internet and 'real life' are always dangerous). Unless you spoke to someone with a photographic memory, you can usually rely on them gradually forgetting what you said, even if you made a fool out of yourself -- that's human.

5 Appearing as the result of a Deja News search, articles lose their original context -- the newsgroup discussion they were part of -- and are given an entirely different one, a context in which by virtue of being so presented they may gain new and potentially questionable authority. As is so often the case for information on Websites, the information in articles which can thus be found through services like Deja News cannot easily be verified or disproven; due to the loss of context, researchers cannot even gain a feel for a writer's trustworthiness in the same way that seasoned newsgroup members can. Neither may they always detect intentional irony and humour: playful exaggeration may easily appear as deliberate misinformation, friendly oneupmanship as an angry attack.

6 The results of author-based searches may be even more potentially damaging, however. Deja News offers various mechanisms that can include searches restricted to the articles written by a particular user, culminating in the 'author profile' that can be used to list all posts ever made by a particular user. One does not need to be paranoid to imagine ways in which such a powerful research tool may be abused -- perhaps even (and most easily) by the Deja News company itself: since, following general Web etiquette, access to the service is free for normal users, Deja News relies on other avenues of income, and doubtlessly it would be tempting to sell the rights to exploit the Deja News database to professional spammers (Internet junk mailers). These could then aim their advertising emails directly towards the most promising target audience -- those who have in their newsgroups postings shown the most interest in a particular product or service. Indeed, Deja News notes, somewhat vaguely, that it "can provide efficient and effective Internet-based marketing for various types of online marketing goals, such as testing messages, building brand awareness, increasing Website traffic or generating leads" ("Company Background", n. pag.).

7While such uninvited advertising may be annoying to unsuspecting Internet users, more damaging and mean-spirited uses of the 'author profile' can also be imagined easily. What if your prospective new employer finds the comments you made about the company in a newsgroup article last year? What if they check your author profile for your attitude to drugs (did you ever write an article in rec.drugs.cannabis) or your sexual orientation (any posts to What if some extremist group targets you over your support for multiculturalism? Thanks to Deja News and similar services, anything you've ever said may be used against you. The virtual walls of cyberspace have ears, come with a perfect memory, and are prepared to share their knowledge with anyone.

8A valid line of argument against such criticism notes that we are all responsible for our own actions: newsgroups are, after all, public discussion fora that are open to all participants -- if you make a controversial statement in such a place, you must be prepared to suffer the consequences. However, the threat of being taken out of context must once more be emphasised at this point: while the articles that can be found through Deja News appear to accurately reflect their writers' views, the background against which such views were expressed is much more difficult to extract. Furthermore, only very few newsgroup participants will be aware that their postings are continually being archived, since newsgroups generally appear as a fairly ephemeral medium: only the last few days or, at most, weeks of newsgroup traffic are usually stored on news servers. An awareness of being archived would help writers protect themselves -- it may also serve to impoverish newsgroup discussion, however.

9Even more importantly, the already-digital, computer-mediated nature of newsgroup discussions has far-reaching implications. Dealing with units of data that come in a handy, easily stored format, services like Deja News tend to archive Usenet newsgroups interminably -- your first flames as a 'clueless newbie', years ago, may therefore today still be used to embarass you. This is the most important new development: analogue, organic, human memory eventually fades; we tend to organise our memories, and remember those things we regard as most important, while others gradually vanish. Many modern legal systems reflect this process by gradually deleting minor and even major offences from a citizen's criminal records -- they forgive as they forget. Other than in cases of extreme Alzheimer's brought about by server crashes and hard disk defects, digital memory, on the other hand, is perfect for unlimited periods of time: once entered into a database, newsgroup articles may be stored forever; ephemeral discussions become matters of permanent record. The computer doesn't forget.

10While its many Internet accolades bear witness to the benefits users have found in Deja News, the ethical questions the service raises have hardly been addressed so far, least of all on the Deja News Website itself. While apparently the inclusion of a header line "X-noarchive: yes" may prevent articles from being archived by Deja News, this isn't advertised anywhere; neither are there any statements justifying the unauthorised archiving of newsgroups, or any easily accessible mechanisms for users to have their own articles deleted from the archives. As has often been the case on the Internet, a private organisation has therefore become a semi-official institution, simply by virtue of having thought of an idea first; ethics and laws are left behind by technological development, and find that they have some catching-up to do.

11Of course, none of this should be seen as condemning Deja News as a malevolent organisation out to spy on Internet participants -- in fact, the company so far appears to have shown admirable restraint in declining to exploit its database. Deja News is at the centre of this controversy only by virtue of having implemented the idea of a 'memory of Usenet' too perfectly: not Deja News is the problem, but those who would use, to their own and possibly sinister ends, information made available without charge by Deja News. Eventually, in any way, Deja News itself may be overwhelmed by its own creation: with the amount of Internet users still continually increasing, and with newsgroup articles accumulating, it is gradually getting harder to still find the few most important postings amongst a multitude of discussions (in a similar way, search engine users are beginning to have trouble locating the most relevant Websites on any specific topic amongst a large number of less useful 'vanity' homepages).

12In the end, then, this new 'perfect' digital memory may have to learn an important capability from its analogue human counterpart: Deja News and similar archives may have to learn how to forget articles of lesser significance. A very simple first step towards this process has already been made: since December 1997, junk mail postings ('spam') are being removed from the Deja News archives (Woods, n. pag.). While such articles, whose uselessness is almost universally agreed upon by the Internet community at large, constitute a clear case for removal, however, any further deletions will mean a significant step away from the original Deja News goal of providing a complete archive of Usenet newsgroups, and towards increasingly controversial value-judgments -- who, after all, is to decide which postings are worth archiving, and which are irrelevant? If memory is to be selective, who will do the selecting?

13Eventually (and even if new memory management technologies help prevent outright deletion by relegating less important information to some sort of second-rate, less accessed memory space), it seems, the problem returns to being an ethical one -- of what is archived where and for how long, of who has access to these data, and of how newsgroup writers can regain control of their articles to protect themselves and prevent abuse. Deja News and the Internet community as a whole would be well-advised to address the problems raised by this perfect memory of originally ephemeral conversations before any major and damaging abuse can occur.