The Affair of the Vanishing Content
"Digitized information, especially on the Internet, has such
rapid turnover these days that total loss is the norm.
Civilization is developing severe amnesia as a result; indeed
it may have become too amnesiac already to notice the problem
properly."
(Stewart Brand, President, The Long Now Foundation )
Thousands of articles and essays posted by hundreds of authors
were lost forever when themestream.com surprisingly shut its
virtual gates. A sizable portion of the 1960 census, recorded
on UNIVAC II-A tapes, is now inaccessible. Web hosts crash
daily, erasing in the process valuable content. Access to web
sites is often suspended - or blocked altogether - because of
a real (or imagined) violation by the webmaster of the host's
Terms of Service (TOS). Millions of other web sites - the
results of collective, multi-annual, transcontinental efforts
- contain unique stores of information in the form of
databases, articles, discussion threads, and links to other
web sites. Consider "Central Europe Review". Its archives
comprise more than 2500 articles and essays about every
conceivable aspect of Central and Eastern Europe and the
Balkan. It is one of countless such collections.
Similar and much larger treasures have perished since the dawn
of the digital age in the 1920's. Very few early radio and TV
programs have survived, for instance. The current "digital
dark age" can be compared only to the one which followed the
torching of the Library of Alexandria. The more accessible and
abundant the information available to us - the more devalued
and common it becomes and the less institutional and cultural
memory we seem to possess. In the battle between paper and
screen, the former has won formidably. Newspaper archives,
dating back to the 1700's are now being digitized - testifying
to the endurance, resilience, and longevity of paper.
Enter the "Internet Libraries", or Digital Archival
Repositories (DAR). These are libraries that provide free
access to digital materials replicated across multiple
servers ("safety in redundancy"). They contain Web pages,
television programming, films, e-books, archives of discussion
lists, etc. Such materials can help linguists trace the
development of language, journalists conduct research,
scholars compare notes, students learn, and teachers teach.
The Internet's evolution mirrors closely the social and
cultural history of North America at the end of the 20th
century. If not preserved, our understanding of who we are and
where we are going will be severely hampered. The clues to our
future lie ensconced in our past. It is the only guarantee
against repeating the mistakes of our predecessors. Long gone
Web pages cached by the likes of Google and Alexa constitute
the first tier of such archival undertaking.
The Stanford Archival Vault (SAV) in Stanford University
assigns a numerical handle to every digital "object" (record)
in a repository.
The handle is the clever numerical result of a mathematical
formula whose input is the number of information bits in the
original object being deposited. This allows to track and
uniquely identify records across multiple repositories. It
also prevents tampering. SAV also offers application layers.
These allow programmers to develop digital archive software
and permit users to change the "view" (the interface) of an
archive and thus to mine data. Its "reliability layer"
verifies the completeness and accuracy of digital
repositories.
The Internet Archive, a leading digital depository, in its own
words:
"...is working to prevent the Internet -- a new medium with
major historical significance -- and other "born-digital"
materials from disappearing into the past. Collaborating with
institutions including the Library of Congress and the
Smithsonian, we are working to permanently preserve a record
of public material."
Data storage is the first phase. It is not as simple as it
sounds. The proliferation of formats of digital content has
made it necessary to develop a standard for archiving Internet
objects. The size of the digitized collections must pose a
serious challenge as far as timely retrieval is concerned.
Interoperability issues (numerous formats and readers)
probably requires software and hardware plug-ins to render a
smooth and transparent user interface.
Moreover, as time passes, digital data, stored on magnetic
media, tend to deteriorate. It must be copied to newer media
every 10 years or so ("migration"). Advances in hardware and
software applications render many of the digital records
indecipherable (try reading your word processing files from
1981, stored on 5.25" floppies!). Special emulators of older
hardware and software must be used to decode ancient data
files. And, to ameliorate the impact of inevitable natural
disasters, accidents, bankruptcies of publishers, and
politically motivated destruction of data - multiple copies
and redundant systems and archives must be maintained. As time
passes, data formatting "dictionaries" will be needed. Data
preservation is hardly useful if the data cannot be searched,
retrieved, extracted, and researched. And, as "The Economist"
put it ("The Economist Technology Quarterly, September 22nd,
2001), without a "Rosetta Stone" of data formats, future
deciphering of stored the data might prove to be an
insurmountable obstacle.
Last, but by no means least, Internet libraries are Internet
based. They themselves are as ephemeral as the historical
record they aim to preserve. This tenuous cyber existence goes
a long way towards explaining why our paperless offices
consume much more paper than ever before.
|