Subscribe
  • Home
  • /
  • Storage
  • /
  • Merge microfilm, digital for successful archiving

Merge microfilm, digital for successful archiving

Government departments need to focus on preserving records for both access and retention.
By Paul Mullon, Information governance executive at Metrofile.
Johannesburg, 16 Sept 2004

By their very nature, governments are compelled to keep mountains of historical information. These are records that not only protect legal rights, but also contain information pertinent to a country`s history. Nowhere is this more relevant than in our own past, where archival information has played an intrinsic role in recording SA`s troubled history and ensuring redress.

As has been the case across all industries, governments are increasingly feeling the need to move document storage to the electronic world, and the South African government is no different. The challenge lies in ensuring that it remains capable of accessing and retaining documents - as it has always done - for the long-term.

Government departments are required to keep records well into the future regardless of the media that happens to be in vogue at any particular time. Archiving requirements remain a particular challenge for government, as it has to retain mountains of incoming original paper documents, as well as a burgeoning wave of electronic information coming in via e-mail.

The retention of electronic records means that the records will be kept as long as legally or historically necessary; that, throughout this period of time, they can be found when needed; that they can be read when found; and that they will be destroyed at the appropriate time. So, while electronic records have at least two advantages over their paper counterparts (because they require minimal storage space and are easy to copy), their retention requires a complex process of identification, isolation and preservation.

Government has begun to see great value in implementing electronic systems. Scanning paper documents is often an effective method of reducing space requirements, improving access and protecting originals. Compared to paper and microfilm, electronic storage also offers rapid and easy access, duplication and instant distribution over networks.

In SA, however, government`s constituents are still largely reliant on paper-based communication. Government is therefore bound to satisfy the demands of these people while also meeting the needs of those who prefer their dealings with government to be conducted electronically. This leads to duplicate systems.

Government therefore has to look at optimising the technologies it uses to cater for both paper and electronic information storage in the short-term, and to consider the best format available for long-term archiving requirements.

In the short-term, the storage requirement is one of access (to enable efficient service), and that is where electronic storage has much value by virtue of the fact that it provides instant access and is available to a vast number of people.

The best way to satisfy both access and retention requirements is to take advantage of the benefits of the electronic world while using media designed to retain information for the long-term.

Paul Mullon, marketing director, Metrofile

For the longer term, the issue is one of retention (to ensure the protection of legal rights). Retention means having the ability to preserve information over extended periods of time - sometimes for over 100 years. It also means ensuring the information is preserved as an exact replica of the original, retaining its integrity. There is, however, also an access component to retention: government employees must have the ability to search, find and retrieve information, even though retrieval will not be instantaneous.

Satisfying the different requirements of access and retention

Electronic information storage is the most appropriate way to go where immediate access is required. This does, however, pose some changes, the first being the fact that technology is changing at a rapid pace. The much-touted Moore`s Law states that chip processing power doubles approximately every 18 months, while costs halve. The obvious cost and speed benefits of evolving technology have resulted in the ongoing evolution of hardware, operating systems and applications.

Long-term retention is dependent on the media in which the information is stored. Here government has a couple of options. The first is to store everything on disk, based on the premise that disks are becoming increasingly cheaper - having Terabytes or even Petabytes of data stored on a single platform sounds just fine at first glance.

But the problem is that with big volumes comes increasing complexity. The time and cost of backups and restores become almost impossible given the limited window that organisations have to perform these tasks. An additional challenge is that while the information might be stored, it is pertinent only to a particular application, and only to a particular version of that application - most applications are backward-compatible for only two generations, which means that government departments wishing to archive information for 100 years can be certain that the application would have gone through at last 50 upgrades.

The next option is to store information on some form of optical device such as CD or DVD. While these were originally thought to have lifespans of 30 years or more, recent studies have shown that the information may not be readable within as little as five years. Temperature, humidity, handling conditions and the quality of the equipment used to write information all compound the problem. In addition, the formats and the media themselves are undergoing such rapid change that there is no guarantee that anyone will be able to read these media types in five to 10 years.

Another option is to store to tape. Tape offers a number of advantages, but is subject to the ravages of time, temperature and usage, and generally lasts no longer than 20 to 30 years. Also, technology obsolescence is once again a problem, as is the fact that different manufacturers have their own standards, making it impossible to guarantee that the tapes will be readable in 30 years` time.

The first possible but highly unlikely solution is to keep copies of every piece of hardware, software, operating system and media type used. Not only would this be prohibitively expensive and unmanageable, but it would also be impossible to find the skills and parts needed to manage those systems.

The next option would be to perform some form of regular migration, which would require taking every archived piece of information from whatever media it may be stored in, and converting it to the latest media format - this would have to be done every five years at least, to keep pace with changes in technology. Not only would this be very costly, but it would also pose a great risk of information loss. In addition, there is also the risk that the converted document will not be a faithful replica of the original, a disaster for the archivist, as it would not stand up in court, resulting in wasted effort all round.

The best way to satisfy both access and retention requirements is to take advantage of the benefits of the electronic world while using media designed to retain information for the long-term. This is fast gaining favour internationally. It involves using digital information storage for short-term access, and the subsequent writing of that to microfilm-based archive media to satisfy retention requirements.

Microfilm-based archiving takes the image directly from the document and writes it immediately to the latest generation of microfilm - designed to last for over 200 years - and produces a faithful replica of the original document at the time that it was generated electronically. The electronic document can be used during the access stage, while the microfilmed copy is sent to a long-term storage site.

No hi-tech skills or equipment are required to read microfilm; all that is needed is a magnifying glass and light. Once a document has been retrieved, digitising on demand can easily move it back to the access phase.

The combination of electronic and microfilm-based storage is the most convenient and cost-effective means of ensuring that government officials can provide efficient service to the public, while also ensuring the protection of legal rights.

Share