Archives New Zealand has launched a programme to increase awareness among government departments of the need to manage data for the long term.
A crucial point being made is ensuring the continued quality and accessibility of digital data as technologies change.
The State Services Commission’s e-government unit sees maintaining data as part of its next phase of development.
“We’ve concentrated initially on establishing an infrastructure,” says policy and development manager Hugh McPhail. “Now we’re moving into delivery of services.”
Considerations of data quality and longevity will be part of this second phase, but not an immediate priority, he says. “Our high priorities for the near future will be authentication and [government agencies’] adoption of [e-procurement service] GoProcure.”
Government agencies do need to be aware of data quality, he says. “But Archives is the expert in this area.”
The e-government unit’s longer-term view is perhaps surprising as Archives labels electronic document management, quality and continued readability “an important factor underpinning the successful move to e-government”.
The question of document longevity was most recently raised in the context of open source software by David Lane, spokesman for the open source ginger group Openz. He notes that documents created in early versions of some office software, such as Microsoft Word and Excel, cannot be read by the current versions of the same proprietary products. He naturally holds out open source as a salvation, offering up such tools as the Neda format, based on XML.
Archives New Zealand, alongside archiving agencies all over the world, has been considering the issues of data quality and longevity for some years, and formulating policy. The programme launched last week is its first major effort to promulgate elements of that policy. Archives NZ has given little attention to open source, says spokesman Michael Hoyle, though some overseas archives organisations have.
“The main way most archives are looking at solving the problem is through migration strategies,” he says, converting the document progressively to each major new version of the application that enables it to be read. Storage media may also require a migration strategy.
An Archives “background paper” on electronic record policies and strategies has “migration” as its sub-heading for discussion of continued readability over time.
“There is acceptance of the fact that not only will the records in custody need migration (or perhaps another preservation technique) but also the fact that the systems that store and manage those records will require a migration strategy in the face of the fast moving IT developments,” says the document, which was prepared for Archives in 2001 by Australian company Recordkeeping Systems.
The problem with migration is that it can often slightly change the format of the document and lose some information, McPhail says. “There are other approaches, such as emulation.”
The preceding section of the document, headed “storage formats”, notes: “Material from most office-based applications is accepted in PDF format, and XML is clearly the preferred format for meta-data and some rendering of records themselves. The database material remains more intractable, with a variety of formats still being accepted from flat files [and] comma-delimited outputs with limited ability to accept arrays and tables.”
Disability spokespeople, whom the e-government unit consults as part of its “accessibility” objective, generally express a strong dislike of PDF, which does not work well with the technologies used by many disabled people.
Last month’s launch was led by State Services Commission chief Michael Wintringham and Archives minister Marian Hobbs.