Better Practice Checklist - 7. Archiving Web Resources

May 2004 (organisational details updated January 2008)

Introduction

Australian Government departments and agencies use a range of new technologies for information provision, service delivery and administration.

A key role of the Australian Government Information Management Office (AGIMO), Department of Finance and Deregulation is to identify and promote 'Better Practice'. This checklist has been created to help agencies to create, capture, manage and retain records of web-based activity for as long as the records are required.

This checklist outlines a number of issues for consideration when determining and implementing appropriate strategies for making and keeping records of online resources. Not all items in the checklist are mandatory, but they have been provided to help agencies to develop a strategy specific to their individual online recordkeeping needs.

This checklist is intended to be a guide to staff responsible for websites, including content authors, website administrators, recordkeeping practitioners and IT staff such as network managers or data administrators. Managers may also find the checklist useful in dealing with contractors, or where these functions are otherwise outsourced. This checklist focuses on non-technical issues.

It should be noted that the checklist is not intended to be comprehensive. Rather, it highlights key issues for agencies. The checklist is iterative and draws on the expertise and experience of practitioners. The subject matter and issues are reviewed and updated to reflect developments.

Comments on the checklist and suggestions for additional checkpoints are welcome.

Download PDF of Checklist 7 - Archiving Web Resources [ - 295 KB]

Acknowledgments

Originally published by the National Office for the Information Economy (NOIE) in 2001 (Version 1), and updated in 2002, (Version 2), this checklist was revised in 2004 with assistance from Australian Government agencies. In particular, we would like to thank the National Archives of Australia.

Why archive online resources?

Under s. 24 of the Archives Act (1983), Australian Government agencies must retain or dispose of their records in accordance with instructions issued by the National Archives of Australia. In particular, agencies must comply with the National Archives' policy statement Archiving Web Resources: A Policy for Keeping Records of Web-Based Activity in the Commonwealth Government [].

For practical advice on how to comply with this policy and how to implement checklist items, agencies can use the National Archives' Archiving Web Resources: Guidelines for Keeping Records of Web-Based Activity [].

There is no generic solution for creating and maintaining records of web-based activity. The most appropriate recordkeeping strategy is likely to use a combination of approaches, depending on the outcome of an analysis of an agency's particular circumstances.

Summary of Checkpoints

check box Assign and document recordkeeping responsibilities in the agency

check box Implement a recordkeeping system

check box Conduct a risk assessment of web-based activity

check box Ensure that the recordkeeping system supports the creation and maintenance of adequate recordkeeping metadata

check box Decide on object-driven or event-driven strategies (or a mixture of the two) for ensuring that records of the agency's web-based activity are created and captured

check box Train staff in using the recordkeeping system to ensure that full and accurate records of web-based activity are created and captured

check box Ensure that the National Library of Australia is informed of government publications that are available only online

check box Implement policies and procedures for managing hardware and software dependency and obsolescence

check box Consider the issues associated with ensuring the accessibility of web-based records over time

check box Wherever possible, use open standards in preference to proprietary standards and platforms

check box Conduct regular review and monitor developments

check box Arrange for archival records to be transferred to the National Archives of Australia

Checkpoints

check box Assign and document recordkeeping responsibilities in the agency

A major component of an agency's internal accountability process should be assigning and documenting responsibilities for web-based recordkeeping. Three or four different groups of staff may have responsibility for making and keeping records of web-based activity and resources, including content authors, website administrators, recordkeeping practitioners and IT staff (such as network managers or data administrators).

For example, responsibility for placing individual records of web-based activity into an agency's formal recordkeeping system could be assigned to the content author or to recordkeeping staff, rather than to the administrator who manages the website itself.

The development of relevant policies, procedures and standards will be crucial, as will ensuring that these are actually followed in the agency.

check box Implement a recordkeeping system

The creation and maintenance of authentic, reliable, accurate and durable evidence of web-based activity are essential if agencies are to retain corporate memory and meet legal obligations and community expectations. For guidance on designing and implementing a recordkeeping system to capture and retain web-based records, agencies should refer to Digital Recordkeeping: Guide to creating, maintaining and preserving digital records []. Following the practical guidance will enable an agency's recordkeeping practices to meet the Australian and International Standard AS ISO 15489, Records Management, and will ensure the creation and capture of full and accurate records in an electronic environment.

Implementing DIRKS will ensure that an agency's ongoing electronic recordkeeping requirements are identified and met. However, additional guidelines, mentioned in the following checkpoints, should be adhered to in order to ensure that web resources are effectively archived and accessible. Some of the checkpoints below identify strategies essential to ensuring that preservation responsibilities are met.

check box Conduct a risk assessment of web-based activity

An agency should be able to demonstrate the likelihood of being held liable or subject to litigation for the contents of its website, and being called on to prove or disprove the existence of particular content on its website at a given date and time. This applies to both past and present versions of the site. Agencies may find it useful to provide a disclaimer on their sites. Guidance on the content and form of this disclaimer can be obtained from the Australian Government Solicitor.

Each agency must take into account the following factors when performing a risk assessment of web-based activity:

public visibility
the purposes for which the web resources are used
the nature of the material on the website
the audience of the web resources
the frequency of changing and updating information on the website.

Websites that change frequently but irregularly are likely to pose the most degree of risk to an agency, because such changes are the most difficult to track.

Further guidance on risk management and the conduct of a risk assessment can be obtained from Australian Standard AS 4360-1999, Risk Management.

check box Ensure that the recordkeeping system supports the creation and maintenance of adequate recordkeeping metadata

Metadata is structured information attached to web-based resources that allows the records to be found, managed, controlled and understood.

Australian Government agencies are required to describe their web-based resources using metadata that is compliant with the metadata elements as outlined in Australian Standard AS 5044-2002, AGLS Metadata Element Set. Consistent application of this standard will assist members of the public to locate and access the government information and services they need.

Guidance for Australian Government agencies on implementing AGLS Metadata Element Set may be found in the Australian Government Implementation Manual for AGLS Metadata [].

Further, the National Archives' Recordkeeping Metadata Standard for Commonwealth Agencies [] specifies additional metadata that agencies should use to describe the context, management, use, preservation and disposal of information resources, including records of web resources and activity.

Recordkeeping Metadata Standard for Commonwealth Agencies is compatible with AGLS Metadata Element Set. Further details about applying metadata to website content is available in Better Practice Checklist 6, Use of Metadata for Web Resources.

check box Decide on object-driven or event-driven strategies (or a mixture of the two) for ensuring that records of the agency's web-based activity are created and captured

The selection of appropriate strategies is contingent on the type or complexity of the web resources being managed.

Object-driven approach. This approach concentrates on managing the 'objects' that constitute or are made available via a website - for example, collections of HyperText Markup Language (HTML) documents, and/or the various objects assembled to create an HTML document when a user queries the site. This approach could entail taking periodic snapshots of collections of web resources in combination with tracking changes to the site and logging transaction details. Alternatively, objects could be separately captured and managed in association with metadata that describes the relationship between specified versions of the object and its Universal Resource Indicator (URI).
Event-driven approach. This approach focuses on capturing 'events' or transactions that occur between the website and the user. This approach may be appropriate for websites that:
- are dynamically generated
- are database-driven
- rely on stored user profiles, and/or
- primarily provide transactional services.

This approach would require the capture and storage of the: date and time of the event; IP or domain address of the user; user profile; query or other action performed; and resource served to the user, with relevant metadata attached.

check box Train staff in using the recordkeeping system to ensure that full and accurate records of web-based activity are created and maintained

In an electronic recordkeeping environment, all agency staff become responsible to some extent for carrying out good recordkeeping. Within their organisation, agencies should raise the profile and general awareness of these responsibilities, especially the need to create and capture records of web-based activities, and should provide the necessary training.

check box Ensure that the National Library of Australia is informed of government publications that are available only online

The National Library of Australia (NLA) is responsible for building the National Collection of Australian Online Publications. It has established the PANDORA archive (Preserving and Accessing Networked Documentary Resources of Australia, pandora.nla.gov.au [] to manage these publications and to provide long-term access to them.

Australian Government publications that are available only online are a high priority for selection for this archive, and the NLA should be notified of their existence. Email notification can be sent to elecunit@nla.gov.au providing the title and the URL, or the notification form available at pandora.nla.gov.au [] can be used.

Online publications eligible for archiving by the NLA include reports, manuals, newsletters, discussion papers and other public accountability documents, as well as any commercial publications. The NLA will negotiate with Australian Government agencies about providing access to any online commercial publications. The NLA does not usually archive working/exposure drafts, internal agency records, press releases, speeches and public notices. If in doubt, it is best to notify the NLA.

check box Implement policies and procedures for managing hardware and software dependency and obsolescence

Information and communications technology (ICT) is subject to ongoing technological obsolescence. Hardware and software become quickly outdated, and thus electronic material can become unreadable. Agencies, especially those taking website snapshots for online or offline storage, need to consider a number of interrelated hardware and software factors - for example, the estimated physical and/or commercial life of the medium on which a website snapshot and its related metadata are stored; and the long-term availability of the hardware and operating system platforms needed to access records stored on different types of media.

check box Consider issues associated with ensuring the accessibility of web-based records over time

These issues may include:

carefully managing records (for example, regularly exercising and refreshing media)
planning for obsolescence by ensuring that records can be copied, reformatted or migrated
using widely supported standards
implementing security measures to protect records against deliberate or accidental alteration
using persistent identifiers
ensuring environmental control and monitoring.

check box Wherever possible, use open standards in preference to proprietary standards and platforms

When designing and building websites, agencies should plan to use software tools and applications that meet accepted (or de facto) standards and that are readily available and fully supported. Factors to consider include maintaining records in storage and data interchange formats that are widely accepted and technology-neutral (such as eXtensible Markup Language - XML), and avoiding the use of non-standard HTML language tag extensions.

check box Conduct regular reviews and monitor developments

A review should be conducted within six months of implementing a strategy for web-based recordkeeping. The review should identify the effectiveness of the approach, measure performance, identify any corrective action or enhancements required, minimise the agency's exposure to risk through procedural failure and develop a program for conducting regular reviews.

Website technology is changing dramatically and rapidly. It is important that agencies monitor web development technologies, emerging trends and business applications, as they will continue to impact on strategies and techniques for recordkeeping and archiving websites.

check box Arrange for archival records to be transferred to the National Archives of Australia

Agencies should contact the National Archives of Australia to discuss suitable arrangements for transferring to archival custody those records of web-based activity that have been appraised as 'Retain as National Archives' under an appropriate National Archives disposal authority.

Other Better Practice Checklists

Download PDF of Checklist 7 - Archiving Web Resources [ - 295 KB]

Contact for information on this page: AGIMO Better Practice Team

Skip links

Main site links

Contact and help

Menu