Better Practice Checklist - 9. Selecting a Content Management System
May 2004 (organisational details updated January 2008)
Introduction
A wide range of Australian Government departments and agencies are planning to select and implement a content management system (CMS). With a large number of products in the marketplace, and capabilities still evolving rapidly, choosing the right CMS can be challenging. The steps to consider in the implementation of a CMS are the topic of Better Practice Checklist 10, Implementing a Content Management System.
A key role of Australian Government Information Management Office (AGIMO), Department of Finance and Deregulation is to identify and promote 'Better Practice'. This checklist is designed to assist agencies in considering key issues when selecting a CMS. The items in the checklist are, however, not mandatory.
This checklist is intended to be used by those staff in an organisation who have been given the responsibility to determine CMS requirements and evaluate products. The information within this checklist may also be relevant to Chief Information Officers, Information Management steering groups and other senior managers. The checklist focuses on non-technical issues.
It should be noted that the checklist is not intended to be comprehensive. Rather, it highlights key issues for agencies. The checklist is iterative and draws on the expertise and experience of practitioners. The subject matter and issues are reviewed and updated to reflect developments.
Download PDF of Checklist 9 - Selecting a Content Management System [
- 305 KB]
Acknowledgments
This checklist was developed with the assistance of Australian Government agencies. In particular, we would like to thank the Bureau of Meteorology and the Content Management Community of Practice.
In addition, we would like to thank Step Two Designs Pty Ltd.
Overview of selecting a CMS
There is no single 'best' CMS. Instead, each CMS product has a mix of strengths and weaknesses that derive from its underlying architectural model or market position. This applies equally to products in all price ranges, from small-scale CMS offerings to large 'enterprise' products.
The content management marketplace is comparatively young compared to other IT systems, and is still undergoing rapid evolution A few years ago there were a small number of vendors, but recently the marketplace has grown rapidly with the entry of a large number of new products.
The challenge is for agencies to select a product that best matches their unique needs, while mitigating the risks inherent in the current marketplace. Another critical issue is implementation of the chosen CMS, which is covered in Better Practice Checklist 10, Implementing a Content Management System. The basic issues associated with managing online content are covered in Better Practice Checklist 8, Managing Online Content.
Summary of Checkpoints
Before you start
Consider the time required to select an appropriate product
Consider the range of products available in the marketplace
Build content management knowledge
Consider risks and risk mitigation strategies
Redesign the site if necessary
Determining project goals and targets
Focus on business outcomes
Specify targets and how their achievement will be measured
Determining and documenting business requirements
Focus on business needs rather than technical solutions
Address compliance needs
Provide descriptions and examples to clarify meaning
Avoid specifying too many requirements
Make use of existing resources
Consider gaining an external review of requirements
Consider the use of scenarios to help document requirements
Producing tender documentation
Include background information
Include any new site design
Specify technical infrastructure
Evaluating and selecting a CMS
Use scenarios during demonstrations
Visit reference sites
Focus on usability and simplicity
Assess the vendor's implementation methodology
Consider the total cost of ownership
Checkpoints
Before you start
Consider the time required to select an appropriate product
Broad industry experience suggests that the evaluation and selection process can take upwards of six to nine months in practice. Failing to allocate sufficient time to these activities can expose agencies to the risk of selecting an inappropriate product, and agencies that have already implemented a system generally emphasise the importance of spending time to fully develop and understand CMS business requirements.
Consider the range of products available in the marketplace
In an environment of rapidly evolving CMS capabilities, agencies may find it useful to increase their familiarity with a range of products as well as the overall state of the marketplace. An up-to-date list of CMS products in the Australian market can be found at www.steptwo.com.au/cm/vendors/australian/index.html [
].
Build content management knowledge
With many agencies purchasing a CMS for the first time, one of the greatest challenges may be obtaining sufficient knowledge about content management tools and processes.
Without this knowledge, agencies may find it difficult to compare products and may encounter unexpected challenges during implementation. Building internal knowledge about CMS products before proceeding to evaluate and select options can help mitigate risks and maximise project outcomes.
A number of sources of knowledge regarding CMS are available, including:
- industry articles and books
- presentations, conferences and workshops
- other agencies
- vendor showcases
Consider risks and risk mitigation strategies
Consider possible project risks, and ways to manage these risks, when selecting a CMS product and planning the implementation project. A number of approaches are now being used in both public and private sector agencies to manage project risk, including:
- Constraining the overall scope of the project, for example, by starting with a single site or focusing on the better-understood business requirements.
- Avoiding very large 'enterprise content management' projects that attempt to address all information needs within an organisation.
- Implementing CMS as part of a 'pilot' project. This allows the suitability of the chosen product to be evaluated, while constraining costs and minimising business risks.
- Focusing on gaining content management knowledge, expertise and capabilities within the agency. By combining this with a pilot project, agencies can use the implementation project to gain a clearer understanding of what their key business requirements are for content management.
Redesign the site if necessary
Agencies may consider using a CMS to address limitations and deficiencies in their current website or intranet. In this situation, the site can redesigned as part of the content management project.
Each CMS has specific capabilities, strengths and weaknesses in its ability to handle specific site design, structure and navigation needs. For example, some systems can only handle three levels of navigation. Depending on the design of the site, this may be desirable (as it simplifies the CMS interface) or it may be a major limitation. It may be worthwhile to redesign the site first, and then the new designs can be used as part of the selection criteria for the CMS.
Redesigning websites is a major undertaking that involves content audits, user-centred design processes, usability and information architecture techniques, amongst other things. Further details are available in Better Practice Checklist 15, Information Architecture for Websites.
Determining project goals and targets
Agencies may find it useful to determine the business goals and outcomes for the project as the starting point for the selection process. These goals and outcomes can then guide the requirements-gathering process and form the basis for 'weighting' the various aspects of CMS functionality during evaluation.
Focus on business outcomes
Content management projects that focus on broader business outcomes may be more effective than projects that do not. Examples of business-focused goals could include improving customer satisfaction, better informing the public, or improving knowledge-sharing within the organisation.
Specify targets and how their achievement will be measured
For each of the project goals it may be useful to identify targets and determine how their achievement will be measured. This will help determine the success of the project as well as provide a basis for assessing the ongoing viability of the CMS. Attributes to be measured could include:
- site usage
- system usage
- information quality
- user feedback
- maintenance costs and savings
- staff efficiency
- customer satisfaction.
Determining and documenting business requirements
A key step in choosing a CMS can be the identification and documentation of business requirements. As the CMS products on the market are so diverse in their designs and features, it may not be possible to directly compare one product against another. Instead, CMS products can be evaluated on how closely they fit the organisation's unique business requirements.
Focus on business needs rather than technical solutions
Specifying detailed business requirements rather than technical solutions provides vendors with an opportunity to propose methods or technologies that can meet the business goals of the website.
Address compliance needs
A range of compliance issues may need to be considered, including:
- Recordkeeping. The National Archives of Australia has published A Policy for Keeping Records of Web-based Activity in the Commonwealth Government []. This is also the subject of Better Practice Checklist 7, Archiving Web Resources.
- Accessibility. Government websites are required to be accessible by disabled users and users of assistive technology. These requirements have been outlined by the W3C Web Accessibility Initiative (WAI) www.w3.org/WAI. []
- Legal risk. Agencies should also assess the legal risks inherent in providing information via their website or intranet, and take appropriate actions to mitigate these risks (such as version control and audit trails).
- Security risk. Agencies should ensure that the CMS product does not contain any security flaws that can increase the likelihood of the agency's website being compromised.
Provide descriptions and examples to clarify meaning
There is little consensus regarding content management terminology within the industry. To avoid misunderstandings, consider carefully the use of jargon and acronyms, and spell out requirements in as much detail as possible. Examples showing concrete situations and business needs are an effective way of supporting requirements.
Avoid specifying too many requirements
Care should be taken not to 'overload' the list of business requirements. While documenting more requirements may be beneficial, there is the risk of forcing the purchase of an expensive solution where a more modest option would suffice. This would then impact upon training, product usability, licence fees and ongoing maintenance costs.
Make use of existing resources
A number of existing resources can be used as the starting point for further requirements gathering, including:
- Victorian Government Content Management and Standards [].
- Commercially available toolkits/planners/reports. A number of commercially published reports, toolkits and planners are available to support organisations when developing CMS requirements and selecting products.
Consider gaining an external review of requirements
The clarity and viability of CMS requirements can be hard to assess by the business team who developed them. Agencies may therefore consider gaining an external review of the requirements from similar agencies (particularly ones that have already obtained a CMS) or from others.
Consider the use of scenarios to help document requirements
Scenarios are narrative descriptions (stories) that outline how something will work in practice. In the context of a CMS project, scenarios are a very effective way of documenting key CMS requirements, and they complement the formal lists of functional requirements typically found in tender documents.
The following is an example of a scenario that could be developed to assist documenting CMS requirements:
Robert is an author who is responsible for maintaining some sections of the website.
A new press release needs to be created for the site. Robert creates a new page in the CMS, indicating that it will be a press release. This provides a number of specific fields to be filled in (such as title, release date and contact details for the press officer), as well as an area for the body of the press release (as formatted text).
Robert then specifies the AGLS Metadata for the page (this has been made very simple by the CMS, which has defaulted many of the values). He also indicates that the press release should be released at 9 am tomorrow morning, in sync with when the press release will go to the media.
Having completed the mandatory metadata for the press release, Robert forwards it to his manager, Lyn, for her review. Lyn is happy with the release, so she forwards it to Sandra (one of the web team) to do a final quality check, before approving it for release.
Scenarios can also help to:
- Clarify issues. When writing is in a narrative format, any gaps, errors or areas of uncertainty become very clear. Ensuring that the scenarios are fully understood therefore resolves any issues before the tender is conducted.
- Make abstract requirements concrete. By focusing on specific activities and tasks, the functional requirements (which can be abstract for most readers) are made concrete and placed in context of the broader business processes.
- Facilitate internal review. Detailed functional specifications are very hard for business users to understand and review, and this difficulty often leads to requirements being omitted or misunderstood. By documenting the requirements in a short, simple and easy-to-read format, scenarios help internal stakeholders to contribute meaningfully to the project.
Producing tender documentation
Contracts or procurement units in agencies can provide assistance in developing tender documentation and should be contacted to ensure that the agency's requirements are adhered to. Government procurement should be undertaken in accordance with the Commonwealth Procurement Guidelines.
The primary aim of a CMS tender is to communicate the business requirements to vendors, as well as to allow internal stakeholders to determine whether their needs have been accurately captured.
Effective tender documents should focus on providing a clear and simple description of needs, with a minimum of 'legalese' or other formality. Additional information should be provided wherever possible, to further explain requirements.
Include background information
The tender should include sufficient background information to provide the vendor with a context for the business requirements. This should include project goals, current site features and issues, and other agency characteristics.
Include any new site design
If the site is being redesigned as part of the project, the prototype designs and site map can be included in the tender documentation, to allow vendors to demonstrate how they would accommodate the site.
Specify technical infrastructure
The specific IT environment and constraints within the agency should be specified within the tender, including the preferred operating system, database platform, workstation configuration and web browser. Specifying exact requirements allows the compatibility of the CMS with the current agency environment to be meaningfully assessed.
Evaluating and selecting a CMS
When selecting a CMS, agencies should follow the standard tendering and procurement processes used for obtaining software solutions, as well as consider other processes as necessary.
Use scenarios during demonstrations
Scenarios are an effective way of ensuring that vendor demonstrations clearly show how the CMS will meet the specific business needs. They also allow demonstrations to be compared and scored.
Visit reference sites
Vendors may be able to provide contact details for similar organisations where their CMS solution has been installed. These sites can then be visited (without the vendor) to determine how well the product works in practice. This is one of the most effective mechanisms for reducing the risk of choosing an inappropriate product.
Focus on usability and simplicity
While a number of products may match the identified functional requirements, the overall usability and simplicity of the CMS may be a key consideration. Complex or counter-intuitive interfaces can impact upon training needs and staff uptake and, in extreme cases, can lead to project failure.
Assess the vendor's implementation methodology
Agencies may look to the vendor (or implementation partner) when deploying the CMS. It may therefore be useful to assess the implementation methodology proposed by the vendor, paying particular attention to the non-technological aspects (such as training, change management, usability and information architecture).
Consider total cost of ownership
The total cost of ownership, not just the initial purchase price, may be an important consideration when selecting a CMS. Total cost of ownership may include the:
- amount of customisation required
- technical skills and knowledge required by internal staff
- degree of ongoing reliance on the vendor
- licensing models and fees
- third-party products required
- IT infrastructure (hardware and software) required.
Other Better Practice Checklists
- Providing Forms Online
- Website Navigation
- Testing Websites with Users
- Use of Cookies in Online Services
- Providing an Online Sales Facility
- Use of Metadata for Web Resources
- Archiving Web Resources
- Managing Online Content
- Selecting a Content Management System
- Implementing a Content Management System
- Website Usage Monitoring and Evaluation
- Online Policy Consultation
- Knowledge Management
- Designing and Managing an Intranet
- Information Architecture for Websites
- Implementing an Effective Website Search Facility
- Spatial Data on the Internet
- Digitisation of Records
- Access and Equity Issues for Websites
- Marketing E-government
- ICT Support for Telework
- Assistive Technology for Employees of the Australian Government
- Decommissioning Government Websites
- ICT Asset Management
- Managing the Environmental Impact of ICT
Download PDF of Checklist 9 - Selecting a Content Management System [ - 305 KB]
Contact for information on this page: AGIMO Better Practice Team