Home >> Publications >> ICT Procurement >> Guide to Open Source Software for Australian Government Agencies >> Risk mitigation procedures

Guide to Open Source Software for Australian Government Agencies

Risk mitigation procedures

Before deciding to adopt any open source product, it is necessary to undertake a careful review of the technology, its history, patterns of use, popularity and feedback from existing users. Depending on the sourcing option, this review is the responsibility of either the agency or the agency in conjunction with its external service provider. The review needs to address the following issues:

• Confirm the software operates as described by the project’s website and documentation;
• Ensure performance matches or exceeds the agency’s mandatory requirements;
• Confirm the project is being maintained. Indications of this include:
  - ongoing and currently announced and released versions, security fixes and feature updates;
  - updates and announcements published on the project website and information forums;
  - an active user discussion forum or mailing list, archives of which are publicly available;
  - a publicly available project roadmap detailing technology direction; and
  - a growing user base; and
• Confirm the availability of local service providers to support the product (these providers are sometimes listed on the project’s website).

Agencies may wish to consider completing this due diligence process for each major component of an open source solution.

A project mailing list or forum is an excellent source of information about the suitability, stability, quality and robustness of a particular technology. Look for comments from users to indicate that they are generally happy. Also look for comments from users indicating that they have been using the software in production for some time with no serious issues.

Assessing sourcing risks

Depending on an agency’s sourcing strategy, managing open source procurement may require little or no change to existing business processes. On the other hand, certain scenarios may dictate a considerably different approach. By examining the spectrum of sourcing options, it is possible to highlight any project management changes required.

In-house sourcing

Procurement by direct download of open source software represents a new approach for agencies. As might be expected, downloading any type of software directly from the Internet involves a higher level of risk. Unless agency staff have the particular technical skills required to check the veracity of the download, there are potential issues with infection by malicious software such as viruses, trojans and key-loggers.

Another issue is the question of indemnification and warranties, covered in detail in the previous chapter. Since in-house sourcing of open source solutions does not provide agencies with warranties and indemnities, alternative risk mitigation strategies need to be considered. This may include establishing indemnification insurance policies with a specialised software insurance risk broker. There are insurance firms that specialise in providing such policies to users of open source software.

Agencies need to investigate the risks involved in deploying or migrating to open source solutions. Detailed understanding of the technologies involved increases the likelihood of successful deployment and integration with existing ICT infrastructure and platforms.

Wherever possible, an agency should only adopt open source software if it has identified more than one vendor offering local support. Identifying support vendors in advance is a worthwhile step even when an agency decides to use in-house sourcing. Local support vendors offer a safety net for the agency in situations where in-house expertise is not able to resolve a serious issue.

This is particularly important in situations where the software in question is used for mission-critical activities. Agency technical staff might undertake most day-to-day operational duties. However, the availability of reliable second and third-level technical support is an important step in reducing risk when adopting open source solutions.

If an agency selects an in-house sourcing model, they must ensure technical and procurement staff allocate the time necessary to understand the platforms and components that are expected to form the proposed open source solution. Many solutions are comprised of numerous interlinking components. Familiarity and understanding of the relationships between these components makes it easier to determine if the solution is fit for purpose.

This process is covered in detail in Appendix C, “Open source software packaging”.

External sourcing

The starting point for any external sourcing arrangement is a vendor selection process that includes risk assessment. In this context, deploying OSS solutions is no different to other solutions. The onus is on the provider to cover the agency for any legal risk that may arise. Agencies need to undertake thorough due diligence to mitigate technology risks and change management risks.

If an agency selects an external sourcing model for OSS solutions, the vendor should assume all support responsibilities. Agencies need to ensure that the vendor takes explicit responsibility for all appropriate risk mitigation procedures for the software in question. If the agency has policies about the security standards of all software introduced onto its network, vendors should be required to follow such policies as part of their risk analysis. Compliance with security standards should be included in the commercial agreement, although the agency may need to enact a system of checks to verify compliance.

The agency should require the vendor to undertake a full audit of all software components (both proprietary and open source) to establish their pedigree and verify licence agreements and risk assessment documentation. Software integration issues should be understood and documented. In addition, the agency may undertake its own assessment to verify the vendor’s performance.

Procuring open source solutions through any vendor also introduces risks associated with vendor competence, reach and maturity. These issues may be more acute when dealing with smaller or less established vendors. Each potential vendor needs to undergo an appropriate due diligence assessment. Such an assessment must cover:

• Financial strength and stability;
• Endorsed Supplier status;
• Risk management credentials;
• Evaluation of internal controls;
• Review of business continuity plan;
• Analysis of third-party exposure;
• Accounting policies and practices;
• General capability overview;
• Commercial management;
• Service delivery and management; and
• Other factors outlined in the Australian Government publication A Guide to ICT Sourcing[4].

Assessing long-term product viability

Open source projects succeed and become popular only when they are useful to constituent user communities. To avoid being left with potentially unsupported OSS products, agencies should assess the long-term viability of software options before they enter service. For example, if an open source project has a shrinking user base this could imply a problem with the technology. It could also imply dwindling resources dedicated to the product’s upkeep. Either point should be considered as negative when assessing the risk of the product.

While it is possible (if deemed important enough) for a single agency to continue maintaining an open source project after other users have discontinued use, as the last remaining user the agency assumes all maintenance responsibilities for that product. This includes responsibilities for support, security fixes and ongoing technical development. Such a scenario should be avoided unless the software in question is particularly unique and valuable.

Before undertaking such a primary maintenance role, an agency must be able to justify its position. The costs and effort involved in ongoing support of an open source project need to be weighed against the costs and effort involved in moving to more broadly supported software. The costs of migrating to a newer solution may outweigh the advantages, justifying continued maintenance of the existing solution. In such cases, the existing open source codebase can be maintained until this equation changes. Agencies need to factor such scenarios into the agency’s risk mitigation strategies.

The aim of this analysis is to reduce the chances of any open source product used in production becoming ‘orphaned’. Similar guarantees cannot be made for proprietary products. Once the vendor decides to reduce or terminate support for a particular proprietary technology, agencies often have no alternative source for support, security fixes and ongoing technical development. This risk must also be factored into product acquisition.

Assessing technology roadmap risks

Agencies should seek technology roadmaps for all important software used in production. A technology roadmap is a document that provides an outline of projected future improvements and delivery timelines. Its purpose is to help users of the software to understand what lies ahead.

Obviously there is no guarantee that what a vendor states in its technology roadmap will come to fruition. This is equally true for open source development groups and vendors of proprietary software. However, it is important to understand the future direction or vision of the product whenever possible, as this knowledge helps an agency measure the future obsolescence risks involved in selecting particular technology.

Equally important is the vendor’s ability to execute on its vision. A vendor that changes its product roadmap or its general technology platform should be classed as a higher risk provider. The agency has less surety that the vendor’s products will evolve the way the agency expects. There may also be less assurance that the product will be supported in years to come.

Agencies should factor in the risks associated with functional variations to the product in future versions. It is possible that the vendor may take the product on paths leading away from the agency’s original requirements. Agencies should request vendors to make their future intentions clear in product roadmaps so they can assess strategic ICT impact.

The agency should regularly review the product roadmap – perhaps every six to 12 months – in order to better gauge future risks. Regular reviews can assist the agency to develop an understanding of the vendor’s credibility and ability to execute on the roadmap. This review process can then be fed into future planning assumptions.

As a minimum, the roadmap assessment and review process should:

• Locate and record the roadmap presented by the product’s developer consortium or vendor;
• Locate and record information that the vendor offers to its client base and user community, including information about the latest developments, product release dates and versions;
• Check how current the timeline is. For example, if the timeline references future events that are now months or years in the past, this can indicate an information repository that is no longer well maintained – which may in turn signal problems with the development consortium or vendor;
• Contact the nominated representative of the consortium or vendor. Seek information on the frequency of updates to the roadmap and release schedules;
• Once this timetable is established, locate and record separate copies of web pages containing both the product roadmap and release schedules;
• Compare the latest roadmap with previous roadmaps, noting modifications. Specifically, look for the removal of milestones that were actually achieved and the addition of future product milestones;
• Compare the software developer’s latest news items web page with previous versions, noting any additions;
• Compare the timeline and dates found on the current news items web page to confirm how many of the originally proposed milestones were achieved. Note this number for future reference; and
• Repeat this cycle for as long as your analysis period requires.

In time, agencies should be able to determine with some precision how well different open source developer consortia and vendors keep to their published timelines. This information should be tabled as part of the risk assessment for that particular product or vendor.

This process can also alert an agency to problems that may arise in the core group that produces and maintains a particular piece of software. Early warning of problems can lead to more effective management of the risks involved in transitioning from such software.

Managing risks in technology roadmap variation

Some open source development groups or vendors might not provide a technology roadmap of sufficient quality and detail. Alternatively, they may not consistently adhere to their own roadmaps. Such projects may make it harder for agencies to plan future strategies. This is another risk that needs to be managed.

However, in some scenarios the lack of a consistently executed roadmap may be less of a concern. For example, if an agency selects or uses a mature and widely used OSS product, the risks involved in using such software are generally low. The software normally has all the features it needs to fulfil the needs of the agency and other users. New features are rarely necessary.

Similarly, if the product forms part of fixed system infrastructure that works well and the agency has no future plans or requirements for enhancement, then the lack of a roadmap poses minimal risk. Many open source infrastructure components change slowly once they mature. As a result, the development group may not feel a need for lengthy discussion of future directions, which may be the reason for a lack of technology roadmap.

The following checklist may be useful for agencies when considering procurement of open source software and services.

Due diligence risk mitigation checklist

If the project needs additional features to meet agency requirements, is there an obvious process for requesting new features or enhancements from the
development team?

Contact for information on this page: SourceIT@finance.gov.au

Back to top