Release of Cloud Procurement Discussion Paper

Author: 
John Sheridan - AGCTO & PC

Hello All,

After significant consultation with government agencies and industry, I am happy to announce the release of the Cloud Procurement Discussion Paper regarding an upcoming approach to market to establish a Cloud Panel.

The Discussion Paper incorporates agency and industry feedback sought through working group sessions, blog posts, surveys and discussions with vendors. It proposes a number of innovative approaches to the procurement and I invite you to comment on the Discussion Paper, in particular, on the following:

  • An iterative approach of refreshing the panel
  • Flexibility in adding categories and suppliers
  • Just in time insurance
  • Liability cap set on a contract by contract basis
  • Funding model
  • Statement of Requirement
  • Specification template of initial nine categories
  • Sample evaluation scenarios

Consultation on the Discussion Paper is open until 5.00 pm (AEST) Tuesday, 19 August 2014. Please provide your comments to this blog or send them to CloudProcurement@finance.gov.au

I look forward to your comments.

Thanks,
John

Comments (15)

Hi John

Great paper, looking forward to seeing the comments.

My only suggestion at this stage would be to consider a supplier application fee (which I think is the correct model) but base the cost on their size (e.g. turnover / headcount) such that smaller startups (low turnover < $500k, <5/10 employees) would have one tier, a mid tier and a large tier for the larger businesses. This would mean you can operate within your cost-recovery model without overly disadvantaging businesses.

The concern I would have otherwise (and having had some experience with being on panels etc) is that startups would be put off registering due to the lower probability of getting return from a fee (even at $250), particularly for SaaS products that aren't overly expensive (in the $200-800/mo type market is what I'm basing this on) and might have lower margins, at least initially while they seek product/market fit.

Very strongly for the just-in-time insurance policy -- we've had to do the classic upgrade-insurance-to-pitch-but-not-win situation once, and it's excessively frustrating and really limits the capacity/desire for a smaller business to respond to an RFx.

Otherwise I would also encourage thinking about whether a fully transparent priced directory would suit our market, like CloudStore. This discourages suppliers from the practice of charging different rates to different departments/contracts, and also from charging government customers different rates to other customers (for better or worse).

Good work, looking forward to seeing the product.

Hi Hugh.

Thanks for your positive comments and for taking the time to provide feedback. We will take your comments into consideration when finalising the Request for Tender documentation.

Thanks,
Mundi Tomlinson

Hi John

Do you know when the draft panel contract will be available for comment?

Cheers

Peter

Hi John,

Thank you for a very thorough treatment of XaaS and associated services.

However can we suggest the following explicit capabilities also be included as Specialist Cloud Services:

1. Content Delivery Network (CDN) services, up to 30% of all global web traffic is carried on CDNs on any given day.

2. DNS and DNSsec services, auDA (.au Domain Administration) is currently contemplating DNSsec for the .au domain and I understand Finance is formulating a guide for Agencies regarding the .gov.au domain.

3. Distributed Denial of Service (DDOS) mitigation services, the threat landscape for these attacks grows daily in bandwidth and sophistication and they originate from almost anywhere on Earth.

These services could be specifically described as examples in ‘Appendix B – Statement of the Requirement’, page 31, Specialist Cloud Services, under dot-point 2 - Cloud integration and optimisation.

Regards

Dave

Not a mention of recordkeeping requirements anywhere.

Hi David,

Thanks for your feedback. We are always keen to hear the views of agencies and industry. We will take your suggestions into consideration when finalising the Request for Tender documentation.

Thanks,
Mundi

Hi Tony,

Thanks for your feedback on the Cloud Panel Discussion Paper. Recordkeeping is covered in the Head Agreement, which we recently released here:

http://www.finance.gov.au/blog/2014/08/08/draft-head-agreement-for-whole...

Does the Head Agreement cover what you would expect to see for recordkeeping? We would be interested to receive your further feedback.

Cheers,
Mundi

Hi,

On behalf of Delib Australia, I'd like to support Hugh's comments and also note that it would be valuable to have a consistent approach to communicating how a solution meets the ASD's ISM requirements for security and WCAG 2.0 AA for accessibility purposes.

In particular, in our experience every department has a different assessment process for ICT security, despite them all requiring the same information. Having a standard pro-forma providing this information as part of a response to the Cloud Panel would significantly reduce the red tape providers need to undertake when approached by different agencies for this information (in a different way each time).

I think there is a real opportunity here to dramatically cut the burden on both suppliers and government ICT teams, who often go back to suppliers several times for additional information they forgot to ask in the first instance - or have difficulty finding it within their own unique pro-forma response documents.

We've had several occasions where we had to resupply identical information in different formats to the same department to meet the individual format and layout requirements of different ICT sections within the same ICT branch.

Cheers,

Craig

Thanks Mundi. I had a look at your link. To be honest it's not exactly what I was expecting. But it could be that I'm simply missing the point about what this is really all about. To give you an idea of what I was thinking check what the NAA has to say about Federal goverment Cloud recordkeeping expctations here:

http://www.naa.gov.au/records-management/agency/secure-and-store/rm-and-...

Hi John and Mundi,

Good stuff re the consultative approach to development of this procurement arrangement.

Here are some points of feedback to ponder ... some trivial/symbolic and some substantive.

1. Cloud Services … not “cloud”

I am perhaps becoming a crank on this point, but governments and their agencies should stop using the single word “cloud”, or even worse referring to “the cloud”. These words are fluff. It is much better to ground statements in the reality of either “cloud computing” technologies, to be implemented, or “cloud services”, to be consumed.

The words “cloud” and “the cloud” encourage fluffy thinking ... commonly used as shorthand by the IT and media industries to refer to either cloud computing or cloud services. In government IT however these words create and perpetuate lazy thinking. For example, “cloud is untrustworthy for government information” or “everyone is moving to the cloud and so should we”.

Cloud computing technologies such as virtualization, automation and self-service provisioning are state-of-the-art IT. If agencies are committed to in-house IT then investment in cloud computing technologies is simply a logical and incremental extrapolation of the status quo.

Cloud services, in contrast, are state-of-the-art shared services and outsourcing. The cloud services model implies a step change in the way agencies define requirements, procure IT capacity and software, manage operations and fund IT activities. Cloud services adoption requires new mindsets and new skills for agencies to become intelligent consumers of commercially provided, arms-length, shared services that are proven to work.

Simplistic language that refers to “cloud” and moving to “the cloud” makes it too easy to muddle these distinctions and to suspend critical enquiry about benefits, costs and risks. Are we referring to technologies or services? To investments in the modernization of the IT department or of an agency’s policy and service innovation capabilities?

Grounding discussion in terms of the realities of either implementing cloud computing technologies or consuming cloud services encourages a more critical perspective. What technologies or services provided by whom? Consumed by whom? How? Under what terms and conditions? How trustworthy? With what trade-offs of benefits, costs and risks?

2. Panel value-adding logic

The experience of the UK government’s Cloud-Store should provide a salutary lesson for the need to decide upfront what the real value-add of such a panel arrangement is compared to agency-by-agency procurement. In what way is a panel participant more valuable to agencies when purchased via the panel vs. the open market?

• Signed up to standard government terms and conditions?
• Certified to comply with defined service quality standards?
• Certified to comply with defined information security standards/requirements?
• Certified to actually deliver the functionality specified on the label?
• More transparency via the panel around the experiences of other agencies with this cloud service?
• Priced at lower levels due to volume discounts?

My observation is that there is a dilemma between ‘too much’ and ‘not enough’ added value. If the panel provides a high degree of assurance (each participant is carefully vetted) then this takes a lot of time and effort and creates a bottleneck for cloud service adoption. There will also be type 1 and type II errors made. If there is only minimal assurance (anyone who applies is accepted) then what value is really added? The panel is simply a directory of providers. The early years of the UK Cloud Store were a ‘rich learning experience’ for a small team that was swamped by thousands of ‘cloud services … many of which were the proverbial ‘two chaps in a bedroom over a corner shop’ startups and SMEs (noble but not necessarily ‘enterprise grade’ solutions).

The UK Cloud-Store is, however, currently in its fifth iteration and has become an effective procurement vehicle as well as an instrument to drive vendors towards the provision of more rigorously defined, and ‘real’, cloud services.

I think ‘less is more’ when it comes to establishing a 'closed market' panel like this ... at least to get started. The Statement of Requirements and Service Evaluation Scenarios look way too detailed to me – too ambitious. It will take too much effort to do the evaluations and too many errors will be made … so what really is the point of it? What confidence will this really give agencies that a cloud service on the panel is ‘good’? Also, the service offerings are rapidly evolving, so the evaluation teams will be constantly chasing their tails and/or the evaluations will always be out of date. By definition, the fastest evolving (and probably the best) cloud services will probably be too difficult to include in the panel … or will choose not to participate because it is too much effort to continuously update their inclusion in the panel and they are quite capable of selling in the open market.

Better to focus on some key data points relating to TRUSTWORTHINESS and then let agencies make their own assessments with regard to functionality and fit to their requirements.

So, for CRM (as an example) what data points are really useful to an agency?
• Functionality – brief summary and link to the vendor’s website
• Some hard data on operational scale (number of customers globally, in Australia and in this jurisdiction)
• Pricing arrangements
• Verified quality certifications
• Verified security certifications
• Checklist of compliance of standard contract T&Cs to Australian government and good practice requirements (perhaps just on an exception basis – i.e. highlighting non-compliance)
• List of other agencies already using this cloud service and contact detailes/referees
• eBay style feedback from other customers/other agencies.

4. Agency-led evaluation/Peer Panel

One way to overcome the problem of an evaluation bottleneck is to make the panel more of a peer-peer/collaboration resource rather than a centralised procurement arrangement. This could be achieved by a federated approach where any cloud service purchased by an agency could also be included in the panel as well, so that other agencies could benefit and reuse/leverage the buying agency’s activity and procurement artifacts etc.

The problem with these kind of panels is that the evaluation activity is at best theoretical. It is not done for the purpose of actual immediate consumption of the service. When an agency buys a cloud service, on the other hand, it is a real procurement decision, with real requirements … real money … and real risks. The best way to do evaluations is to do them for real … so the panel should try and take advantage of, and leverage, actual cloud service procurement in agencies to encourage sharing, collaboration and reuse behaviors.

3. KPIs for the procurement panel

If the aim is to accelerate cloud services adoption by agencies then there should be some clearly stated KPIs for the panel to assess its effectiveness:

• % of know viable solutions in the market included in the panel (is the panel representative of the market) – if not why not?
• % of cloud services known to be consumed by agencies that are included in the panel (is the panel representative of current agency cloud service consumption) – if not why not?
• Stats on the volume of sales etc. through the panel
• Stats on the work effort/cost of administering the panel
• Stats of the evaluation throughput and backlog of panel applications.

5. Is the panel mandatory?

Must agencies buy off the panel if a cloud service is provided via the panel … or is it simply an optional arrangement? (apologies if I missed this in the doc). If it is mandatory then this puts a much bigger responsibility on the administration team to operate the panel to a high performce standard. If it is not well executed then it will probably be a barrier to the acceleration of cloud services adoption by agencies.

If it is not mandatory, then all parties can assess its value add and participate in it/buy from it based on the degree to which it is useful, reduces risks and saves effort and cost.

My view is that the panel should not be mandatory.

4. Panel participation fee

A participation fee is a good idea if the panel is not mandatory. Cloud services are, by the NIST definition, all about scale … so if a vendor is too small to pay the fee then they are either likely to be too small to actually be an enterprise-grade, scalable, cloud services provider or not seriously committed to the government market. ‘Cloudy is as cloud does’ etc.

If the panel is mandatory, however, then I don’t see how a fee could be charged as this is a closed monopoly market. Vendors would have no choice but to pay for a service … irrespective of whether or not the service delivered them any value ... which is not really a good dynamic. Would it really be defined as a tax?

Cheers,

Dr Steve Hodgkinson
Chief Analyst Global Public Sector & Research Director IT - Asia/Pacific
Ovum
[contact details snipped - JMS]

Hi Steve

Thanks for your comments. I'll respond to them in turn. 

1. I agree. You are being a crank. That said, we could have been more consistent in our language - this is definitely a 'cloud services' panel.

2. Panels work. There are over 150 open ICT panels at the moment. Together they have done some $5 billion worth of business. Some 640 odd vendors are represented on them. The average contract value is over $400,000 and the median is over $100,000. About 65% of the companies on panels are only on one and about 95% are on 5 or less. Panels can considerably reduce the time required to procure goods as services by streamlining responses and evaluations. I think this is pretty good evidence. 

Pre-agreed contracts and terms and conditions are the some of the strongest advantages of panels. The draft contract we've also posted for comments should make this clear. Also, in this case, we've proposed regular refreshes of the panel and the ability for vendors to add services between refreshes. I don't think that's very closed. 

I doubt the SME community shares your views about what they might be able to offer. One of the significant focii of the UK efforts has been to increase the involvement of SMEs in government procurement to something approaching our levels - >30% by value, 60% by volume. I'd be keen to see our SME readers comment on this view.

4. (Not sure what's going on with the numbering here) This is a philosophical argument against panels. I respect your view but the evidence above is weighted against it. We use mechanisms like regular best and final offer rounds, etc,, to deal with this issue. Each procurement still requires the agency delegate to exercise their judgement on value for money.

3. KPIs - You can find the contracts signed under any panel on AusTender (www.tenders.gov.au) and analyse them using the machine readable data on www.data.gov.au. I'm not sure how practical the 'known' solution % is. My experience is that one can only know what is practical by approaching the market. Keeping up with 'known' solutions would be a challenge in itself.

in the Data Centre as a Service Multi Use List, we conducted seven tranches of assessment over two years using the equivalent of two to three FTES. We're not going to do that much again. There wasn't any backlog. there won't be in this process either as we will do annual rounds after the first two years, assuming we proceed as planned.

5. It's not mandatory. The document says so at least twice - on page 4 and just above the discussion on fees.

4. N/A.

Thanks again for your thoughts.

Regards

John

 

 

Hi John and Mundi,

Many thanks for providing an opportunity to comment on your Cloud Procurement discussion paper.

I believe a point that is missing from the discussion paper, but a crucial factor for success of the Cloud Services Panel, is consideration as to how Government Staff will access the panel, how Panel Suppliers will get access to opportunities and be able submit proposals in a transparent manner, and how the panel will be managed (including reporting), AFTER it is established.

I also think it is worth reflecting on recommeondations from the 2014 Audit Report on “Establishment and Use of Multi-Use Lists” by the Australian National Audit Office.

Additionally, it is worth reflecting on best practice, and where panels are being managed well. As an example, I'd encourage you to look at the winner of the 2013 CIPSA Innovation Award - this was a Collaborative Procurement Innitiative focused on optimisatiopn of Panels by the National Procurement Network and VendorPanel - there is a Case Study here: http://www.magnetizedmarkets.com/uploads/5/6/5/4/5654140/case_study_loca...

I have provided more details in a formal submission. Thanks again and kind regards,

James Leathem, CEO Magnetized Markets

Hi Craig and Tony,

Thanks for your commentary regarding the Discussion Paper. We are keen to hear the views of industry and agencies, and will take your suggestions into consideration when finalising the Request for Tender documentation.

We appreciate you taking the time to provide your feedback.

Thanks,
Mundi

Comments for the Discussion Paper are closed. Feedback was very constructive, with over 34 detailed responses from agencies and industry. I would like to take this opportunity to thank all who participated. Please sign up for notifications on AusTender for the release of tender documentation in due course.

http://www.tenders.gov.au

Thanks,
Mundi

Last updated: 06 January 2015