Back to the Future: Another chance to influence COE development

Author: 
John Sheridan - CIO & CISO
The Department of Finance Archive

The content on this page and other Finance archive pages is provided to assist research and may contain references to activities or policies that have no current application. See the full archive disclaimer.

As Kayelle noted earlier in the week, AGIMO’s Common Operating Environment (COE) policy is now finalised and available on the web. Since it was released, there has been a range of discussions about aspects of the policy in posts and comments on several online news sites (e.g. here, here, here and here). Much has also been said on Twitter - mainly at #AGIMO. Support has varied - there have been positive views and negative comments. Several people suggested we hadn't been open about this - despite blog posts (here and here) and various presentations. As Andrea DiMaio from Gartner has pointed out, we really tried hard to get public comment. Since people seem to have more to say, we have reopened the discussion in the hope that constructive criticism will inform future versions of the policy.

Before reading new comments, please take a moment to view the background on the document standard issue.

What standard does AGIMO support?

The intent of the policy is to mandate a file format that fully supports the primary office productivity suites used within government agencies. Based on a survey conducted in 2010, a large number of agencies (representing the majority of the desktop fleet) have signalled their intention to move to either Office 2007 or 2010 as part of their next upgrade.

To support the capability of these office productivity suites, the Office Open XML format, based on the ECMA-376 1st edition, was chosen to provide the greatest level of compatibility. The Office 2007 format is based on the ECMA-376 1st edition and the Office 2010 default format is based on the ISO/IEC 29500 “transitional” standard. The ISO/IEC 29500 “transitional” standard is very close (in practical terms) to ECMA-376 first edition. Because of the similarity between the standards, files created in Office 2010 can be used without significant issues in Office 2007.

Other formats were considered, but after careful consideration and discussion with the agencies it was agreed that many existing documents would not be properly converted by these other formats.

Importantly the policy does not exclude other formats from being used but seeks to ensure that at a minimum one common format can be accessed on all Australian Government computers.

The following table shows a range of office suites and their compatibility with the various standards:

Office suites and standards compatibility Product Available Standards Capabilities Microsoft Office 2007 SP2 ECMA-376 1st Edition Read/Write   ODF Read/Write Microsoft Office 2010 ISO/IEC 29500 Transitional Read/Write   ISO/IEC 29500 Strict Read   ODF Read/Write Microsoft Wordpad 6.1 ISO/IEC 29500 Read/Write   ODF Read/Write SoftMaker Office 2010 ECMA-376 1st Edition Read/Write   ODF Read/Write OpenOffice 3.2 and above ECMA-376 1st Edition Read   ODF Read/Write iWork ECMA-376 1st Edition Import   ECMA-376 1st Edition Import TextEdit ODF Read/Write IBM Lotus Notes ECMA-376 1st Edition Import   ODF Read/Write WordPerfect ODF Read/Write Kingsoft Office ECMA-376 1st Edition Import   ODF Read/Write Google Docs ECMA-376 1st Edition Import   ODF Read/Write

Note: ODF refers to the ISO/IEC 26300:2006 Standard.

What’s the difference between the minimum standard of interoperability we are seeking to achieve between agencies and the public-facing publication of documents by agencies?

The use of common standards promotes interoperability, provides common functionality and supports a consistent user experience. These standards applied to all users regardless of how the desktop environment is delivered. By using the agreed common standards, agencies can share services and reduce the need for duplication. Agencies will be in a position to implement an application or service, which can then be reused by other government agencies. The COE will enable agencies to respond more quickly to changing technology cycles as it facilitates more cost-effective upgrades and supports a move to more rapid adoption cycles, enhancing agency and government agility.

The COE policy represents the set of minimum requirements that agencies are required to use. Agencies are able to make changes to meet their business requirements. Agency modifications must not decrease or weaken the level of IT security endorsed by the policy.

The COE policy only affects the manner in which documents are exchanged between agencies. The requirements for accessibility mean that documents available to the public through government websites, for example, are published in alternative formats to ensure the needs of citizens are met.

What did the SOE survey discover?

In April and May 2010, AGIMO conducted a survey of Government Agencies to collect information about their current desktop ICT environments. The survey identified more than 265,000 PC operating environments across Australian Government agencies. Of these, more than 99.5% are Windows based operating systems. MACOS, Solaris and Linux were also represented but each had less than 0.5% combined representation. With regard to office suites, MS Office is used on more than 86% of the PCs. IBM Lotus Symphony is used on just under 13%, with Corel WordPerfect, Apple Office 2004/2008 and Open Office each being used on less than 1% of the PCs. In the survey, agencies were also asked to identify the major components to which they were planning to upgrade. Windows 7, Office 2007 and Office 2010 were identified as major components. No other Office Productivity Suites were identified. The results of this survey highlighted that the majority of agencies are already using or planning to upgrade to the standards identified in the COE Policy.

Which agencies were involved in the consultation for the development of this policy? What other consultations took place?

There has been extensive consultation. CIOs, users and technicians were all asked for and provided feedback. In October 2009, as part of its decision on Whole-of-Government desktop coordinated procurement, the Government agreed to a recommendation of the Desktop Scoping Study, which identified the development of a Common Operating Environment as a critical element in driving future savings in services provisioning and in increasing the flexibility and responsiveness of government operations.

The draft Policy has been available for agency review since June 2010. As changes have been agreed, the document has been updated and agencies notified of the changes.

In mid-April 2010, more than 100 agency Chief Information Officers were contacted, provided with the background of the Project and asked to provide members for the Working Group and the COE collaboration site. All the Government Portfolios (representing all the FMA Act agencies) provided representation. Working Group meetings were held in June, July, September, October and November 2010. The Government’s Chief Information Officer Committee (CIOC) endorsed the draft COE policy in early December, with the Secretaries’ ICT Governance Board (SIGB) endorsing it on 21 December.

As discussed above, the draft policy was published here on the AGIMO Blog as a discussion paper.

The development of the Policy was also informed by a pilot activity which ran from July to October 2010. The pilot was based on a Windows 7 upgrade.

Where to from here?

The COE policy is subject to annual reviews. The first of these will commence in mid-2011. Public comments on this post will be used to inform the review. We have taken the liberty of moving one comment already received about this issue on an unrelated post to this new post to kick the discussion off.

Please note that while we will maintain our liberal post-moderation policy, a Microsoft vs OpenOffice mud-slinging competition will do little to progress the policy.

Regards

John

 

Comments on this blog are now closed. Please let us know if you would like to discuss this post or have any general comments.

Comments (65)

hey John,

I have to say, kudos to you and the team for re-opening this issue after the public feedback. AGIMO is doing a great job right now of driving the open government agenda -- more so than I have witnessed at any stage in the past. It's this kind of open discussion that will take Federal Government IT strategy forward in a really positive way.

Personally, I'm more interested in the browser strategy than in the document format. Microsoft Office's ubiquity is a reality in both business and government right now; the need to interoperate with many stakeholders ensures that.

However, probably the most common pain point I am getting from public sector workers right now is that they are locked into Internet Explorer -- and typically older versions of IE.

It's my view that AGIMO should be encouraging departments to proactively address this issue, with a view to driving staff satisfaction at work. What, after all, would it cost departments to install Firefox, at least, by default alongside IE? Virtually nothing.

Firefox is easily manageable by the IT department centrally, and if there are legacy browser issues, Internet Explorer can remain available. This simple change would fix so many of users' frustrations with their employing department.

Just my 2c -- I'll post a longer comment on this and other issues in reaction to your post on Delimiter on Monday. I can't stomache writing much more personally today -- I've already written too many articles today and my fingers hurt ;)

Cheers,

Renai LeMay
Publisher, Delimiter

John Sheridan I notice you comment about no feedback on the WofG COE. I simply ask for a simple test run MS Office 2010 against ECMA-376 standard. I am sorry to report.

There is not a single office buy able suite in existence that can do it include MS Office. You have asked for the impossible.

Last MS Office that could do true ECMA-376 was MS Office 2007. That is no longer directly buyable downgrade only. Also would be a secuirty risk.

MS Office 2010 is partly migrated from ECMA-376 to ISO-29500 with migration complete to be MS Office 2012. So making it incompatible because features of ECMA-376 are no more and will be fully no more with MS Office 2012.

Main reason you did not get feedback about these issues is no one knew you were doing it. Putting it on a blog somewhere and not advertising what you are doing is asking for it to be overlooked and for you to make fool out of yourself writing something into a standard that is impossible.

Hopefully there are no fines for anyone in breach of policy. You have set administrators up good and proper.

You would have been safer specifying ODF 1.1 that there are exporters for MS office and is supported by MS Office 2010. At least software exist that new machines can come with.

Other formats were considered, but after careful consideration and discussion with the agencies it was agreed that many existing documents would not be properly converted by these other formats.
Could you elaborate on specifically which other word-processor formats were considered and what conversion issues were identified?

Is there avenue to use the CoE as a means to drive adoption of free/open standards? You note above that more packages support import/export of ODF than support either ECMA 376 or ISO 29500.

Perhaps there is room to "strongly recommend" investing in options which support ODF, with a plan to migrate the CoE across to ODF as soon as it is widely and reliably enough supported.

Once ODF is at least mentioned in the CoE, there is scope for discussion of the problems that departments have with using that format, which then allows the developers involved in that project to address those concerns.

Without knowledge of the issues preventing adoption of the ODF standard as implemented in various packages, there is no scope to update the software implementing the standard to address those issues.

@ John Sheridan

John - i have previously posted previously here . I have come across to this site in line with your comment. I note that there are a range of comments being made all of the place, see for example here .

Again, I just want to commend you on your handling of the matter. Many people feel strongly on the issue, and have on some forums expressed themselves in ways that I do not think are helpful. You have not deviated from being open-minded and considerate, so thank you for that.

You have responded to my previous post in terms of market effects and demand. I have two comments.

Firstly, regarding demand for ODT. I would personally like the Government departments that I deal with to be able to receive and understand and author ODT documents. There are a number of posters in other forums expressing that view. I believe several large companies such as Oracle, IBM, and Google have expressed similar views. What more than this would be required to demonstrate market demand for ODF from AGIMO's point of view?

Secondly, and and contrary to my first point, I respectfully disagree with analysis on the basis of market demand.

If I understand you correctly (perhaps I don't?), the adoption of ECMA for AGIMO is a a way of standardizing what people are *currently* doing. However, AGIMO should approach standards from the point of view of the effect that standardization has on what people *plan* to do. In particular, please measure standards by their ability to allow news ways of exchange / market creation.

Say I'd like to interoperate with the Government, and I understand that the Government intends to use an implementation that currently sits somewhere between ECMA and the ISO standard. If my implementation doesn't match, the person receiving my document will think *I* messed up - not my software - not your software - me. To play it safe, I should pragmatically ignore the standard - EMCA, ISO or otherwise - and just buy the same implementation that the Goverment uses. Is that the message AGIMO means to send? If not, and you Microsoft is the best choice - the answer seems to be to ask them to give you a product that *is today* the standard - that avoids the problem of those seeking interoperability hitting a moving target.

Finally, I can see a lot of animosity toward certain companies in all of this debate. Nothing that I have written is intended to suggest anything one way or another about whose software is used. Rather, it is a comment on the effect of standards - anybody that implements the standard in a way that allows me to switch vendors and implement it myself seems like a fine choice. To that extent, I should say that I have been heavily influenced by the lack of available ISO/IEC 29500:2008 implementations listed on the page here . That bias stems from the essentially transitional nature of the ECMA standard. It also means that I would be grateful to learn if there are more implementations out there, because that would make me less attached to the choice of ODF in particular.

Have a great weekend.

The major flaw in the table above is the ECMA-376 1st Edition is actually incompatible with ISO/IEC 29500. As the COE specifies ECMA-376 Microsoft Office 2010 only has Read compatibility for the standard you are proposing, this is referenced at
http://technet.microsoft.com/en-us/library/cc179190.aspx.

It would be wise to test a number of office packages to ensure that they conform to the standard that you are proposing, because as I understand it no software writes documents that comply with the published standard.

(Incidently, this comment form does not submit using Firefox)

If you mandate that one format has to be usable on all computers, why make it the format that is only available from a single monopoly supplier, and one which contains undocumented binary fields which pretty much ensure the status quo?

Seems to me that Microsoft have done their marketing very well. I would presume that this process was corrupted by microsoft in the same way that they corrupted the ISO procedure in making their format a "standard"

I am ashamed of Standards Australia's involvement in the ISO issue, and AGIMO's blind acceptance of microsoft's press releases.

Perhaps you should announce that the Government is going with ODF. You might find that the Bill and Melinda Gates foundation is suddenly interested in donating toward the Qld. flood appeal...

At least you will have received some benefit for the selling of your soul.

Thank you for publishing a document related to whole of government ICT. It certainly is a great step forward in making different APS organisations more compatible with each other. After reading through the document I'd just thought I'd add some comments. Apologies for the length.

Comments
- The document, at least in my opinion, appears to be very Microsoft centric, either because the writers only had experience with Microsoft Windows or for some other reason. Examples of this are even shown in the above:
- MACOS should be Mac OS X
- The ECMA-376 format is only available and fully supported by Microsoft Office 2007 and was replaced by ISO/IEC 29500 which will be the format supported by other suppliers of office suites going forward.
- WordPad 6.1 should be listed as WordPad (Windows 7) to be more easily understood.
- Apple Office does not exist, iWork is provided by Apple and should be listed as iWork '09.
- Microsoft Office 2004/2008/2011 is provided by Microsoft and should be referenced as such.
- Office Web Apps should also be included as it supports the ISO/IEC 29500 standard.

And here are some concerns regarding some of the details mentioned:

Configuration Data :: Security configuration
- DSD and AGIMO have been slow in the past to endorse new configurations and usually only focus on Microsoft Windows

Configuration Data :: User Configuration
- "Platform-specific" guidelines may confict with details in the ISM and other ICT documents, which takes precedence?
- Links should be represented in URL format (file://) to be cross-platform compatible, this should be made clear and is supported on all operating systems
- A minimal interface currently only applies to Windows XP and above, not to Mac OS X or Linux
- Section f. conflicts with Section f. of the previous section

Standard Applications :: Multimedia Viewer
- No comments

Standard Applications :: File Compression Utility
- No comments

Standard Applications :: PDF Viewer
- No comments

Standard Applications :: Internet Browser
- Add-ins need to be clarified as either executable code (such as ActiveX or Mozilla plugins) or HTML (Safari and Chrome)

Standard Applications :: Email client
- Support for MAPI should be removed as this an Exchange Server protocol only.
- Instead the section should be re-written to require that secure connections are required for all connections, regardless of protocol.
- This allows TLS encryption which uses IMAP/POP protocols, not IMAPS/POPS but provides the same level of security.

Standard Applications :: Office Productivity Suite
- See comments above, but should be ISO/IEC 29500

Core Services :: VPN Client
- No comments

Core Services :: Antivirus
- Anti-virus should be part of a base image, not installed afterwards (this may be required as part of Windows XP deployments though)
- A hypervisor AV might not detect viruses in virtualised operating systems depending on security configurations
- All operating systems, regardless of virtualisation state, should have AV

Core Services :: Firewall
- This section should be revised with the inclusion of "signed-applications"

Core Services :: Desktop Management
- No comments

Core Services :: Message Classification
- AGIMO should work with messaging providers in order to provide this support for government agencies.
- No application fully complies with this requirement, with the closest support being provided by Microsoft Outlook and Apple Mail, both requiring plug-ins.

Core Services :: End Point Security
- These services are currently only available on Microsoft Windows, particularly Windows Vista and above,
other operating systems can be configured without the use of a client.

The System :: Encryption
- No comments

The System :: Codecs
- Codecs should be specified to comply with current standards, such as H.264 MPEG-4 and AAC (this then complies with the N-1 policy as specified elsewhere)

The System :: Application Frameworks
- What is an "Application Framework" in the terms of this document?
- Is this .NET, Python etc?

The System :: Storage
- This section should be classified to allow for remote users who may not have access to the Internet when performing their duties
- In the case above data must be saved to a local drive

The System :: Operating System
- I'm not aware of any operating systems that fully support the details of this entire policy without substantial investment in additional software
- A 64-bit operating system deployment should be required, not optional. 32-bit versions should be deployed ONLY to support legacy applications in virtualised environments

The System :: Network
- No comments

The System :: Hardware
- No comments

Other Sections
- Almost all application must download updates from the vendor's web site, not from a local source.
- The section regarding the sharing of virtualised applications should be removed, as it is a violation of many application's license agreements.
- The logging section is based exactly off the way Microsoft Windows Vista/7 store their logs, this should be generalised

Suggestions
A few things I think a lot of people would like to see:
- A list of operating systems that meet the requirements described in the document
- A list of office suites that meet the requirements described in the document

One final note, could AGIMO comment as to whether their deployment fully supports all the requirements of this document, as I doubt any APS organisation does now or will be able to without spending a signifigant amount of money and time, something which all ICT people can say is not easy to get.

John, thanks for publishing the table of available standards.

A cursory glance would seem to indicate that ODF is way ahead of ECMA-376, especially in terms of _write_ ability (only two products can write ECMA-376 according to your table).

So, given that all listed packages support ODF, why was ECMA-376 chosen for the WofG COE? Something about conversion issues for existing documents? Surely we should be looking forward and dealing with the issues of converting existing files as secondary? By putting the conversion issue as primary, you effectively lock in this issue forever.

OK so it's pretty clear from the table that with no legacy concerns ODF is the obvious choice. While there's something to be said for not letting historical cruft make new decisions for you that's not really practical in this case.

The PS has /lots/ of highly complex documents saved in MS formats, either legacy or current. Do ODF proponents really expect J{o,an}e Public Servant people to be happy about doing the post-processing work that you know in your heart of hearts they'll come across eventually? And yes there might be some of that same work required moving to an Open XML standard but let's be honest - those translators have had far more love than the ODF ones.

Let's not forget what this is; migration to ODF seems sensible but this is not a migration plan, this is not a fixed specification for desktops, this is not even a document of best practices, this is a Common Operating Environment. The goal of a COE is to make things Just Work when they move between different IT domains. With the huge pile of complex documents weighing down the decision, I'm sorry, I just don't see there was any other choice. In a few years when documents are happy in the ECMA form then the translation to ISO29500 or other standard is likely to be a realistic option but we just aren't there yet.

@John Sheridan

Qwote

"In the survey, agencies were also asked to identify the major components to which they were planning to upgrade. Windows 7, Office 2007 and Office 2010 were identified as major components. No other Office Productivity Suites were identified."

How many agencies said that they liked their current XP based SOE and weren't going to upgrade at all?

How many would prefer to keep using Office 2003?

What were their reasons for needing to upgrade these components?

I think a lot of people would love to know the answers to these questions.

For what reasons did they desire to upgrade those components?

What about earlier (non-standardised, but more widely supported) versions of Microsoft Word's format?

PS I've had a problem posting this comment as WordPress seems to think I've made the same comment earlier, though I haven't. The error message I'm getting is:

Duplicate comment detected; it looks as though you’ve already said that!

Hopefully this postscript will help to get past the faulty filter!

Three comments:

(i) you should never define a standard to be used by the operating environment in use, you should pick the standard and look for the operating environment to comply over time. Any large organisation that has already assumed an upgrade path rather than alternatives is doing so because of a lack of standards to comply with.

(ii) ECMA376 is not a near equivalent of ISO/IEC 26300:2006, it is the near equivalent of OASIS Open Document Format. More or less all modern office productivity packages can read/write ODF v1.2 either natively or with widely available plugins.

(iii) developing a COE is not about yesterday and today, it is about tomorrow, about strategy; a good strategy for govt should look to reduce costs for all stakeholders. Right now this one appears only to be a statement of what is being done and further prevents any use of alternatives by those who would be early adopters.

This is disappointing. I'd like to see a truly open document format such as "ODF" being used by the government and using my tax dollars.
I don't buy the argument that migration will be a pain. Migration from one office app to another of any persuasion is difficult. The costs saved from not paying for your office software will be more than enough to pay for the cost of migration.

But regarding browser choice. This is actually an even larger issue for the government than document standard. Having developed websites for the Defense department I know the pain and trouble website developers go through having to put up with ancient versions of Internet Explorer. It takes at least an extra 45% of your development time fixing IE bugs, on top of your normal estimated development time. This is time that the government will have to pay for. Firefox / Chrome are much better choices in terms of standards support.

As someone who spent months going over the 6000+ page "standard" that was submitted to Ecma and then ISO and writing a submission to Standards Australia only to see the dog and pony show that finally resulted in the "standard" being ratified because the 3000+ errata could be fixed as part of the development of the standard, this is a hugely disappointing announcement. Go searching for the status of the working group that was set up to handle fixing these errata, they've done nothing, not a single issue has been fixed! In fact, the chair of the working party is now expressing doubts that the "standard" should have been accepted.

It is obvious that whoever made the decision to standardise on this non-standard has not done even the slightest searching on what it really consists off. Let me tell you, a file that Microsoft Office 2007 saves, is not Ecma 376 and is not IS29500. Then there is the issue of legacy, verses transitional verses strict dialects.

Looks like you've also been sucked in by the "OOXML is the only format that is compatible with all the legacy documents out there" marketing line. This basically translates into "we've kept all the implementation bugs from every version of Microsoft Office and codified them into a published specification document". 1900 is a leap year anyone?

A modern *file format* should not have be bug-by-bug compatible with a legacy *file format*, only a converter needs to be able to load the legacy document and save it in a modern standardised format. There will be migration bugs no matter what new product is chosen, modern MS Office does not load legacy MS Office files flawlessly and there will be formatting and font problems anyway.

There is only one worthwhile office productivity document standard available that is current, well maintained and being actively developed and that is ODF. A "choice" of Ecma 376, or lets face it a "choice" of Microsoft Office 2007, will be the same as making the choice to standardise on IE6 ten years ago and will result in greater vendor lock and massive future legacy problems. Consider the cost of converting away from the chosen format *now* as part of the cost of getting into it, rather than the cost it will take to get away from it in the future.

For your consideration.

Why on earth would you not select the OpenDocument standard?:
http://en.wikipedia.org/wiki/OpenDocument

It is an XML format; it is an OASIS standard; it is an ISO/IEC standard (26300:2006 Open Document Format for Office Applications) and, most importantly, it is vendor neutral.

Have you read the FMA Act and Regulations? Have you read the Commonwealth Procurement Guidelines? Have you not heard of the phrase 'vendor lock in'?

Selecting a standard that is only implemented by one single vendor, and even then not properly, is completely inconsistent with the requirement for an APS purchasing decision to be an efficient, effective and ethical use of resources. By perpetuating vendor lock in to Microsoft, the Commonwealth deprives itself of choice and loses any chance whatsoever of any competitive tension in the market for operating system and office software.

Do your reseach - no other vendor even wants to implement the MS format. Just read the Wikipedia article and see how many vendors support the ODF standard. See how many other governments have selected the ODF standard. Australia's National Archives, who know a thing or two about maintaining documents, have chosen ODF as their preferred standard for archiving digital documents.

How on earth do the benefits of the MS standard outweigh the above? Even Microsoft Office supports the ODF standard! So you can continue to use MS Office if that's what you're concerned about.

You [moderated - foul language] inept bunch of mind [moderated - anti-discrimination] need to be [moderated - suggests violence]. You have set a new 'standard' in negligence.

How have you addressed the issue of archiving documents. Microsoft document formats including Ecma 376 and IS29500 are not suitable for archiving. ODF is suitable for archiving and is the standard used by the National Archives of Australia.

As an Irish Java Software Developer here for the last 7 years Australian Government "IT" is a bleedin' joke! It's clear Microsoft has paid nicely for this decision in its favour.

p.s. I use my own Macs for all contract work--any objections disappear, or I walk. Q.E.D.

I was under the belief that LibreOffice was supposed to have much better support for these OOXML-type formats than OpenOffice.org... why did you not test that? A stable release is due any day now, so there's no excuse for not investigating the release candidate's compatibility.

If the LibreOffice support for the required format* is sufficient, you should make Government departments aware that they can upgrade to a solution that doesn't involve licensing costs, and support can be had cheap as it's not limited to any particular vendor.

Help save us some of our tax dollars, and avoid vendor lock-in so we save more down the road.

*Note I too disagree it should be a required format.

H.264 could never ever get close to winning. Look at current web browser usage trends.

In less than 2 years, Firefox may very well be in front of IE (with Chrome not far behind). Firefox *cannot ever* support H.264, and yet it is the only true cross-platform browser out of all of those (unless you count Chromium as Chrome). Google was thinking long-term. The Australian Government is only thinking short-term.

If you really want a standard to always be supported, you basically have no choice but to go for a free software solution such as LibreOffice. Even if by chance you are in a position to negotiate a deal that will ensure the Office suite developer will never ever drop support for your preferred format, you cannot guarantee the company will always be around - and wait until you see the conversion problems you'll have then. You also cannot guarantee that you will always want to do business with that company. Free software breaks you away from these problems.

Lots of companies *are* using ODF right now.
http://opendocument.xml.org/deployments

I bet just about all of these are too:
http://www.documentfoundation.org/supporters/

Any chance of an HTML version of the Whole-of-Government Common Operating Environment Policy, to read online?

Only RTF and PDF versions are available at
http://www.finance.gov.au/e-government/strategy-and-governance/SIGB-communique-december-2010.html
* Whole-of-Government Common Operating Environment (WofG COE) Policy [RTF Document 3.24 MB]
* Whole-of-Government Common Operating Environment (WofG COE) Policy [PDF Document 622 KB]

Marghanita

Standard Applications - Office Productivity Suite
Other formats were considered, but after careful consideration and discussion with the agencies it was agreed that many existing documents would not be properly converted by these other formats.
I'm assuming this was the predominate factor against the selection of ODF as the standard format. There are a number of factors to consider with such a conclusion.

When is there a need for conversion?
Answer 1. The conversion is needed there is a need to communicate entity that can only read the destination format.

Going by the survey results that are expecting a majority to migrate to Microsoft Office 2007/2010 within 3 years. So within three years of existing government agency planning there is going to be strong ODF support.

Answer 2. There is a need for conversion when a document needs to be preserved beyond the life of the current format as described An Approach to the Preservation of Digital Records - Green Paper

So whatever incompatibles exist now with conversion also need to be considered for when agencies dispose of records to the National Archives. By delaying an ODF standard I'm expecting this will cause further hardship for the preservation of government records. The real trouble will be if the proprietary behaviours of the ECMA-367 contain important information that would prevent a researcher or government inquiry (or enquiry) of making sense of the document because the implementation of the proprietary read facility no longer exists.

What is the impact of prescribing ODF as COE standard now?

(as mentioned earlier) - An agency that deploys a new SOE that only supports ODF will have communicating documents to agencies that don't yet have ODF support. Within the next 3 years this could be significant.

What is the impact of prescribing read support for ECMA-376 now and read/write support ODF as COE standard now?

All agencies would have ECMA-367 as an interoperate format for the next three years. Agencies that upgrade to ECMA-376 read only support will have some interim difficulties communicating with agencies without ODF read capabilities. Early deployment of ODF capable software for those cases could be done.

At the end of the three years however there will be a common ODF read/write capability that will facility a much wider communication within government, to external governments, and to citizens that are using the vast range of ODF supported applications.

Is this possible?

Standard Applications - Email Client
Looking at the Email Client section of the COE policy it prescribes: Must support MAPI/SPOP/SIMAP protocols.

These are just the protocols used to communicate between the client and the server within an agency as I'm sure you know. There is no intra/inter-governmental operability issues regardless of what client-server protocol is used. I recommend removing these protocol descriptions (see later note about IMAP).

The real issue with email is making sure it is the same when it is sent to compared to when it is read. Like the Office Productivity Suite this is predominately a format issue. The format, Internet Message Format (RFC5322) should be prescribed here instead of the protocols. Luckily this is an open format and commonly supported. If S/MIME is required for whole of government perhaps prescribing RFC 3851 is prudent to get a baseline software support however key distribution aspects also need to be prescribed to the email client (e.g. LDAP lookup via prescribed schema).

If the goal was to prescribe a email client that could work with any server side upgrade then a protocol is useful. In this case however only a specification single protocol that supports folders would be useful. This includes IMAP and MAPI. Given MAPI is far more complex to implement restricting this policy to just IMAP and specifying this as "IMAP4rev1 (RFC3501)" with extensions RFC5738 (UTF-8) and RFC2595 TLS. POP3 is a fetch only protocol that conflicts with the COE requirement to The System - Storage a. Data should not be stored on the local systems which is also inconsistent with a. Must be able to work offline.

Note: Mentioned earlier was that migration was not really a goal of the COE however server IMAP support or an open format of email storage would assist in any migration and long term preservation.

d. Must support shared calendars and contacts
If inter-department shared calendars are required then a protocol for transport and format are required to make this a reality. Same with contacts. Was this another AGIMO project?

Thanks for opening the review John. Sorry to see it was needed and the feedback earlier was limited.

John Sheridan - AGIMO wrote 21 Jan 2011 3:35pm:

> ... AGIMO’s Common Operating Environment (COE) policy is now finalised and available on the web ...

But why isn't the COE published in OOXML format, as per the document? ;-)

More seriously, the COE looks good, apart from the tricky issue of which office format to use and a lack of Gov 2.0 formats.

National Archives of Australia developed its Xena open source archiving software> to convert documents to ODF format for electronic archiving. If the rest of the government generates its documents in OOXML, then everything will have to be converted to be archived. It would be a shame not to be able to use the same format for working documents and the archive.

However, what might be more significant is the lack of web standards in the COE. To make government processes more efficient and more transparent, as per Gov 2.0, there has to be less emphasis on creating large documents handed down from Canberra like the ten commandments and more on two way communication, as we are doing here.

An "Internet Browser" is included as a standard application in the COE, but there are no versions of web document formats mandated, such as HTML or CSS. Also there is no mention of ebooks.

In my view the use of office applications suites, be they producing OOXML or ODF, are of questionable value. These applications encourage authors to create poorly structured, hard to read documents which waste system resources and staff time.

The COE also specifies the ISO/IEC 32000-1:2008 version of PDF (equivalent to Adobe PDF 1.7). The "Australian Government’s study into the Accessibility of the Portable Document Format for people with a disability" recommended the continued use of alternate formats to PDF. In my view packaged web formats such as the ePub ebook format, would be preferable to PDF.

According to the table above only Microsoft Office 2007 SP2 and SoftMaker Office 2010 actually support the ECMA-376 or both read and writing. ODF however is supported by most listed in the table.

Your point about ODF support in Office 2003 is incorrect, there are free and open source converters that exist.

Considering the EOL for Microsoft Office 2007 is 10/04/2012 only 1 year from now. Why spend the thousands upgrading when you have to upgrade again a year later? Why not choose Google Docs, where the TCO is very minimal, you get free updates and supports ECMA-376 importing? Also Google Docs supports read and write to .doc, which is the defacto standard that the Australian Government already uses.

Simple fact close is not the same as. ECMA-376 1st and The ISO/IEC 29500 “transitional” standard are not the same. For the simple reason transitional is just that. MS is free to remove transitional features at any time.

Also your compare chart has a error a big error "OpenOffice 3.2 and above ECMA-376 1st Edition" If you had of used test suites you would have found out that. OpenOffice filters are closer to "ISO/IEC 29500 Strict" than ECMA-376 1st Edition.

Any not commonly used feature of ECMA-376 is not implemented in the OpenOffice implementation. So yes ECMA-376 is a trouble maker to all OpenOffice users.

"ISO/IEC 29500 Strict Read" On MS Office is wrong. Openoffice was the first to produce a ISO/IEC 29500 Strict write. This has been disabled from source base due to the fact nothing else can read it right yet. Yes transitional status also allows programs do not have to be able to read ISO/IEC 29500 Strict without bugs.

"Google Docs ECMA-376 1st Edition Import" Again did you test suite this. I see no. It is very much with the same problem as openoffice. Its not ECMA-376 1st Edition at all. But ISO/IEC 29500 Strict with ISO/IEC 29500 transitional implementation of commonly used parts.

ECMA-376 1st addition does not allow for missing features. Its all or nothing. I very much suspect if you truly do test-suite you will find basically nothing is ECMA-376 other than MS Office 2007. Saying use ECMA-376 is basically impossible and anyone with the test-suites will be able to show it.

"We know though that the actual Microsoft .docx format as written by Office 2010 can be read by Office 2007"
There is the problem. It is both directional. Office 2007 docx read by Office 2010 also loses content due to the differences between ECMA-376 1st and ISO/IEC 29500 “transitional” standard.

Worse part is any feature in the "transitional" standard but not in the main standard will keep on disappearing in fact if you read the standard documents is called transitional because its ment to disappear. At this stage MS Office cannot save as ISO/IEC 29500 Strict. Transitional if you read the standard means you don't have to completely implement it to have ISO/IEC 29500 Transitional.

So for long term documents. That you might want to open in 5 years time as they are today if saving today. Your only option is ODF 1.1.

The Office suit table compare is trash. Its not based on testsuites.
Just because feature lists say it supports something does not mean it works. Wordpad cannot open complex ISO/IEC 29500 Strict or Complex ODF 1.1 either.

Openoffice provides a free usable baseline to ODF. Libreoffice also provides a free usable baseline. This is particularly important if you are sending documents out to a person and they don't have MS Office they are free to install OpenOffice or Libreoffice todo business if ODF is used.

Libreoffice has something that call LOOXML Laugh OOXML what is basically implementing all the transitional features used. Its not implementing ECMA-367 ever. Mostly LOOXML is called Laugh OOXML because its mostly old doc format interpreter altered to see inside OOXML. So it will be as buggy as doc files moving between openoffice and msoffice now.

The issue here is not just what government uses. It what us public have to use because government departments demands. Are you going to give us free copies of MS Office as well as the 1 free copy of MS windows we get now from the government so we can interface with you.

Really this would be a interesting one. Go to Microsoft and ask how much for 1 copy of Office and 1 copy of Windows for every Australian citizen you should be able to get bulk rate discounts and report back what the cost would be. It should be cheaper then us public having to go buy a copy.

On everyone who does there own tax return we can claim a copy of MS Windows because this is the only way we can do our tax return. Really why? This is a waste of tax payers money and is purely caused by lack of platform support.

Basically times the cost of the COE per machine in software by the complete population of Australia because in a lot of places the public will be able to claim back items like MS Office and MS Windows if they are required to have them due to departments demanding documents been sent in formats that only those two can do effectively. Basically 1.2 billion+ since what you are describing could work out to 1000+ dollars per person. I personally don't want my tax dollars being wasted like this.

Very quickly making sure you have a Linux Live CD that can interface with government departments is a very good thing. Basically a public min interface standard. Must be made from all open source software that cannot be restricted in future. There are existing free virtual machines. Then at the worst to a Windows user you could say please run this virtual image. No tax office payout for require software. Linux Live cd are also highly virus resistant. Ie I could not submit something because my compuer was virus infected would not cut it anymore.

COE really should read this is internal requirements. But all external sent documents should be send-able in the following formats with ODF being one of them.

IBM Lotus Symphony by the way has OpenOffice source at its core. So it highly compatible with each other. Yes 13 to 14 percent compatible with everything.

"existing documents would not be properly converted by these other formats"

Sorry you have a issue were documents will not convert why. The format they were done in was a defacto standard. Going to another Defacto standard means you will suffer from the same problem again in under a year as the ISO/IEC 29500 Transitional starts disappearing.

Yes the existing formats will always be a bug bare. But as some point you have to chose not to keep on sending more and more money forward converting documents and move to a format that is a true standard. Basically please stop wasting my tax dollars. Spend a little to sort it out once and for all.

Linux Live systems are very good for secuirty. http://www.thinstation.org/ Problem again you don't buy this and you stated it had to be bought. This moves all processing to central server. Client machines don't require hard drives and as soon as they turned off there is nothing there.

Yes this fully does support only particular people being allowed to use the local drives. Big advantage thinstation could be used with machine that otherwise would be scraped.

Also if you read the documentation thinstation can operate transparent to operator. Ie they don't know the machine in front of them is a thinstation.

Then there is extra complexities. World has moved on. Will Motorola ATRIX phones with docking stations be permitted? They don't run windows. But they can dock and make a workstation. There are secuirty advantages to this. The dock contains no data. All there data is in the phone that should be on the person. Not left somewhere. Also the phone is installable with software like prey to track it down as well. Now Motorola ATRIX are an android running Linux at core with the means to run ODF compatible software simply. But not so simple to run the format you chose.

So for a large section of your current desktop by the end of the year could be void. So can large section of business desktop be void.

Finally no where covers what software should be used for document sharing. http://www.alfresco.com/ makes a prity good solution.

Notice something here. Again no costs. No purchase required. If you need or want tech support you pay.

Next advantage if you want to implement something in the open source frameworks for business to use to interface you can. So keeping there costs down as well.

If you have a COE there is no reason why you could not have a central tech support solution for all goverment departments who job includes creating any software any goverment department needs.

Advantage of this. If your systems are unique attacking them will be harder. Outsourcing to MS and others means you are using the same as everyone else with the same defects and will always have a high rate of infections and secuirty breaches.

The thing that I wonder most about the COE/SOE is why the need t mandate protocols and standards at all? Why not specify the desired outcomes and allow the agencies to sort out the details for themselves?

Eg.
* That all communications must be encrypted
* That the office productivity suite must produce documents that are interoperable with all other government departments
* That any software choice must be N or N-1 at most
* Require centralised configuration management and policy enforcement (this would allow for the use of tools such as Puppet)

Etc...

Hamrag

Overnight I had a few more thoughts about this...obviously these thoughts are long-term based, rather than something to be considered immediately, but as John Sheridan has expressed above, these comments will be taken on board for future versions...so...

Would AGMIO be looking at wikis (specifically expanding the use of their own govdex site) for intra- and inter-departmental document exchange and collaboration (this could also be applicable for interactions with the public too)?

This would bring many benefits, including no longer have to mandate an specific operating system and productivity suite, just that the web browser is standards compliant (probably HTML5 by the time this might be applicable). You'd also be able to gain back the expense of Windows and Office licences, a significant saving to government with 250,000+ desktops.

What do others think of this?

Hamrag

As a government employee, I'm glad to see that the committee has reopened this for discussion, but I am concerned that this happened only after the committee has made up it's mind.

1) It seems that the idea of supporting current implementations is good, but we've had that for over two decades - after all, it's how Microsoft managed to dominate the market in the first place. It leaves a lot of people out in the cold just because they aren't the majority, which kinda makes a joke of the idea of interoperability. "You can communicate with anyone ... as long as you use Microsoft." I'm annoyed by this stance, as it's made it impossible for me to submit eTax electronically for the last five years from my home Linux machines.

2) I am additionally concerned that the committee, having made a decision, now have a mental stance of defending that decision - despite new viewpoints being expressed.
http://en.wikipedia.org/wiki/System_justification
I'm hoping the committee are aware of this potential source of bias and are examining all arguments while keeping this psychological effect in mind, as I have been attempting to do while reading their arguments.

Paragraphs 27 and 28 of the COE policy may be worth considering in more detail.

Your "n-1" policy ties your vendor's release cycle to your own standards. Is this what you want?

For example, those complying with the policy will (by definition) introduce changes without your explicit consideration. Since you plan to revisit the policy, you may want to define a more restrictive change management policy. The alternative is that vendors can break your SOE by introducing an incompatible change and making a release. The breakage may not be evident except when you try to interop them - particularly since a fair bit of the policy is defined by application behaviour rather than standards.

Additionally, I see that the aim of the principle is to reduce the complexity and support requirement for the environment. Does it achieve this? It strikes me that at any given time, it will be fine to install the latest or second latest version of software. Over time, I'd imagine every release of anything remotely major will be installed - so really, the policy seems to amount to anything less than the n-2 version being removed to stay within policy. Is that right?
I mean - that would literally mean removing some older, stable system, and updating to something less testing, even though your requirements are stable. That seems odd given your aim.

Daniel Black's comments above are excellent in pointing out some of the things that aren't prescribed that probably should be, and things that are prescribed and should be. If capital 'S' standards aren't feasible for your deployment, could you perhaps specify a couple of example applications for which you've done some testing? That would to some extent reduce the uncertainty, because at least then it would be a bit clear what must have happened during testing (ie: "We've tested using Dovecot x.x and Exchange y.y using opportunistic TLS and the result was X").

Thanks for the information.

So the actual number is 99% of Government Agencies use Office.

Centrelink recently purchased Office and deployed it on all their PC's. Interestingly they did that (and spent X millions of dollars) without going to the market or conducting any review of the alternatives listed above. Further more they did it at the same time they deployed Lotus Notes 8 which included Symphony.

Somehow the FMA does not apply to purchasing Microsoft Products.

The reality is that the Government has chosen to standardise on the MS Office format because it had already decided to standardise on Microsoft Office. Very simple.

If "greatest level of compatibility" means means mandating a technically broken, obsolete, largely unsupported format that was twice rejected by ISO, then it is apparent AGIMO has no foresight. Perpetuating vendor lock-in when there are open source alternatives that do a much better job of adhering to widely accepted open standards is a grossly irresponsible use of my taxes.

The rest of the world is going ODF. 37 national governments have adopted as the sole standard for exchanging office documents. It is also mandatory within NATO and its 26 member countries.

If "the greatest level of compatibility" means mandating a broken, obsolete and largely unsupported standard that was twice rejected by ISO, it is clear that AGIMO has no foresight. Perpetuating a vendor monopoly is a grossly irresponsible misuse of my taxes.

The rest of the world is going ODF. This is the only truly open standard and is supported by many office productivity packages, including FREE ones.

37 national governments have adopted ODF. It is mandatory for NATO and its 26 member states.

Australia should also be looking further afield in its considerations.

NATO has adopted ODF for instance and whilst not a member we are considered allies.

Just a heads up that there is a world out there and not everyone is going the MS route.

The ECMA-376 version is so technically flawed that was soundly rejected by ISO.

1) It does not have specifications for dates before 1900 and incorrectly treats 1900 as a leap year. ECMA agreed that this was a flaw which was corrected in ISO/IEC 29500 to support the full range of ISO 8601 dates.

2) It includes compatability settings like "AutoSpaceLikeWord95" and "truncateFontHeightsLikeWP6" without defining the intended behaviour. ECMA agreed this this was a flaw and the behavour was fully defined in ISO/IEC 29500 without dependency on legacy products.

3) One of the largest sources of criticism was the inclusion of VML. Many ISO National Bodies commented that it should be removed completely. ECMA agreed with this. ISO/IEC 29500 notes that this is deprecated and included "for legacy reasons only".

Mandating "write" capability for a standard that even ECMA acknowledges is flawed will perpetuate the creation of office documents containing non-standard and deprecated features with dependencies on legacy products.

As a Taxpayer I don’t like the idea of my taxes being railroaded to one supplier when there is a perfectly good alternative for the 250,000 Government users. ODF is being adopted by many countries as the default standard. This will also allow the 250,000 Government users the option at any stage to transition from MS Office to an Open source alternative at a great saving to the taxpayer. To paraphrase FDR “you have nothing to fear but fear itself “, if you leave Microsoft as a (NON)standard. Remember MS haves lots of document standards for documents from the five flavours of .DOC to DOCx and new “open” XML. Pick a standard, yes one; MS have lots to choose from. Will they have a new Standard next year?

Thank you for re-opening this debate. I sincerely hope that it's not merely a gesture, and that the Government is taking our concerns seriously.

Is this file format choice about interoperability and compatibility? Or is it about ease of transition to something other than Microsoft's current proprietary office formats?

If it's about compatibility, then OOXML (in any flavour) is simply a bad choice. As your own table shows, ODF is far more widely supported and is a true free format.

The ECMA-376 and ISO 29500 transitional versions of OOXML support proprietary extensions, which means that they are not truly open anyway. Interoperability therefore suffers, as other office suites who implement the "standard" will not be able to read all of the Microsoft specific content. This in turn, will keep the Government locked to Microsoft's products (even though we would be using an "open standard"), without the freedom to move elsewhere if we so desired (which has been the situation for decades).

Groklaw covered much of this OOXML interoperability subject last December (http://ur1.ca/2n3c7).

If ease of transition is a top priority, then it makes sense that converting existing Microsoft proprietary office documents into a newer Microsoft format is accomplished more accurately using Microsoft technology. However, moving to a format which is not truly open is not actually solving the problem. Having said that, other office suites have decent migration support too, however Microsoft specific technology such as VB Macro support may cause headaches, but again, not fixing this properly just keeps us locked into a specific vendor.

The reason it's harder to migrate our current documents to an open format is because they were created in a proprietary file format in the first place. By mandating OOXML EMCA-376 (or ISO 29500 Transitional) the Government is simply perpetuating the problem. These formats are still tied heavily to Microsoft products and was one of the reasons that the format was flat-out rejected by ISO in the first place.

The Strict version of ISO 29500 (on which the ISO approval was based) is the more open format and if the Government is serious about long-term interoperability, then this should have been chosen above the outdated and soon-to-be-phased out ISO 29500 Transitional and ECMA-376 file formats. However, the fact that Microsoft, in the almost 3 years since the standard passed ISO, still has not implemented write support for the Strict version (as they were required to do) should be sounding alarm bells. Other office suites have, why not Microsoft?

Alex Brown, the convener of ISO's OOXML subcommittee, confirms that Microsoft is not meeting their obligations now that their standard has been approved. He wrote an article called "Microsoft fails the standards test" back in March last year (http://ur1.ca/swii).

By having the Government’s data in a truly free format such as ODF, we would free ourselves from being held at the mercy of a foreign corporation. Interoperability is actually possible, because any office program in the world is able to freely implement the file format and be able to successfully read all content.

When will the Australian Government set itself free? Freedom of the Government's information is far more important that any so-called short term "compatibility". Yes, migration of old documents might pose a challenge, but the bullet has to be bitten at some point, why not now? By continuing to support non-free formats, the Government only compounds the problem and makes it harder to tackle down the road. Long term, choosing a free format is the only way to guarantee compatibility and interoperability.

As Kim Holburn and Tom Worthington have pointed out, we the National Archives of Australia uses ODF to best ensure long term access to the Commonwealth's data. It makes sense that the Government should also be supporting this format from the outset, for the same reasons (and more). It's ISO approved, completely free, open, community driven, freely implement-able and royalty free. Not to mention, that as your table shows, ODF is by far the most compatible (and interoperable) of the lot already anyway. As a democracy, access to information is a right for the public, and this should be done in a freely accessible manner.

I sincerely hope that the Government re-visits this issue.

Chris Smart

The problem with backwards compatibility is that you are locked into archaic technology or standards forever. To continue this way of thinking is to continue using paper, because all existing documents are on paper and handwriting can't be read by computer. Or to keep using floppy disks, because they won't fit into a DVD drive.

We are now locked into a proprietary standard and we always will be, unless the decision is made to change.

Imagine how much money could be saved on licensing alone if the switch was made to Libre Office or Open Office. Software and templates could be customized to streamline workflows.

And once the decision is made, work could be undertaken to facilitate importation of non-compliant documents. Compatibility is not something that is fixed in stone.

Just to reiterate what others have said in order lend some weight: clearly for a universal cross-government standard, ODF is superior to the Microsoft standard owing to its read/write compatibility with a larger number of products. Moreover, the government should avoid being locked into a proprietory format, rather, open standards are a natural fit for government work.

Long term, it's likely that ODF is going to win this little war so, if the choice is between ECMA 376 and ODF then I'd have to say go with ODF.

Plug-ins are indeed available for Microsoft Office to support ODF and Microsoft are supporting both formats themselves. My only concern would be the ecosystem of products that can use these formats. Currently, OOXML is more widely supported in third-party applications and while ODF support may increase in the future this isn't currently the case.

One final point for people to consider; whilst Microsoft controls the OOXML format, Oracle and IBM control the ODF format......just something to bear in mind for the future.

I've been having a further look at the different versions of ODF and found a few interesting things to add to the debate.

Firstly, the only ODF versions to be passed by ISO/IEC is ISO/IEC 26300:2006, equating approximately to the 1.0 version of the standard released by OASIS. Future versions (1.0 and above) have not been submitted. This would mean that AGIMO would have to mandate the use of ODF 1.0 in order to meet the required standard. APS organisations could not use future versions until they were ratified by the ISO/IEC.

Also, APS organisations tend to use Tracked Changes a lot when sharing documents between APS organisations and others. The ODF format does not appear to work as well as it should; see http://blogs.msdn.com/b/dmahugh/archive/2009/05/13/tracked-changes.aspx

I checked the sample against LibreOffice 3.3 RC4 and Lotus Symphony 3 and can confirm that the issues identified are still the case. This would be of extreme concern, especially for research based organisations.

It should also be noted that I have not found any application that claims to be 100% in-line with the ODF spec, which is also annoying. Does anyone know of an app that is?

I agree that ODF is an excellent idea in principle and for a majority of users these issues aren't likely to be a problem, but if a standard is going to be chosen, it needs to meet the needs of all those it is likely to effect.

The fundamental difference between ODF (1.0 and later) and OOXML is this:
1. ODF is not the product of a single company. The process under which it is being developed is collaborative and not owned by one corporation. The goal of the standard, and proposed enhancements is real interoperability.
2. OOXML is the product of a single corporation. The process under which it is developed is unclear. The single corporation (Microsoft) have very clear (painfully obvious) commercial incentives to thwart practical interoperability while simultaneously representing themselves as 'pro Open Standards".

Ultimately, no software will provide 100% compatibility with a complex standard. That said, if the *motivation* and *incentives* are aligned with achieving practical interoperability, and there are *multiple viable implementations* the outlook is far better, and gaming by one member is far less likely. This is especially the case if one or more credible implementation is open source, and can be used by other vendors to provide insight into behaviour and help resolve ambiguities inherent in complex standards.

ODF is the only viable open standard option currently available.

I’d like to thank you all for the degree of involvement shown here. We had thought that the original post or the follow up post would have achieved this level of discussion. It didn’t but clearly there is a high degree of interest in at least one area of the COE Policy. In this comment, I will attempt to summarise the state of the debate as it stands and answer a few of the questions posed above.

We’ve all heard clearly that there are differing views on and various levels of support for OOXML and ODF (and the various standards that reflect those formats). Of course, the best format doesn’t necessarily win. Not all of our correspondents might be old enough to recall the VHS vs BetaMax video format wars. The market has a means of working these things out. Arguing about the merits here won’t necessarily influence that outcome.

The COE Policy does not attempt or pretend to determine that one standard is better than the other. Rather, it seeks to ensure that government agencies converge to ensure interoperability is increased. In the first iteration of the policy, ECMA-376 Ed 1 was preferred. That may change in future iterations. Comments on this post will help inform those considerations. I note that even the president of Linux Australia, John Ferlito, doesn’t seem too concerned (thanks @lukehopewell).

Given the large number of comments on the post now, not all readers may have noticed that this policy does not affect how documents are shared with the public on government websites. It also doesn’t require immediate change. It provides direction as to how agencies should develop their next standard operating environment (SOE). It was deliberately not vendor-specific. The government’s coordinated procurement arrangements generally direct how agencies are to procure various ICT goods and services but not what they should procure. Specifically, Microsoft is not mandated for anything.

Office software doesn’t stand alone. Agencies use it in combination with collaborative software, content management, financial applications, etc. Changing the office suite in an agency is not a simple matter and the licence fees for software are a relatively small portion of this cost. There are also the indirect costs of change to be considered and the subsequent effects on agency business procedures. Of course, this is true for all major enterprises. Decisions to change such systems are not made lightly and not without a strong business case.

So where does this leave us all. The COE Policy annual review will commence in mid-2011. These and other comments made in the future will inform the review. We’ll keep this post open for comments for a while yet. Depending on interest and the level of comment, we’ll eventually close it but there will be other options. We find that if we keep a post open for too long, it becomes an obvious target for spammers.

Finally, a few words on Gov2 and interactions like these. Like any conversation, rational, well-expressed and logical comments are preferred. Emotion-laden comments or unreasonable diatribes lower the value of discussion. We’ve maintained our normal, relaxed post-moderation regime here. We have had to moderate two comments – one you can see above and another yet to be finalised.

Of particular concern are comments that, without evidence, allege conspiracy or corruption in these matters. If correspondents use these comment opportunities to attack public servants, unfairly or personally, or to elevate issues above the status they deserve in inflammatory ways, then it is likely that APS members will be reluctant to engage. None of us wants this.

Regards

John

As an APS and someone who does not have a strong viewpoint one way or the other on the COE I thought I would make two points:

1. I am impressed by the preparedness of AGIMO to engage positively and assertively with its critics. I think it is good to dispute some of the knee jerk 'conspiracy' in the OSS community.

2. I am impressed with the social media engagement of AGIMO. I only wish that the (large) agency I worked for was this forward thinking.

Keep up the good work.

Tom

The reactions from the people who use Free and Open Source software are the result of long observation of how Microsoft has manipulated the document standards process and how many organisations appear to consider non-proprietary alternatives to Microsoft products in order to obtain further discounts on Microsoft products. Governments should not be getting themselves locked in to using proprietary products, instead they should be internally and externally be storing and communicating data via formats and protocols that are specifically not patent encumbered (that is, the formats and protocols used should be able to implemented without paying royalties).

First of all, thank you John for taking the time to engage with the public in this way. Although I suspect we do not agree on every point, I admire your willingness to engage and I sincerely ask any critical reader of this body of comments would view John's role for what it is - a positive and progressive one. In particular, to ignore the baseless, aggressive and frankly, unfair ones.

I look forward to reading further comments in the forum.

Perhaps the Government could choose a standard which they can validate (and warrant) that their documents are stored in, and test against various implementations? They could audit their documents to ensure that they comply to whatever standard is chosen. Can anybody here recommend a robust testing regime for documents?

John - reflecting more on the COE generally- I am starting to wonder whether there is anything standardized for inter-department authentication. Specifically, do you issue an employee in a first department a way to prove their identity in a second department without a separate provisioning them there? I am thinking something along the lines of a personal certificate signed by a trusted CA.

IMO, AGIMO is jumping to a standard document format too early.
As shown by the number of comments and heat in other places, there isn't currently an accepted Industry Standard.

7 out of 8 FedGov desktops run MS-Office, the best you can do is to select a default file-format that:
* is readable by all current SOE versions of MS-Office, and
* is readable and writable by all/most the other desktop products used

Without knowing the detail of all the other products, this might well be Office 2004 or 2007 format.

If the goal is "simply", 'whatever one person writes/sends outside their own agency, including being widely published, is easily readable', then there is already a complete, sufficient and ubiquitous solution: PDF.

Internal documents should be simply formatted, perhaps with diagrams and illustrations. Nobody should be creating novels or
documents that require sophisticated typesetting. Interoperability problems arise only for "complex documents". Are these necessary for day-to-day tasks?

If you want complex typesetting, then MS-Office is not the best tool for the job. Again there is a free, complete and sufficient solution: LaTEX.

If Academics use it to write papers on Genomics and Mathematicians and Physics use it to write books/papers/etc, it is definitively "the bees knees". It's multi-platform as well and produces PDF for free. And the documents are simple text, suitable for versioning and digital archive. What's not to like :-)

The biggest blunder AGIMO can make is to repeat the GOSIP debacle. The USA started this programme around 1988, published FIPS 146-2 in 1995 and abandoned it early 1998 with a new policy "adopt Industry Standards where used".

Perhaps the most salutary lesson for AGIMO would be the near collapse of IBM in 1989/90.

Computing companies don't last forever, but COBOL and ASCII will. [And now 'C', UTF-8 and text-only XML.]

Prior to declaring the biggest corporate loss in US history, IBM recorded record profits each and every year. It's share price reflected its dominance of the Industry and this strong management and profit performance. Which all blew apart in under 12 months.

Is it prudent for AGIMO to tie the Australian Government to a vendor who's performance for the last decade is not even average, but well below the rest of the market? The agency and its officers need to establish a reputation for foresight and fine judgement. This will be wrecked if Microsoft stumbles and is no longer "flavour of the day".

Until there is a clear Industry Standard, defined as "every vendor reads and writes it without error", like email and SMTP/MIME, then I believe AGIMO should hold off.

The line "The COE policy only affects the manner in which documents are exchanged between agencies." gave me pause.

John Sheridan's comment that "document exchange" specifically excludes "documents .. shared with the public on government websites." gives me heart.

My problem is:
spreading writable copies breaks the "One True Version" rule.

Outside the original author(s) nobody should have read/write access to any document.
"exchanging documents" is something we did with floppies in 1985 before LAN's, the Internet and ubiquitous email, not only because it was simple, but it was all we could do.

The current definition, for me, conflates many problems with conflicting requirements:
a. intra-agency (works by definition, read/write)
b. inter-agency (read-only, good representation)
c. external publishing (read-only, universal formats)
d. collaborative authoring (MS-Office unsuitable)
e. complex and large document authoring (MS-Office sub-optimal)
f. Archives (read-only, perfect representation)

Perhaps if you recast the COE policy question a little more finely, as you've done with external publishing, it might provide a way to cut this Gordian Knot.

There is every reason for Agencies to save working documents in native formats, whatever they are.
Complex and large "documents" are a problem unto themselves, like web-publishing, and should be separately considered.

As is the intermediate formats of collaborative works.
The final published document should be non-editable and a perfect representation... I've no idea if OOXML or ODF supports collaboration, which implies multiple access version control.

My first thought is a policy which states:
"editable documents are not to be sent to non-authors nor published."

I haven't fully considered this, others may be able to knock it into shape or demonstrate why it's wrong.

BTW: the Open Source world 20 years ago definitively solved the "multiple access version control" problem for text files. eg. program source and document formats like LaTEX and DocBook.

I find the various proprietary attempts at "Collaboration Suites" that ignore this work to be amusing and more than a little sad...

There are millions of lines of source code under the control of CVS, SVN and GIT.

Every line is traceable, full audit trails of all changes, automatic change merge with conflict resolution.

It works with documents as well.
Tom Limoncelli and Christine Hogan co-authored an entire book using a hosted server, CVS and LaTex. The team included a copy editor and a specialist graphic design team (figures/diagrams in LaTex)

Hogan and Limoncelli were introduced at a conference and within weeks had started their collaboration - working on different sides of the country. The systems, editing and typesetting tools they each used are irrelevant.

Their publisher was so impressed at their work they asked how they'd done it and could they help others. This wasn't some trivial work, but they nailed it effortlessly.

Use the right tools, things can get very simple...
Simple is Robust, Flexible, Extensible - and cheap and very, very fast.

The "secret sauce" was a basic Unix dictum: everything is text file.

Binary formats are the cause of this current confusion.

A quick comment regarding migration and licensing costs:

I feel strongly that the argument that ongoing license fees (to Microsoft) are lower than the cost of migrating to a non-Microsoft solution (training, reformatting, etc) is a red herring. Two points:

1) Migration costs are sunk into the Australian economy, Microsoft license fees go overseas

2) Migration is a one-off. Microsoft license fees are ongoing (and will be set as high as Microsoft can get away with).

Pages

Last updated: 28 July 2016