Draft Roadmap for data.gov.au

Author: 
Pia Waugh
Category: 

The Department of Finance Archive

The content on this page and other Finance archive pages is provided to assist research and may contain references to activities or policies that have no current application. See the full archive disclaimer.

UPDATE: data.gov.au went live on CKAN with details at http://agict.gov.au/blog/2013/07/17/new-datagovau-%E2%80%93-now-live-ckan

As part of my role as Director for Gov 2.0 in the Technology and Procurement Division of Finance (under John Sheridan), I have responsibility for enhancing data.gov.au. This is quite exciting and an excellent opportunity to contribute to opening government data in Australia. We plan to make some significant progress in this area and would love your feedback on the draft roadmap below. We plan to move data.gov.au to an open data platform for all Agencies to use, and prioritise technical support initially for a few keen agencies who want to have their data accessible by API for policy analysis, improved applications development and data visualisation purposes. Then we can scale support for additional agencies as skills and capacity are built up and add new features and functionality over time. We will take a consultative, iterative and collaborative approach. We are running a number of information and discussion sessions for agencies and the public on Friday 19 April: UPDATE: We have shuffled the sessions around a little to accomodate more participants across Australia and to include a public briefing.

  • Agency briefing (0900-1100): If you are an Agency interested in hearing more, we will have a data.gov.au Agency Briefing in Canberra from 9am-11pm on 19 April which includes a telepresence option for all capital cities.
  • Public brieing (1130-1230): for anyone in the public to hear about and ask questions about data.gov.au. Also available via telepresence for all capital cities.
  • Developer briefing (1300-1500): If you are interested in hearing more about the tech and contributing your thoughts, please come to our data.gov.au Developer Session. Also available via telepresence for all capital cities. We’ll publish the technical documentation here on the blog for comment too, but this is a chance to talk tech with other open data geeks, data custodians and developers, get into the gritty details of the data.gov.au implementation and maybe even look at how we can improve the code for Australia. If there is sufficient interest we might live stream and record this for others to tune in and comment on.

All briefings will include a little information about GovHack, the upcoming national event where developers compete on clever ways to use government data for improving services, transparency, analysis and more. Please register your interest for any of the sessions above at http://datagovauroadmap13.eventbrite.com/. Please note the venue(s) to be confirmed pending registration numbers.

Draft Roadmap

Phase 0: consultation – April

  • We are continuing consulting with existing data.gov.au publishers about the plan and to assist them in transitioning skills to use the new platform and take advantage of the additional functionality.
  • We are talking to all the open data people we can find throughout federal, state/territory and local governments as well as our colleagues in New Zealand, building a community of government implementers to share knowledge, build expertise and collaborate where possible.
  • We are running a developer day 19 April for developers and technical people in the public, industry and government to come together to look at the technical details of what we are planning. We would be looking for technical feedback, feature requests and any specific tweaks the community can make to the platform to make it better serve the needs of data publishers and users.
  • We are asking for feedback from the Australian Government Linked Data Working Group, the Office for Spatial Policy, the Office of the Australian Information Commissioner, the Office of the Australian Government CIO, the APS Policy Visualisation Network and some of the Archives and Records agencies for suggestions to tweak the platform for improved data publishing and integration with other data sources.

Phase 1: data.gov.au platform – by end of April

  • data.gov.au is being moved to the CKAN platform, which is used notably and very successfully by the UK Government to publish data and enable better public engagement with government. Finance has had discussions with the data.gov.uk team, as well as with open data experts around the world to ensure the data.gov.au refresh is based on global best practice, with a specific focus on the best quality data publishing possible. We will leverage the work of the UK team including the additional plugins they’ve developed. Thanks Toni and the data.gov.uk team! Also thanks to all the Australian Federal/State/Territory people who are working in this space for sharing your knowledge and experience!
  • We will migrate all data from the current data.gov.au to the CKAN instance.

Phase 2: working with partnering agencies – over 12 months

  • We will work with a small number of interested agencies to get their data published and automatically updating where possible.
  • Additional agencies will be encouraged to use the platform, and to become partners of the initiative to get access to support services.
  • Finance will support public hackathon events to encourage use of data.gov.au datasets, including GovHack 2013

Phase 3: improved reporting – 3-6 months

  • We will put in place customised reporting of progress that takes into account useful metrics such as API access, hours saved in automated publishing per dataset, applications developed (where possible to find), and other stuff on which we will consult publicly.

Phase 4: data visualisation – 6-9 months

  • We are considering putting in place some front end data visualisation services for people to mashup, visualise, analyse and interact with government datasets. We are considering a number of analysis and visualisation toolkits and would appreciate feedback from the community.

Phase 5: government apps register – 9-12 months

  • We are considering enhancing the australia.gov.au mobile and apps page so that agencies and the public can more easily register mobiles applications and responsive web projects. In this way we can differentiate between government and non-government initiatives, people can rate and comment on mobile initiatives based on government data or services, and we can have a single place where people can find how to access government one their mobile devices. Something functionally similar to http://apps.usa.gov/

If you are interested in information about the CKAN API and other technical details, please see the CKAN documentation. Thanks all, and looking forward to reading your thoughts in the comments below, or feel free to get in touch with me pia.waugh at finance.gov.au. We’ll try to report our progress on a monthly basis so you can see how we are going. Pia Waugh Director of Coordination and Gov 2.0 Technology and Procurement Division

Comments (6)

In terms of dataviz, and aside from the CKAN2 "Refine" data browser, there is also DataWrapper http://datawrapper.de/ which can provide a platform to combine datasets and publish the result as a graph. The work in this space should be focused on providing tools for a wider audience than statisticians though! http://gigaom.com/2013/04/07/we-need-a-data-democracy-not-a-benevolent-d... http://gigaom.com/2012/12/22/we-dont-need-more-data-scientists-just-simp...

Perhaps this phase of functionality is better described as "data exploration". For example, for any geographic polygon dataset a tool could let you put in an address/postcode and instantly see what data is there without downloading the data or having any GIS tools/experience.

CKAN includes an Apps repository plugin eg. http://data.gov.uk/apps This could be extended to differentiate between community and offical apps.

Excellent post and thanks for the update. I am interested to see how the CKAN migration and API access will fit into the upcoming GovHack. With a consistent and continually updated data stream (APIs) I see more longevity in the potential GovHack applications.

Cheers,
Zane

Hey! What's this "under" John Sheridan? "with" pleease. John's going through what every NREN network manager has been learning for a decade = network architecture is transitioning from client/server to collaborator/cloud. He still hasn't addressed BYOD, BYOA(pp)and BYOC(content); as he says on his blog. But that's OK. His dutch equivilent could probably show him a thing or two.

Would you keep in mind that I'm trying to herd cats too; just between countries. I wouldn't mind involving .gov/edu.au in this one.

Looking at the platform (good choice) You've got it all in your head. I ended up here, trying figure out, as usual, how to aggregate conversations of working groups/task forces in different domains/archives. The hard part,from the comms side, is simply (ho, ho) changing the 'delivery' culture to be more inclusive. E.g. That cone-of-silence you're using for the agency brief could, with a few tweaks, be your TV station, and "connected classroom" for developers, et al. Just plug in some Google hangouts. stream and record. (I've mention it to them)

I'm trying, like you, to get the silos to BE SEEN to be collaborating. That's quite impossible until we have the 'higher ups' in gov.au agree on a strategy, which i can't find anywhere in gov.au,
So if you've got anything like this could you hang it on the govspace directory or maybe, considering you're taking the UK lead here, adopt/edit this one. Also NB.

P.S. Re: the courses. A quick overview, into to tools and join the community is the easiest way to go. People might not understand what you're doing. But they'll learn by imitation. I need ten Pia clones please:)

Pia

In your presentation yesterday at the Intersect workshop on "Sharing Government Data: the Role of the Research Sector ..." in Sydney, you showed a slide of the draft Road Map. Is there any chance you can share it with us, or even just me?

Bruce

This is good to see.

Please also note the Open Data Site Generations work I've done in my blog eGovAU. It might help influence thinking or culture change.

Cheers,

Craig

Pia

This is great news. I think the success of the new CKAN platform will be influenced by how much data you can get from other agencies to be automatically loaded and updated into the CKAN DataStore (as opposed to sitting in a un-useful csv blob, with timely updates dependent on constant vigilance).

Just out of curiosity, do you guys have any plans to mirror ABS data in your CKAN instance (using the ABS SDMX api to pull the data)? Could be a nice test case and might make the data more accessible due to the simplicity of CKAN's API compared to the mind boggling complexity of SDMX (at least for part-time hobbyists like myself!).

Mark

Last updated: 08 August 2016