Why so low data.gov.au?

John Sheridan -...

One of the most common questions we have got since the data.gov.au upgrade to CKAN is about why the numbers of datasets have dropped from 1200 to 500. It is a fair question, and one we really should have addressed up front.

In the first instance, the migration meant a review of the 1200 datasets. We unfortunately found that a third of the “datasets” were just links to webpages or files that either didn’t exist anymore, or redirected somewhere not useful to genuine seekers of data. In the second instance, the original 1200 number included each individual file. On the new platform, a dataset may have multiple files. In one case we have a dataset with 200 individual files where before it was counted as 200 datasets.

Most recent statistics

Number of contributing government organisations   123
Number of datasets    529
Number of individual files 1610 Data Resources
User accounts            16 Organisational Admins, 350 registered users

Number of new and changed datasets in last month:

Month   Activity Type Count
2013-10 Changed Organisation 3
2013-10 Changed Package 154
2013-10 Changed User 13
2013-10 Deleted Package 1
2013-10 New Organization 1
2013-10 New Package 2
2013-10 New Users 45
2013-11 Changed Organisation 43
2013-11 Changed Package 18
2013-11 Changed User 1
2013-11 New User 14

Finally, due to the excellent work happening in the State and Territory Governments, several of the State/Territory publishers from the old site have moved their data to new state and territory portals. This is quite exciting and we are working closely with the new portals to look at search federation options so hopefully, you will then be able to search any individual portal to find data across them all. More on that soon!

But the numbers are still too low!

Regardless of whether it is 1200 or 500, we have a lot of work to do. A lot of momentum has built around open data in Australia, including within governments around the country and we are pleased to report that a growing number of Federal agencies are looking at how they can better publish data to be more efficient, improve policy development and analysis, deliver mobile services and support greater transparency and public innovation. The Australian Government has a lot of fantastic data sets that many agencies are working hard to make public.

We need your help!

Today we launch the data.gov.au Request Site at http://datagovau.ideascale.com/ where you can publicly submit your government data requests, vote up ones you think are important, and add your comments. We will ensure agencies are alerted to dataset requests they are custodians for, and then we can also publicly resolve requests as we go so you can see how we are doing.

We look forward to growing data.gov.au, will continue to work with other jurisdictions on open data initiatives, and look forward to collaborating with you to get more value from opening up government data.

John Sheridan
Australian Government CTO

Comments (2)

Hi John,

I appreciate the work being done to grow data.gov.au, but do not understand how the Department of Finance overlooked Australian providers of ideation tools and selected the US company IdeaScale to provide the ideation system.

As a provider of ideation tools in Australia, an Australian company that employs Australians, pays Australian taxes, complies with all Australian legislation, meets Australian Government security requirements (including having undergone a DSD review), meets the Australian Government's public/private cloud requirements and actively supports and advocates for Government 2.0 in Australia, I'd like to understand your decision to use a US service, which meets none of these requirements.

Is there something that Australian providers are not doing right to not be considered by your Department?


Craig Thomler

Managing Director

Delib Australia

Hi Craig,

Thanks for your comment. I was the delegate for this matter. As the delegate, I am required to satisfy myself that the procurement represented value for money. I did so and remain satisfied that this is the case.

While I am not required to explain my decision, I note that there are no security requirements at all in this matter. Considerations regarding where a company pays tax, who its beneficial owners are, etc, were also not relevant in this case.



Add new comment


  • Allowed HTML tags: <a> <strong> <em>
  • Lines and paragraphs break automatically.
  • Web page addresses and e-mail addresses turn into links automatically.

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.