W3C home > Mailing lists > Public > public-egov-ig@w3.org > May 2013

Re: New open source catalog and list of APIs on Data.gov

From: Josh Tauberer <tauberer@govtrack.us>
Date: Thu, 23 May 2013 11:31:02 -0400
Message-ID: <519E3636.60402@govtrack.us>
To: Brand Niemann <bniemann@cox.net>
CC: "'Holm, Jeanne M \(1760\)'" <jeanne.m.holm@jpl.nasa.gov>, 'eGov W3C' <public-egov-ig@w3.org>
Brand,

Since the new site is based on CKAN, you can read the CKAN API 
documentation yourself to figure out how to download the data catalog:

http://docs.ckan.org/en/ckan-2.0/api.html

The API is running really slow right now (probably all of us checking 
out the new site), so I'd suggest *not* hitting the API link below right 
now, but I've mirrored it so you can see the output quickly:

https://catalog.data.gov/api/3/action/package_list
   => Produces a JSON list of entries in the catalog
   => Here's a static mirror: 
http://razor.occams.info/files/catalog_data_gov_api_action_package_list.json

The package_search API function returns complete metadata rather than 
just a list of package names. It requires an HTTP POST, so here's an 
example of that:

     curl -d '{"rows":1, "start": 0}' 
https://catalog.data.gov/api/3/action/package_search
     => static mirror: 
http://razor.occams.info/files/catalog_data_gov_api_action_package_search.json

- Josh Tauberer (@JoshData)

http://razor.occams.info

On 05/23/2013 10:34 AM, Brand Niemann wrote:
>
> Thanks, Jeanne. Please send me the link to download the entire and new 
> data catalog.
>
> I am getting ready for: On Thursday, May 23^rd at 2:00PM ET, Federal 
> Chief Information Officer Steven VanRoekel will host a conference call 
> to discuss the 1-yr Anniversary of theDigital Government Strategy 
> <http://whitehouse.gov/digitalgov> and ongoing Administration efforts 
> to drive innovation through open data and other initiatives.
>
> Brand
>
> *From:*Holm, Jeanne M (1760) [mailto:jeanne.m.holm@jpl.nasa.gov]
> *Sent:* Thursday, May 23, 2013 10:10 AM
> *To:* Brand Niemann; 'eGov W3C'
> *Subject:* Re: New open source catalog and list of APIs on Data.gov
>
> Brand--
>
> As John pointed out, the entire and new catalog (as of this morning) 
> is available at https://catalog.data.gov/dataset 
> <http://catalog.data.gov>
>
> The catalog at 
> https://explore.data.gov/Other/Data-gov-Catalog/pyv4-fkgv is our 
> previous catalog of "raw" data that is not part of the larger catalog 
> above.
>
> The number differs from our earlier number of datasets:  The total 
> number of datasets reflects datasets plus data series. A data series 
> may contain a large number of additional products or files of the same 
> type. The previous Data.gov catalog counted the individual datasets 
> and not just the series and therefore had a higher total number of 
> datasets noted. The change in number does not reflect a change in the 
> size of the catalog, but rather a different structure of data products 
> in the new catalog.
>
> Hope this helps to address any confusion.
>
> --Jeanne
>
>
> **********************************************************
> Jeanne Holm
> Evangelist, Data.gov
> U.S. General Services Administration
>
> Cell: (818) 434-5037
> Twitter/Facebook/LinkedIn: JeanneHolm
> **********************************************************
>
> ------------------------------------------------------------------------
>
> *From:*Brand Niemann <bniemann@cox.net <mailto:bniemann@cox.net>>
> *To:* "'Holm, Jeanne M (1760)'" <jeanne.m.holm@jpl.nasa.gov 
> <mailto:jeanne.m.holm@jpl.nasa.gov>>; 'eGov W3C' 
> <public-egov-ig@w3.org <mailto:public-egov-ig@w3.org>>
> *Sent:* Thursday, May 23, 2013 8:10 AM
> *Subject:* RE: New open source catalog and list of APIs on Data.gov
>
> Jeanne, Thank you. I heard Doug Nebert announce this yesterday at the 
> UCGIS 2013 Symposium:
>
> http://ucgis2.org/event-item/preliminary-program
>
> When I look for the 73,651 data sets, I find only 7,808 at: 
> https://explore.data.gov/Other/Data-gov-Catalog/pyv4-fkgv
>
> So where are the other 65,843?
>
> My audit for reproducible results is at: 
> http://semanticommunity.info/An_Open_Data_Policy
>
> Thanks, Brand
>
> Dr. Brand Niemann
>
> Director and Senior Data Scientist
>
> Semantic Community
>
> http://semanticommunity.info <http://semanticommunity.info/>
>
> http://gov.aol.com/bloggers/brand-niemann/
>
> 703-268-9314
>
> *From:*Holm, Jeanne M (1760) [mailto:jeanne.m.holm@jpl.nasa.gov]
> *Sent:* Thursday, May 23, 2013 8:42 AM
> *To:* eGov W3C
> *Subject:* New open source catalog and list of APIs on Data.gov
>
> Hi all--
>
> I invite you to visit Data.gov to see the new catalog for browsing 
> U.S. open data: http://catalog.data.gov <http://catalog.data.gov/> We 
> have combined raw and geospatial data from many sources across the 
> U.S. and presented it through an open source tool, CKAN.
>
> In connection with the U.S. Digital Strategy we have also created a 
> new list of government APIs: 
> http://www.data.gov/developers/page/developer-resources
>
> Find out more at: 
> http://www.data.gov/blog/datagov-launches-new-catalog-and-apis
>
> --Jeanne Holm
>
>
> **********************************************************
> Jeanne Holm
> Evangelist, Data.gov
> U.S. General Services Administration
>
> Cell: (818) 434-5037
> Twitter/Facebook/LinkedIn: JeanneHolm
> **********************************************************
>
Received on Thursday, 23 May 2013 15:31:26 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 19:43:28 UTC