Re: New open source catalog and list of APIs on Data.gov

Brand,

Since the new site is based on CKAN, you can read the CKAN API 
documentation yourself to figure out how to download the data catalog:

http://docs.ckan.org/en/ckan-2.0/api.html

The API is running really slow right now (probably all of us checking 
out the new site), so I'd suggest *not* hitting the API link below right 
now, but I've mirrored it so you can see the output quickly:

https://catalog.data.gov/api/3/action/package_list
   => Produces a JSON list of entries in the catalog
   => Here's a static mirror: 
http://razor.occams.info/files/catalog_data_gov_api_action_package_list.json

The package_search API function returns complete metadata rather than 
just a list of package names. It requires an HTTP POST, so here's an 
example of that:

     curl -d '{"rows":1, "start": 0}' 
https://catalog.data.gov/api/3/action/package_search
     => static mirror: 
http://razor.occams.info/files/catalog_data_gov_api_action_package_search.json

- Josh Tauberer (@JoshData)

http://razor.occams.info

On 05/23/2013 10:34 AM, Brand Niemann wrote:
>
> Thanks, Jeanne. Please send me the link to download the entire and new 
> data catalog.
>
> I am getting ready for: On Thursday, May 23^rd at 2:00PM ET, Federal 
> Chief Information Officer Steven VanRoekel will host a conference call 
> to discuss the 1-yr Anniversary of theDigital Government Strategy 
> <http://whitehouse.gov/digitalgov> and ongoing Administration efforts 
> to drive innovation through open data and other initiatives.
>
> Brand
>
> *From:*Holm, Jeanne M (1760) [mailto:jeanne.m.holm@jpl.nasa.gov]
> *Sent:* Thursday, May 23, 2013 10:10 AM
> *To:* Brand Niemann; 'eGov W3C'
> *Subject:* Re: New open source catalog and list of APIs on Data.gov
>
> Brand--
>
> As John pointed out, the entire and new catalog (as of this morning) 
> is available at https://catalog.data.gov/dataset 
> <http://catalog.data.gov>
>
> The catalog at 
> https://explore.data.gov/Other/Data-gov-Catalog/pyv4-fkgv is our 
> previous catalog of "raw" data that is not part of the larger catalog 
> above.
>
> The number differs from our earlier number of datasets:  The total 
> number of datasets reflects datasets plus data series. A data series 
> may contain a large number of additional products or files of the same 
> type. The previous Data.gov catalog counted the individual datasets 
> and not just the series and therefore had a higher total number of 
> datasets noted. The change in number does not reflect a change in the 
> size of the catalog, but rather a different structure of data products 
> in the new catalog.
>
> Hope this helps to address any confusion.
>
> --Jeanne
>
>
> **********************************************************
> Jeanne Holm
> Evangelist, Data.gov
> U.S. General Services Administration
>
> Cell: (818) 434-5037
> Twitter/Facebook/LinkedIn: JeanneHolm
> **********************************************************
>
> ------------------------------------------------------------------------
>
> *From:*Brand Niemann <bniemann@cox.net <mailto:bniemann@cox.net>>
> *To:* "'Holm, Jeanne M (1760)'" <jeanne.m.holm@jpl.nasa.gov 
> <mailto:jeanne.m.holm@jpl.nasa.gov>>; 'eGov W3C' 
> <public-egov-ig@w3.org <mailto:public-egov-ig@w3.org>>
> *Sent:* Thursday, May 23, 2013 8:10 AM
> *Subject:* RE: New open source catalog and list of APIs on Data.gov
>
> Jeanne, Thank you. I heard Doug Nebert announce this yesterday at the 
> UCGIS 2013 Symposium:
>
> http://ucgis2.org/event-item/preliminary-program
>
> When I look for the 73,651 data sets, I find only 7,808 at: 
> https://explore.data.gov/Other/Data-gov-Catalog/pyv4-fkgv
>
> So where are the other 65,843?
>
> My audit for reproducible results is at: 
> http://semanticommunity.info/An_Open_Data_Policy
>
> Thanks, Brand
>
> Dr. Brand Niemann
>
> Director and Senior Data Scientist
>
> Semantic Community
>
> http://semanticommunity.info <http://semanticommunity.info/>
>
> http://gov.aol.com/bloggers/brand-niemann/
>
> 703-268-9314
>
> *From:*Holm, Jeanne M (1760) [mailto:jeanne.m.holm@jpl.nasa.gov]
> *Sent:* Thursday, May 23, 2013 8:42 AM
> *To:* eGov W3C
> *Subject:* New open source catalog and list of APIs on Data.gov
>
> Hi all--
>
> I invite you to visit Data.gov to see the new catalog for browsing 
> U.S. open data: http://catalog.data.gov <http://catalog.data.gov/> We 
> have combined raw and geospatial data from many sources across the 
> U.S. and presented it through an open source tool, CKAN.
>
> In connection with the U.S. Digital Strategy we have also created a 
> new list of government APIs: 
> http://www.data.gov/developers/page/developer-resources
>
> Find out more at: 
> http://www.data.gov/blog/datagov-launches-new-catalog-and-apis
>
> --Jeanne Holm
>
>
> **********************************************************
> Jeanne Holm
> Evangelist, Data.gov
> U.S. General Services Administration
>
> Cell: (818) 434-5037
> Twitter/Facebook/LinkedIn: JeanneHolm
> **********************************************************
>