Re: A Data Provider Tool for Data.gov

The Data.gov Catalog has keywords (DCMI Subjects) and Custom Meta Data (provided by the Agency).
Only the Custom Meta Data is informative and has provenance and validity.
This is a keyword list, exactly what it looks like to you.

--Gannon




----- Original Message -----
From: John Erickson <olyerickson@gmail.com>
To: Gannon Dick <gannon_dick@yahoo.com>
Cc: Brand Niemann <bniemann@cox.net>; "Holm, Jeanne M (1760)" <jeanne.m.holm@jpl.nasa.gov>; eGov W3C <public-egov-ig@w3.org>
Sent: Tuesday, June 4, 2013 3:45 PM
Subject: Re: A Data Provider Tool for Data.gov

Sorry --- maybe I'm too traveled and otherwise overworked lately ---
could someone please explain in actual *words* how this is helpful?

Mostly, I don't understand the provenance/validity of this. It looks
like a very loose aggregation, missing context.

Again, maybe it's just me...

John



On Tue, Jun 4, 2013 at 4:07 PM, Gannon Dick <gannon_dick@yahoo.com> wrote:
> Oh yes, another thing: I think some eye popping evangelical presentations
> could be done with this data.
>
> The Challenge announced today is the other element:
> http://impressjs.challengepost.com/
>
> You might snag a T-Shirt *while* sucking up.  Life gets better than that ?
>
> --Gannon
>
> ________________________________
> From: Gannon Dick <gannon_dick@yahoo.com>
> To: Brand Niemann <bniemann@cox.net>; "'Holm, Jeanne M (1760)'"
> <jeanne.m.holm@jpl.nasa.gov>; 'eGov W3C' <public-egov-ig@w3.org>
> Sent: Tuesday, June 4, 2013 2:40 PM
> Subject: A Data Provider Tool for Data.gov
>
> Who on the Open Government Data Science Team is responsible for sucking up
> to the Boss ?  Today The White House announced five executive actions and
> seven recommendations for Congress to protect innovators from lawsuits, or
> so I'm told.
>
> I made an index of Agencies and Subdivisions of record (177) for the current
> catalog download, and a cross-reference of the Keyword Sets used, by Agency.
> It is in the Catalog Zip File[*] or here: http://tinyurl.com/agency-keywords
>
> The intent is to give Agencies who are preparing documents for Data.gov some
> indication of the Keywords used before.  This is a SEO type function, and
> unrelated to "Linked Data" (Custom Metadata provided by the Agency), but it
> is something an Agency can do to promote message coherence, reproducibility
> and sucking up (see above).
>
> Brand, you should be able to copy and paste the HTML (Index) Table into a
> spreadsheet.  I did clean up duplicate names, eliminated TLA's (they are a
> Linked Data encoding nightmare) and unicode escape entities.  If you have
> any problems, let me know.
>
> --Gannon
>
> [*] http://www.rustprivacy.org/2013/egov/catalog/DataDotGovCatalog.zip
>
>
> ________________________________
> From: Brand Niemann <bniemann@cox.net>
> To: 'Gannon Dick' <gannon_dick@yahoo.com>; "'Holm, Jeanne M (1760)'"
> <jeanne.m.holm@jpl.nasa.gov>; 'eGov W3C' <public-egov-ig@w3.org>
> Sent: Sunday, June 2, 2013 5:27 AM
> Subject: RE: New open source catalog and list of APIs on Data.gov
>
> Thanks again Gannon!
>
> We have an Open Government Data Data Science Team as follows:
> Steven VanRoekel - Federal CIO - Directs the Digital Government Strategy
> Jeanne Holm - Data.gov Evangelist - Evangelizes the Availability of the Data
> Gannon Dick - Data Preparation - Prepares the Data for Analysis
> Brand Niemann - Data Scientist - Provides the Data (Catalog and Results) in
> a Data Platform
>
> http://semanticommunity.info/An_Open_Data_Policy#Story
>
> Dr. Brand Niemann
> Director and Senior Data Scientist
> Semantic Community
> http://semanticommunity.info
> http://gov.aol.com/bloggers/brand-niemann/
> 703-268-9314
>
> From: Gannon Dick [mailto:gannon_dick@yahoo.com]
> Sent: Saturday, June 01, 2013 3:54 PM
> To: Brand Niemann; 'Holm, Jeanne M (1760)'; 'eGov W3C'
> Subject: Re: New open source catalog and list of APIs on Data.gov
>
> You are welcome, Dr. Niemann.
>
> In the meantime, I realized I could do something interesting with the
> catalog having chopped it up into fields.
>
> There are keywords (dct:subject in Dublin Core parlance).  They are in a
> comma delimited list (like a CURI - Compact URI, although they don't have a
> namespace).  But there is no reason not to make a list of individual terms
> by Agency.  The catalog also contains "Custom Metadata" from the Agencies.
> I collated that too.  In addition, I was doing some work with the FACA
> Database and made a list of identifiers for 2011 (it has the Agency URLs,
> Acronym, etc.).
>
> Spreadsheets are in
> http://www.rustprivacy.org/2013/egov/catalog/DataDotGovCatalog.zip
>
> If you just want to take a quick look ...
> http://www.rustprivacy.org/2013/egov/catalog/AgencyMetadataPopulated.csv
>
> --Gannon
>
>
> ________________________________
> From: Brand Niemann <bniemann@cox.net>
> To: "'Holm, Jeanne M (1760)'" <jeanne.m.holm@jpl.nasa.gov>; 'eGov W3C'
> <public-egov-ig@w3.org>
> Cc: Gannon Dick <gannon_dick@yahoo.com>
> Sent: Friday, May 31, 2013 8:21 PM
> Subject: RE: New open source catalog and list of APIs on Data.gov
>
> Jeanne, Thank you. The Data.gov Catalog and a specific example (Federal Data
> Consolidation Initiative) and the New Data.gov Catalog (thanks to Gannon)
> and a specific example (USGS Sediment Texture Database) using the same Data
> Platform (Spotfire) for the catalogs and the examples are found at:
>
> http://semanticommunity.info/An_Open_Data_Policy
>
> http://semanticommunity.info/GIS_Data_Science
>
> Dr. Brand Niemann
> Director and Senior Data Scientist
> Semantic Community
> http://semanticommunity.info
> http://gov.aol.com/bloggers/brand-niemann/
> 703-268-9314
>
> From: Holm, Jeanne M (1760) [mailto:jeanne.m.holm@jpl.nasa.gov]
> Sent: Thursday, May 30, 2013 2:49 PM
> To: Brand Niemann; 'eGov W3C'
> Subject: Re: New open source catalog and list of APIs on Data.gov
>
> Hi all--
>
> Here's the way to download the entire catalog.  The Data.gov catalog
> (exclusive of geospatial data) can still be accessed at
> https://explore.data.gov/Other/Data-gov-Catalog/pyv4-fkgv  You can use the
> export button at the top right to be able to download it in a variety of
> formats.
>
> The API is another way to access the entire catalog.
>
> We are exploring other ways to make the newly integrated catalog available.
>
> --Jeanne
>
> **********************************************************
> Jeanne Holm
> Evangelist, Data.gov
> U.S. General Services Administration
> Cell: (818) 434-5037
> Twitter/Facebook/LinkedIn: JeanneHolm
> **********************************************************
>
> From: Brand Niemann <bniemann@cox.net>
> Date: Thursday, May 23, 2013 7:34 AM
> To: Jeanne Holm <Jeanne.M.Holm@jpl.nasa.gov>, eGov W3C
> <public-egov-ig@w3.org>
> Subject: RE: New open source catalog and list of APIs on Data.gov
>
> Thanks, Jeanne. Please send me the link to download the entire and new data
> catalog.
>
> I am getting ready for: On Thursday, May 23rd at 2:00PM ET, Federal Chief
> Information Officer Steven VanRoekel will host a conference call to discuss
> the 1-yr Anniversary of the Digital Government Strategy and ongoing
> Administration efforts to drive innovation through open data and other
> initiatives.
>
> Brand
>
> From: Holm, Jeanne M (1760) [mailto:jeanne.m.holm@jpl.nasa.gov]
> Sent: Thursday, May 23, 2013 10:10 AM
> To: Brand Niemann; 'eGov W3C'
> Subject: Re: New open source catalog and list of APIs on Data.gov
>
> Brand--
>
> As John pointed out, the entire and new catalog (as of this morning) is
> available at https://catalog.data.gov/dataset
>
> The catalog at https://explore.data.gov/Other/Data-gov-Catalog/pyv4-fkgv is
> our previous catalog of "raw" data that is not part of the larger catalog
> above.
>
> The number differs from our earlier number of datasets:  The total number of
> datasets reflects datasets plus data series. A data series may contain a
> large number of additional products or files of the same type. The previous
> Data.gov catalog counted the individual datasets and not just the series and
> therefore had a higher total number of datasets noted. The change in number
> does not reflect a change in the size of the catalog, but rather a different
> structure of data products in the new catalog.
>
> Hope this helps to address any confusion.
>
> --Jeanne
>
> **********************************************************
> Jeanne Holm
> Evangelist, Data.gov
> U.S. General Services Administration
> Cell: (818) 434-5037
> Twitter/Facebook/LinkedIn: JeanneHolm
> **********************************************************
>
> ________________________________
> From: Brand Niemann <bniemann@cox.net>
> To: "'Holm, Jeanne M (1760)'" <jeanne.m.holm@jpl.nasa.gov>; 'eGov W3C'
> <public-egov-ig@w3.org>
> Sent: Thursday, May 23, 2013 8:10 AM
> Subject: RE: New open source catalog and list of APIs on Data.gov
>
> Jeanne, Thank you. I heard Doug Nebert announce this yesterday at the UCGIS
> 2013 Symposium:
> http://ucgis2.org/event-item/preliminary-program
>
> When I look for the 73,651 data sets, I find only 7,808 at:
> https://explore.data.gov/Other/Data-gov-Catalog/pyv4-fkgv
>
> So where are the other 65,843?
>
> My audit for reproducible results is at:
> http://semanticommunity.info/An_Open_Data_Policy
>
> Thanks, Brand
>
> Dr. Brand Niemann
> Director and Senior Data Scientist
> Semantic Community
> http://semanticommunity.info
> http://gov.aol.com/bloggers/brand-niemann/
> 703-268-9314
>
> From: Holm, Jeanne M (1760) [mailto:jeanne.m.holm@jpl.nasa.gov]
> Sent: Thursday, May 23, 2013 8:42 AM
> To: eGov W3C
> Subject: New open source catalog and list of APIs on Data.gov
>
> Hi all--
>
> I invite you to visit Data.gov to see the new catalog for browsing U.S. open
> data: http://catalog.data.gov We have combined raw and geospatial data from
> many sources across the U.S. and presented it through an open source tool,
> CKAN.
>
> In connection with the U.S. Digital Strategy we have also created a new list
> of government APIs: http://www.data.gov/developers/page/developer-resources
>
> Find out more at:
> http://www.data.gov/blog/datagov-launches-new-catalog-and-apis
>
> --Jeanne Holm
>
> **********************************************************
> Jeanne Holm
> Evangelist, Data.gov
> U.S. General Services Administration
> Cell: (818) 434-5037
> Twitter/Facebook/LinkedIn: JeanneHolm
> **********************************************************
>
>
>
>
>
>



-- 
John S. Erickson, Ph.D.
Director, Web Science Operations
Tetherless World Constellation (RPI)
<http://tw.rpi.edu> <olyerickson@gmail.com>
Twitter & Skype: olyerickson