Re: Machine-Readable Performance

I certainly don't mean to argue that the labs or other contractors of 
government agencies shouldn't be attempting to publish information in 
the most open way possible. The issue for this group is whether we can 
depend on that for assessing credibility. At this point in time, I think 
the answer is no.

On 2/20/20 3:19 PM, Owen Ambur wrote:
>
> Annette, OMB Circular A-11 
> <https://www.whitehouse.gov/wp-content/uploads/2018/06/a11.pdf> 
> provides the following definition:
>
>     Machine Readable Format. Format in a standard computer language
>     (not English text) that can be read automatically by a web browser
>     or computer system. (e.g., xml). Traditional word processing
>     documents, hypertext markup language (HTML) and portable document
>     format (PDF) files are easily read by humans but typically are
>     difficult for machines to interpret. Other formats such as
>     extensible markup language (XML), (JSON), or spreadsheets with
>     header columns that can be exported as comma separated values
>     (CSV) are machine readable formats. It is possible to make
>     traditional word processing documents and other formats machine
>     readable but the documents must include enhanced structural elements.
>
> Unfortunately, Circular A-11 has been published in PDF so I cannot 
> directly reference that provision. However, I have converted part of 
> it to StratML format so I can point directly to this key section 
> <https://stratml.us/carmel/iso/A11-240wStyle.xml#_aada4d70-a90a-11e9-a082-d69e4d1a686a>, 
> in which OMB has finally committed to piloting implementation of 
> section 10 of GPRAMA with a few agencies this fiscal year.
>
> Yes, technically speaking, GPRAMA only applies to Federal agencies.  
> However, that is a poor and bureaucratic excuse for failing to apply a 
> good practice, particularly when public funding is involved.  The OPEN 
> Government Data Act (OGDA 
> <https://www.linkedin.com/pulse/open-gov-data-act-machine-readable-records-owen-ambur/>) 
> extends that good practice to all government records.  Moreover, the 
> Grant Reporting Efficiency and Agreements Transparency Act 
> <https://www.congress.gov/bill/116th-congress/house-bill/150> of 2019 
> requires the establishment and use of data standards for information 
> reported by recipients of federal grants.  Why should the national 
> labs be excluded?  Should they not instead be helping to lead the way?
>
> A case can also be made that contracts should be written as 
> performance plans. 
> https://en.wikipedia.org/wiki/Performance-based_contracting StratML 
> Part 2 <https://stratml.us/#Part2> is an XML vocabulary and schema for 
> performance plans and reports.
>
> What is credibility if not a measure of the quality of performance?  
> Without such data, isn't anyone's opinion (belief) as good as anyone 
> else's? https://en.wikipedia.org/wiki/Credibility Short cuts may lead 
> to predictably bad results.
>
> See also https://en.wikipedia.org/wiki/Consequentialism versus 
> https://en.wikipedia.org/wiki/Deontological_ethics As the saying goes, 
> the road to hell 
> <https://en.wikipedia.org/wiki/The_road_to_hell_is_paved_with_good_intentions> 
> ... (and some politicians seem hell-bent on taking us there).
>
> As unlikely as it may be, it appears Congress is far ahead of "many 
> contractors" and perhaps most laureates in recognizing the importance 
> of open, standard, machine-readable documents (records).
>
> Owen
>
> On 2/20/2020 4:55 PM, Annette Greiner wrote:
>>
>> Unfortunately, the GPRAMA is unlikely to lead government contractors 
>> like the national labs to make their performance reports available in 
>> truly machine-readable formats, as (a) it applies to agencies, not 
>> contractors, and (b) it does not specify a definition of 
>> machine-readable. For many people, PDF counts as "searchable and 
>> machine readable", and indeed many of the contractors already meet 
>> that bar. From what I can see, the GPRAMA really doesn't do more than 
>> require that information about the planning of the government itself 
>> be made available to humans via computers. It's a good step, but to 
>> my mind at least, it doesn't exemplify particularly tech-savvy 
>> legislation. I don't see it as a means to glean reliable credibility 
>> signals.
>>
>> -Annette
>>
>> On 2/19/20 4:41 PM, Owen Ambur wrote:
>>>
>>> Point well taken, Annette.  Beyond peer recognition however, it 
>>> would be good to make salient the underlying performance indicators 
>>> specifying what excellence truly means.
>>>
>>> In the case of U.S. federal agencies, section 10 
>>> <https://www.linkedin.com/pulse/open-machine-readable-government-owen-ambur/> 
>>> of the GPRA Modernization Act (GPRAMA) requires them to publish 
>>> their performance reports in machine-readable format.  It would be 
>>> good if some of the laureates associated with DOE and LBNL could 
>>> help lead the way.
>>>
>>> In the meantime, on their behalf, I have published their strategic 
>>> plans in open, standard, machine-readable StratML format at 
>>> https://stratml.us/drybridge/index.htm#DOE
>>>
>>> Perhaps someday news organizations will be held accountable not only 
>>> for doing likewise but also paying greater deference to reliable 
>>> data than to story telling based so heavily on personal 
>>> perspectives.  If not, more of what we already see is what we are 
>>> likely to get, both literally as well as figuratively.
>>>
>>> BTW, here's OKF's data journalism guide in StratML format: 
>>> https://stratml.us/carmel/iso/DJH5MFGwStyle.xml Unfortunately, it 
>>> says noting about the Foundations for Evidence-Based Policymaking 
>>> Act (FEBPA <https://stratml.us/drybridge/index.htm#FEBPA>), 
>>> including Title II, the OPEN Government Data Act (OGDA 
>>> <https://www.linkedin.com/pulse/open-gov-data-act-machine-readable-records-owen-ambur/>).
>>>
>>> It is ironic that Congress, which is held in such low regard, seems 
>>> to be so far ahead of the news media, the "knowledge" community, and 
>>> the W3C in recognizing the importance of open, standard 
>>> schema-compliant, machine-readable public records. 
>>> https://en.wikipedia.org/wiki/Machine-readable_document
>>>
>>> Owen
>>>
>>> On 2/19/2020 6:59 PM, Annette Greiner wrote:
>>>>
>>>> One of the things that the awards idea makes me think about is 
>>>> evaluating not just a site but the organization that publishes it. 
>>>> Scientific organizations don't get journalism awards, but their 
>>>> researchers may well get prestigious scientific awards, like Nobel 
>>>> Prizes and Fields Medals. I work at a lab that's pretty conspicuous 
>>>> for its Nobels, so I don't want to emphasize that more than it 
>>>> deserves, but in general I want to make sure this list doesn't end 
>>>> up only making sense for journalistic sites.
>>>>
>>>> -Annette
>>>>
>>>> On 2/19/20 9:21 AM, Sandro Hawke wrote:
>>>>>
>>>>>
>>>>> On 2/19/20 11:48 AM, Sastry, Nishanth wrote:
>>>>>>
>>>>>> Hello Sandro, all,
>>>>>>
>>>>>> This just a quick email to introduce myself as a new member to 
>>>>>> the group, from King’s College London. I had applied to the 
>>>>>> credible web WG several months back, but got approved by our 
>>>>>> University contact just days before, and have since been added to 
>>>>>> this email list.
>>>>>>
>>>>>> We have done a bunch of work looking at
>>>>>>
>>>>>>  1. hyper partisan websites, in the context of the US
>>>>>>     Presidential elections:
>>>>>>
>>>>>>   * https://nms.kcl.ac.uk/nishanth.sastry/publication/nrswww-2018-b/
>>>>>>       o This provided inputs for a major expose by Buzzfeed News:
>>>>>>         https://www.buzzfeednews.com/article/craigsilverman/inside-the-partisan-fight-for-your-news-feed
>>>>>>   * https://nms.kcl.ac.uk/nishanth.sastry/publication/nrswww-2020/
>>>>>>       o Showing that right leaning sites track more intensely
>>>>>>         than left leaning sites (Covered by WIRED:
>>>>>>         https://www.wired.com/story/right-left-news-site-ad-tracking/)
>>>>>>
>>>>>>
>>>>>>  2. bias in news and social media during political crises
>>>>>>
>>>>>>   * https://nms.kcl.ac.uk/nishanth.sastry/publication/karamshuk-16-slant/
>>>>>>
>>>>>>  3. And finally, on transferring trust across domains (which is
>>>>>>     very aligned with what I see in the signals draft. We also
>>>>>>     use age as an “ungameable” signal to transfer trust across
>>>>>>     domains. We do this for IDs of individuals rather than
>>>>>>     domains, but the paper develops ways to calibrate trust,
>>>>>>     answering questions such as – is a 10 year-old Facebook ID
>>>>>>     more trustworthy than a 15 year old Gmail ID, for example):
>>>>>>
>>>>>>   * https://nms.kcl.ac.uk/nishanth.sastry/publication/nr-swww-16/
>>>>>>
>>>>>
>>>>> Very nice.  I'd love to get into signals about individuals, but we 
>>>>> it looked like websites would be a little simpler, and we wanted 
>>>>> to start in the simplest possible place.  Hopefully we can get 
>>>>> into such things fairly soon.
>>>>>
>>>>>>  *
>>>>>>
>>>>>>
>>>>>> I will join the Zoom at 7pm GMT, and can add any further details 
>>>>>> that may be interesting to the group. Looking forward.
>>>>>>
>>>>>
>>>>> Great, looking forward to meeting you.  This meeting will be 
>>>>> mostly about wrapping up this little sprint, but then hopefully we 
>>>>> can expand a bit for the next phase.
>>>>>
>>>>>      -- Sandro
>>>>>>
>>>>>> Best wishes
>>>>>>
>>>>>> nishanth
>>>>>>
>>>>>> *From: *Sandro Hawke <sandro@w3.org>
>>>>>> *Date: *Wednesday, 19 February 2020 at 15:51
>>>>>> *To: *Credible Web CG <public-credibility@w3.org>
>>>>>> *Subject: *journalism award signals
>>>>>> *Resent from: *<public-credibility@w3.org>
>>>>>> *Resent date: *Wednesday, 19 February 2020 at 15:51
>>>>>>
>>>>>> I did a bit more work on the Journalism Awards, framing it as a 
>>>>>> general signal and one more specific signals.
>>>>>>
>>>>>> I put them into the "reviewed signals" draft, marked as "pending".
>>>>>>
>>>>>> Here's a dated version of that draft: 
>>>>>> https://credweb.org/reviewed-signals-20200219/ 
>>>>>> <https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcredweb.org%2Freviewed-signals-20200219%2F&data=01%7C01%7Cnishanth.sastry%40kcl.ac.uk%7C1c1f34f786234483d96c08d7b5539380%7C8370cf1416f34c16b83c724071654356%7C0&sdata=Vf7GBSfAxU5%2BtP8oOqK1vMf0Oxw2DgDXLWtBoQ8f4k0%3D&reserved=0> 
>>>>>> (The undated version presumably wont show them as pending after 
>>>>>> today, which could confuse someone reading this later.)
>>>>>>
>>>>>> Meeting in about 3 hours, as usual. Agenda 
>>>>>> <https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.google.com%2Fdocument%2Fd%2F1-KcB121I6D6J2ZdQET-qatqCaqv3ttlZkfhgyWEk7nM%2Fedit&data=01%7C01%7Cnishanth.sastry%40kcl.ac.uk%7C1c1f34f786234483d96c08d7b5539380%7C8370cf1416f34c16b83c724071654356%7C0&sdata=fu%2FG4cV3ziND%2BcDFacnZsRAKJLiVYuYwRF9c9Ik%2FSFM%3D&reserved=0>.
>>>>>>
>>>>>>        -- Sandro
>>>>>>
>>>>>>
>>>>>
>>>> -- 
>>>> Annette Greiner (she)
>>>> NERSC Data and Analytics Services
>>>> Lawrence Berkeley National Laboratory
>>>>
>> -- 
>> Annette Greiner (she)
>> NERSC Data and Analytics Services
>> Lawrence Berkeley National Laboratory
>>
-- 
Annette Greiner (she)
NERSC Data and Analytics Services
Lawrence Berkeley National Laboratory