RE: A Credibility Use Case: Lessig v New York Times ("Clickbate Defamation")

Something to keep in mind as we work on this is that even respected and credible news sources are telling a version of the truth. I’m not commenting on whether the NYT was being truthful or not in their article, but I think it’s important to remember that credibility and truth are not the same thing. It is impossible (I think) to write a formula to detect the truth, especially since my version of the truth and your version of the truth might be different.

Tzviya Siegman
Information Standards Principal
Wiley
201-748-6884
tsiegman@wiley.com<mailto:tsiegman@wiley.com>

From: Daniel Schwabe <dschwabe@gmail.com>
Sent: Monday, August 30, 2021 10:30 AM
To: Credible Web CG <public-credibility@w3.org>
Cc: Sandro Hawke <sandro@hawke.org>; Bob Wyman <bob@wyman.us>
Subject: Re: A Credibility Use Case: Lessig v New York Times ("Clickbate Defamation")

⛔
This is an external email.
Hi all,

I believe that indeed credibility signals would contribute little to steer the reader towards questioning NYTs credibility, unless it systematically publishes such misleading headlines and is slow to respond to rebuttals - as Sandro points out.

It seems that the issue revolves more about readers consuming only the headline vs. the full text. By doing this, there is an expected harm  to Lessig’s reputation that an incomplete reading of the story may cause (ie, his credibility as evaluated by the readers of the story). This harm would come because by reading only the headline, people would believe he holds certain moral values regarding receiving “dirty money” - although the entire text of the story does not warrant such attribution.

This is a perfect example of what I argued about information disorders (I think it should really be called communication disorders…) - a “dispute of narratives”, each contains statements about people and events claimed to be true - ie. that each author (NYT or Lessig) would like the reader to accept as “facts”. They are competing because the statements being made are contradictory to each other, so if a reader needs to make a decision on some action based on these statements (e.g., support some initiative by L. Lessig, or tweet about it, etc...), s/he will need to choose between them.

There are numerous factors that will make a reader prefer one of the narratives over the other, and it is obvious to me that credibility signals will only cover some of them, as they are typically focused on the provenance aspect, especially the author. Granted, this is an important factor, but not the only one. This example illustrates this quite well - both have strong credibility signals, so these would not be the determining factor, and further detailed analysis would be necessary.

One could ask, if credibility signals are not enough, is there something that could mitigate this problem? In this particular case, I believe the best that could be achieved would be for the NYT do provide equal access to L. Lessig’s points, so that the reader would be immediately  made aware that even the headline is being contested. In fact,  a venue providing this mechanism (or not) could itself become another credibility signal, but one would still need to dig into the details of a particular story to make up one’s mind.

I think this is an excellent opportunity of us to try scope more precisely what are feasible credibility signals - ie. ones that provide some kind of information that can be useful to readers in choosing among competing narratives, AND could effectively be implemented  - also making it clearer what they can and cannot address.

To help organize the discussion, I take the opportunity to argue that we should have a layered architecture, where the base is a layer that provides authenticity to the signals in the layer(s) above. Some examples that would go in this layer, IMHO, are C2PA and JournalList. It seems to me that work on signals in this layer is more advanced, having implementations and already being tried out…

Cheers
Daniel





On 29 Aug 2021, at 22:10, Bob Wyman <bob@wyman.us<mailto:bob@wyman.us>> wrote:

Sandro wrote:
it's perhaps the kind of situation where ANYONE willing to spent 30 minutes hard work can evaluate it.
I chose the Lessig v NYT example, in part, because it is so easy to evaluate. In fact, the New York Times' own defense was to say that anyone who actually read the article would realize that the headline was inaccurate. The problem here is that, given the New York TImes' reputation for credibility, few people will actually question the headline's accuracy unless some signal can be attached to it to indicate that further evaluation is needed or, at least, that it is controversial. To amplify your Alice & Hitler example: It's as though a headline said: "Alice likes Hitler," even though she's quoted in the article's paragraph 25 as saying: "I hate Hitler, but I like his style of painting." While we might question Alice's aesthetic sense, there is no reasonable way to go from her statement to the headline's clear meaning. But, given that the quote is in paragraph 25, few readers will discover the contradiction on their own. Most readers will forever think of Alice as someone who likes Hitler and may remain comfortable with that belief because it is based on statements by the highly credible New York Times. ("Who am I to dispute what the NYT has written?") Had the claim been made by some other source, many readers might have either ignored it or questioned it. In such cases, credibility is a problem since it discourages readers from questioning statements.

bob wyman


On Sun, Aug 29, 2021 at 8:37 PM Sandro Hawke <sandro@hawke.org<mailto:sandro@hawke.org>> wrote:
Interesting scenario.  A few thoughts.

This is not low-hanging fruit. Evaluating contradictions between credible sources, like this, seems like harder territory to address than some common cases like: a source with no obvious credibility rating, so people end up believing it because it says something they like or uses a nice layout.

But maybe it's not that hard either.  The easy thing about this is that it's perhaps the kind of situation where ANYONE willing to spent 30 minutes hard work can evaluate it.  Lessig's argument is that the NYT headline and lede are dangerously misleading summaries of his article and the interview.  No special information or access or skill is necessary to evaluate Lessig's claim that the NYT got it wrong; just read the article and interview and think a little.  That kind of thing can be sort of crowd-sourced. It's not like a public health claim or a mystery.

It's quite hard among self-contained fact-checks, though, because the text has some nuance and the topic is extremely sensitive.  It's like if Alice makes a false derogatory statement about Adolf Hitler, then who wants to wade into the muck, fact check her, and get branded a Hitler apologist?  And then if her argument required some nuance, ... it's a total mess.

On the mechanical issues:

It's clear to me we need a credibility layer independent from the content layer.

When you're viewing content from X, you need independent annotations and filtering based on signals from your whole credibility network, beyond the control of X.

Ways this could be done:
1. When you're getting your content through social media, the credibility layer can be implemented by the social media platform. I expect social media vendors will be happy to do this once the system is proven to work well but not until then.
2. When you're getting your content through a browser, the credibility layer could be implemented in a browser extension for now and eventually in the browser itself. I think browser vendors would love to do this if the system is proven to work well, but not before then.
3. The website for X *could* provide the credibility layer branded as a neutral 3rd party service that they swear they wont tamper with. I think this is more likely for product vendors than news vendors. X could be a manufacture of Widgets, and they want you to use your credibility network to see that people generally have a great experience with X's Widgets. I think it could also work for news sites, though.

I think the most promising deployment strategies are via (1, above) finding a small social media platform that's interested in making this their distinctive feature, and via (3, above) finding some high traffic sites who are very confident of their credibility rating among people who know them and want to spread the word to others.

    -- Sandro