Re: A Credibility Use Case: Lessig v New York Times ("Clickbate Defamation")

Hi all,

This sounds like an interesting case to me, and I agree entirely with what
has been said so far, especially by Sandro and Daniel.

My thoughts in short, but I'm happy to elaborate more when useful:
- indicators could provide laypeople a handle to grasp a debate. We could
think of non-experts delegating to the indicators the job of ascertaining
"who is right" especially when these people lack time/resources/expertise
to do it on their own; in this case, though, source-level indicators are
not sufficient to tackle the problem, thus we should dig deeper into the
actual documents/debates (so I agree with having a multilayered approach;
also this additional investigation might be automated);
- I tend to read indicators in probabilistic terms, so while a credible
source is likely to be right, inevitably sometimes it will be
wrong/inaccurate/imprecise/etc.;
- for the same reason, just looking at the difference in credibility scores
to determine who is right between two sources might be too simplistic.

With some colleagues, we started investigating the case of the covid debate
in Italy, when experts started debating several statements (e.g., regarding
the severity of the pandemic), taking different stances, in a period when
evidence about the actual correctness of such stances was scarce. There we
looked at the dynamic and structure of the debate (first results are
available here https://doi.org/10.1145/3462203.3475907), and we are
extending this line of work.

Best,

Davide

Il giorno lun 30 ago 2021 alle ore 16:31 Daniel Schwabe <dschwabe@gmail.com>
ha scritto:

> Hi all,
>
> I believe that indeed credibility signals would contribute little to steer
> the reader towards questioning NYTs credibility, unless it systematically
> publishes such misleading headlines and is slow to respond to rebuttals -
> as Sandro points out.
>
> It seems that the issue revolves more about readers consuming only the
> headline vs. the full text. By doing this, there is an expected harm  to
> Lessig’s reputation that an incomplete reading of the story may cause (ie,
> his credibility as evaluated by the readers of the story). This harm would
> come because by reading only the headline, people would believe he holds
> certain moral values regarding receiving “dirty money” - although the
> entire text of the story does not warrant such attribution.
>
> This is a perfect example of what I argued about information disorders (I
> think it should really be called communication disorders…) - a “dispute of
> narratives”, each contains statements about people and events claimed to be
> true - ie. that each author (NYT or Lessig) would like the reader to accept
> as “facts”. They are competing because the statements being made are
> contradictory to each other, so if a reader needs to make a decision on
> some action based on these statements (e.g., support some initiative by L.
> Lessig, or tweet about it, etc...), s/he will need to choose between them.
>
> There are numerous factors that will make a reader prefer one of the
> narratives over the other, and it is obvious to me that credibility signals
> will only cover some of them, as they are typically focused on the
> provenance aspect, especially the author. Granted, this is an important
> factor, but not the only one. This example illustrates this quite well -
> both have strong credibility signals, so these would not be the determining
> factor, and further detailed analysis would be necessary.
>
> One could ask, if credibility signals are not enough, is there something
> that could mitigate this problem? In this particular case, I believe the
> best that could be achieved would be for the NYT do provide equal access to
> L. Lessig’s points, so that the reader would be *immediately*  made aware
> that even the headline is being contested. In fact,  a venue providing this
> mechanism (or not) could itself become another credibility signal, but one
> would still need to dig into the details of a particular story to make up
> one’s mind.
>
> I think this is an excellent opportunity of us to try scope more precisely
> what are feasible credibility signals - ie. ones that provide some kind
> of information that can be useful to readers in choosing among competing
> narratives, AND could effectively be implemented  - also making it
> clearer what they can and cannot address.
>
> To help organize the discussion, I take the opportunity to argue that we
> should have a layered architecture, where the base is a layer that provides
> authenticity to the signals in the layer(s) above. Some examples that would
> go in this layer, IMHO, are C2PA and JournalList. It seems to me that work
> on signals in this layer is more advanced, having implementations and
> already being tried out…
>
> Cheers
> Daniel
>
>
>
>
>
> On 29 Aug 2021, at 22:10, Bob Wyman <bob@wyman.us> wrote:
>
> Sandro wrote:
>
>> it's perhaps the kind of situation where ANYONE willing to spent 30
>> minutes hard work can evaluate it.
>
> I chose the Lessig v NYT example, in part, because it is so easy to
> evaluate. In fact, the New York Times' own defense was to say that anyone
> who actually read the article would realize that the headline was
> inaccurate. The problem here is that, given the New York TImes' reputation
> for credibility, few people will actually question the headline's accuracy
> unless some signal can be attached to it to indicate that further
> evaluation is needed or, at least, that it is controversial. To amplify
> your Alice & Hitler example: It's as though a headline said: "Alice likes
> Hitler," even though she's quoted in the article's paragraph 25 as saying:
> "I hate Hitler, but I like his style of painting." While we might question
> Alice's aesthetic sense, there is no reasonable way to go from her
> statement to the headline's clear meaning. But, given that the quote is in
> paragraph 25, few readers will discover the contradiction on their own.
> Most readers will forever think of Alice as someone who likes Hitler and
> may remain comfortable with that belief because it is based on statements
> by the highly credible New York Times. ("Who am I to dispute what the NYT
> has written?") Had the claim been made by some other source, many readers
> might have either ignored it or questioned it. In such cases, credibility
> is a problem since it discourages readers from questioning statements.
>
> bob wyman
>
>
> On Sun, Aug 29, 2021 at 8:37 PM Sandro Hawke <sandro@hawke.org> wrote:
>
>> Interesting scenario.  A few thoughts.
>>
>> This is not low-hanging fruit. Evaluating contradictions between credible
>> sources, like this, seems like harder territory to address than some common
>> cases like: a source with no obvious credibility rating, so people end up
>> believing it because it says something they like or uses a nice layout.
>>
>> But maybe it's not that hard either.  The easy thing about this is that
>> it's perhaps the kind of situation where ANYONE willing to spent 30 minutes
>> hard work can evaluate it.  Lessig's argument is that the NYT headline and
>> lede are dangerously misleading summaries of his article and the
>> interview.  No special information or access or skill is necessary to
>> evaluate Lessig's claim that the NYT got it wrong; just read the article
>> and interview and think a little.  That kind of thing can be sort of
>> crowd-sourced. It's not like a public health claim or a mystery.
>>
>> It's quite hard among self-contained fact-checks, though, because the
>> text has some nuance and the topic is extremely sensitive.  It's like if
>> Alice makes a false derogatory statement about Adolf Hitler, then who wants
>> to wade into the muck, fact check her, and get branded a Hitler apologist?
>> And then if her argument required some nuance, ... it's a total mess.
>>
>> On the mechanical issues:
>>
>> It's clear to me we need a credibility layer independent from the content
>> layer.
>>
>> When you're viewing content from X, you need independent annotations and
>> filtering based on signals from your whole credibility network, beyond the
>> control of X.
>>
>> Ways this could be done:
>> 1. When you're getting your content through social media, the credibility
>> layer can be implemented by the social media platform. I expect social
>> media vendors will be happy to do this once the system is proven to work
>> well but not until then.
>> 2. When you're getting your content through a browser, the credibility
>> layer could be implemented in a browser extension for now and eventually in
>> the browser itself. I think browser vendors would love to do this if the
>> system is proven to work well, but not before then.
>> 3. The website for X *could* provide the credibility layer branded as a
>> neutral 3rd party service that they swear they wont tamper with. I think
>> this is more likely for product vendors than news vendors. X could be a
>> manufacture of Widgets, and they want you to use your credibility network
>> to see that people generally have a great experience with X's Widgets. I
>> think it could also work for news sites, though.
>>
>> I think the most promising deployment strategies are via (1, above)
>> finding a small social media platform that's interested in making this
>> their distinctive feature, and via (3, above) finding some high traffic
>> sites who are very confident of their credibility rating among people who
>> know them and want to spread the word to others.
>>
>>     -- Sandro
>>
>>
>>
>