Re: credibility networks (was Re: Is Alice, or her post, credible? (A really rough use case for credibility signals.))

We have been playing with the concept of vouch over in the indieweb world:
https://indieweb.org/Vouch

Different stack since based on webmentions but the workflow pretty much the
same.

The goal is to create semi-private posts for community members vouched by
others and as a trust network.

XFN pretty defunct but I use rel="muse" on my poetry follower list as a
trust signal

https://indieweb.org/XFN



On Wed, Aug 18, 2021 at 11:35 AM David Karger <karger@mit.edu> wrote:

> We've been working for a few years on this kind of trust network.  I
> recognize the subject-dependence of trust, but I think that trying to work
> that into systems being developed now is too ambitious.  Right now the
> value of a trust network can be demonstrated more effectively by starting
> with a simpler system that works in terms of generic credibility rather
> than subject-specific.  What you want are people who know what they know
> and don't claim to know more.   Yes, you'll lose out on your friend who
> knows everything about global warming but is anti-vax, but I think there
> are enough generally trustworthy individuals to drive a network of
> assessments.
> On 8/18/2021 9:46 AM, connie im dialog wrote:
>
> As an additional thought, perhaps to bridge the exchange between Annette
> and Bob, and Sandro: one aspect that I see missing in the scenario below is
> the underlying knowledge/perspective framework or approach that ties
> signals together: could be understood as a schema or rubric.  This is a
> different way to tie signals together from trust networks, and is probably
> underlying those relationships.
>
> What I mean by this is: all of the signals proposed are meant to be
> understood as potential indications of credibility, but they only gain
> meaning when some of them brought together in a specific interpretive
> framework.  Implicit in the development of many of the current signals
> proposed is belief, or trust, in a scientific method of evidence and
> evaluation of claims using methods such as verifiability. It's also tied to
> things like expertise and the development of professions.
>
> This framework of knowledge is different than a moral order that trusts
> inherited wisdom, or tradition, for example.  (I'm going to sidestep the
> elites for now since the power dynamic depends on what kind of elite one
> is.) Just because they are different does mean that they can't in fact
> share one or more signals, but the dominance of certain signals over others
> I think varies.  And because we aren't always consistent, we may hold both
> of these or more frameworks given a certain context or topic.
>
> So I guess I see Bob's suggestion as much in the line of a number of
> crowdsourced wisdom projects, which can be valuable.  When you think of
> historical or even current examples, such as genocide reporting, it's very
> critical to include as many on-the-ground reports as possible, even as
> those claims also need to be validated as much as possible. In these
> contexts, there are many indications of what makes for credible witness
> reports which isn't the same as expertise.
>
> But in some cases, on some topics, you can't go with any crowd
> <https://wearecommons.us/crowd-wisdom-public-wisdom-regarding-misinformation-at-large/>.
> That is at least if you hold to for example a scientific method of
> evaluation and validation.  As with Annette, I have no problem with
> deferring to expertise understood in this framework, and think it's even
> worth being explicit about the theoretical framework: X claim works if you
> believe or agree with Y approach.
>
> My assumption in the cases of when something is complicated, or new to me
> is to agree with Sandro but to add on a little more: if he tells me someone
> is good at something, I'll likely think that someone is good, but what's
> driving this is trust from experience in his knowledge about certain things
> at certain times at certain topics (back to the framework or approach).
>
> Thoughts?
>
> One article that I recently came across seems related --  I just started
> working through it -- is "Beyond subjective and objective in statistics" by
> Andrew Gelman and Christian Hennig with a number of responses including by
> L.A. Paul so sharing in case of interest
> https://www.lapaul.org/papers/objectSubjectPerspectives.pdf
>
> --connie
>
> On Tue, Aug 17, 2021 at 10:53 PM Sandro Hawke <sandro@hawke.org> wrote:
>
>> It seems to me we can unify these views using credibility networks. We
>> can let anybody say anything about anything, as long as we only propagate
>> that content only along credibility network links. I'll simplify a bit
>> here, saying a "good" source is one which should be believed or one which
>> has interesting and non-harmful content.
>>
>> So let me see content from sources I've personally assessed as "good",
>> and also from sources my software predicts will be "good".  If I say
>> Clarence is good, and Clarence says Darcy is good, and Darcy says Edward is
>> good, then show me Edward's content, sure.
>>
>> On the other hand, if there is no one in my network vouching for Edward
>> in any way, I'm not going to see his content. Essentially, total strangers
>> -- people with whom I have no positive connection, direct or indirect --
>> are blocked by default. I'm talking here about content appearing in search
>> results, news feeds, comments, annotations, etc.  If I ask for something
>> specifically by URL, that's a different matter. Whoever gave me that URL is
>> essentially vouching for the content. If they give a link to bad content, I
>> can push back.
>>
>> This general approach subsumes the trust-the-elites model. If someone
>> only says they trust pulitzer.org, then they'll get an old-media/elite
>> view of the available content.  If they only say they trust infowars.com,
>> they'll get a very different view.
>>
>> My hope is most people have an assortment of sources they find credible
>> and the software can help them flag where the sources disagree.
>>
>> (This is what I was prototyping in trustlamp. Many details remain to be
>> solved.)
>>
>>     -- Sandro
>>
>>
>>
>> On 8/17/21 8:46 PM, Annette Greiner wrote:
>>
>> I don’t think I have the solution, but I offered my comment to help
>> better define what would be a reasonable solution. Another way to think
>> about it is that the signal should not be game-able. As for what you refer
>> to as “elites” and “hierarchies”,  I have no problem with harnessing
>> expertise to fight misinformation. Turning up the volume does not improve
>> the signal/noise ratio.
>> -Annette
>>
>> On Aug 17, 2021, at 2:44 PM, Bob Wyman <bob@wyman.us> wrote:
>>
>> On Tue, Aug 17, 2021 at 4:37 PM Annette Greiner <amgreiner@lbl.gov>
>> wrote:
>>
>>> I don’t think this is a wise approach at all.
>>>
>> Can you propose an alternative that does not simply formalize the status
>> of existing elites and thus strengthen hierarchies in public discourse? For
>> instance, the existing Credibility Signals
>> <https://credweb.org/reviewed-signals/> (date-first-archived,
>> awards-won, ..) would seem to provide useful information about only a tiny
>> portion of the many speakers on the Web. By focusing on the output of
>> awards-granting organizations, while not providing signals usable by
>> others, they empower that one group of speakers (those who grant awards)
>> over the rest of us. Can you propose a mechanism that allows my voice, or
>> yours, to have some influence in establishing credibility?
>>
>> We are seeing now that fraudsters and misinformation dealers are able to
>>> gain traction because there is so little barrier to their reaching high
>>> numbers of readers.
>>>
>> Today, the "bad" folk are able to speak without fear of rebuttal. Neither
>> the fact-checking organizations nor the platforms for speech seem to have
>> either the resources needed, or the motivation required, to usefully remark
>> on the credibility of more than an infinitesimal portion of public speech.
>> How can we possibly counterbalance the bad-speakers without enabling others
>> to rebut their statements?
>>
>> In any case, the methods I sketched concerning Alice's statements would
>> empower formal fact checkers as well as individuals, For instance, a
>> "climate fact-checking" organization would be able to do a Google search
>> for "hydrogen 'only water-vapor
>> <https://www.google.com/search?q=hydrogen+%22only+water-vapor%22>'," and
>> then, after minimal checking, annotate each of the hundreds of such
>> statements with a common, well formed rebuttal that would be easily
>> accessed by readers. Organizations could also set up prospective searches,
>> such as a Google Alert, that would notify them of new instances of false
>> claims and enable rapid response to their proliferation. I think this would
>> be useful. Do you disagree?
>>
>> Any real solution must not make it just as easy to spread misinformation
>>> as good information.
>>>
>> I have rarely seen a method for preventing bad things that doesn't also
>> prevent some good. The reality is that the most useful response to bad
>> speech is more speech. Given more speech, we can discover methods to assist
>> in the process of separating the good from the bad. But, if we don't
>> provide the means to make alternative claims, there is little we can do
>> with the resulting silence. False claims will stand if not rebutted.
>>
>> It must yield a signal with much much less noise than the currently
>>> available signals.
>>>
>> What "currently available signals?" Other than platform provided
>> moderation and censorship, what is there?
>>
>> Increasing the level of he-said/she-said doesn’t help determine what is
>>> reliable information. Adding to the massive amounts of junk is not the
>>> answer.
>>> -Annette
>>>
>>> On Aug 16, 2021, at 11:52 AM, Bob Wyman <bob@wyman.us> wrote:
>>>
>>> The thrust of my post is that we should dramatically enlarge the
>>> universe of those who make such claims to include all users of the
>>> Internet. The result of enabling every user of the Web to produce and
>>> discover credibility signals will be massive amounts of junk, but also a
>>> great many signals that you'll be able to use to filter, analyze, and
>>> reason about claims and the subjects of claims.
>>>
>>>
>>
>>
>
> --
> connie moon sehat
> connieimdialog@gmail.com
> https://linkedin.com/in/connieatwork
> PGP Key ID: 0x95DFB60E
>
>

-- 
J. Gregory McVerry, PhD
Assistant Professor
Southern Connecticut State University
twitter: jgmac1106