Re: credibility networks (was Re: Is Alice, or her post, credible? (A really rough use case for credibility signals.))

We've been working for a few years on this kind of trust network.  I 
recognize the subject-dependence of trust, but I think that trying to 
work that into systems being developed now is too ambitious.  Right now 
the value of a trust network can be demonstrated more effectively by 
starting with a simpler system that works in terms of generic 
credibility rather than subject-specific.  What you want are people who 
know what they know and don't claim to know more.   Yes, you'll lose out 
on your friend who knows everything about global warming but is 
anti-vax, but I think there are enough generally trustworthy individuals 
to drive a network of assessments.

On 8/18/2021 9:46 AM, connie im dialog wrote:
> As an additional thought, perhaps to bridge the exchange between 
> Annette and Bob, and Sandro: one aspect that I see missing in the 
> scenario below is the underlying knowledge/perspective framework or 
> approach that ties signals together: could be understood as a schema 
> or rubric.  This is a different way to tie signals together from trust 
> networks, and is probably underlying those relationships.
>
> What I mean by this is: all of the signals proposed are meant to be 
> understood as potential indications of credibility, but they only gain 
> meaning when some of them brought together in a specific interpretive 
> framework. Implicit in the development of many of the current signals 
> proposed is belief, or trust, in a scientific method of evidence and 
> evaluation of claims using methods such as verifiability. It's also 
> tied to things like expertise and the development of professions.
>
> This framework of knowledge is different than a moral order that 
> trusts inherited wisdom, or tradition, for example.  (I'm going to 
> sidestep the elites for now since the power dynamic depends on what 
> kind of elite one is.) Just because they are different does mean that 
> they can't in fact share one or more signals, but the dominance of 
> certain signals over others I think varies.  And because we aren't 
> always consistent, we may hold both of these or more frameworks given 
> a certain context or topic.
>
> So I guess I see Bob's suggestion as much in the line of a number of 
> crowdsourced wisdom projects, which can be valuable.  When you think 
> of historical or even current examples, such as genocide reporting, 
> it's very critical to include as many on-the-ground reports as 
> possible, even as those claims also need to be validated as much as 
> possible. In these contexts, there are many indications of what makes 
> for credible witness reports which isn't the same as expertise.
>
> But in some cases, on some topics, you can't go with any crowd 
> <https://wearecommons.us/crowd-wisdom-public-wisdom-regarding-misinformation-at-large/>. 
> That is at least if you hold to for example a scientific method of 
> evaluation and validation.  As with Annette, I have no problem with 
> deferring to expertise understood in this framework, and think it's 
> even worth being explicit about the theoretical framework: X claim 
> works if you believe or agree with Y approach.
>
> My assumption in the cases of when something is complicated, or new to 
> me is to agree with Sandro but to add on a little more: if he tells me 
> someone is good at something, I'll likely think that someone is good, 
> but what's driving this is trust from experience in his knowledge 
> about certain things at certain times at certain topics (back to the 
> framework or approach).
>
> Thoughts?
>
> One article that I recently came across seems related -- I just 
> started working through it -- is "Beyond subjective and objective in 
> statistics" by Andrew Gelman and Christian Hennig with a number of 
> responses including by L.A. Paul so sharing in case of interest 
> https://www.lapaul.org/papers/objectSubjectPerspectives.pdf 
> <https://www.lapaul.org/papers/objectSubjectPerspectives.pdf>
>
> --connie
>
> On Tue, Aug 17, 2021 at 10:53 PM Sandro Hawke <sandro@hawke.org 
> <mailto:sandro@hawke.org>> wrote:
>
>     It seems to me we can unify these views using credibility
>     networks. We can let anybody say anything about anything, as long
>     as we only propagate that content only along credibility network
>     links. I'll simplify a bit here, saying a "good" source is one
>     which should be believed or one which has interesting and
>     non-harmful content.
>
>     So let me see content from sources I've personally assessed as
>     "good", and also from sources my software predicts will be
>     "good".  If I say Clarence is good, and Clarence says Darcy is
>     good, and Darcy says Edward is good, then show me Edward's
>     content, sure.
>
>     On the other hand, if there is no one in my network vouching for
>     Edward in any way, I'm not going to see his content. Essentially,
>     total strangers -- people with whom I have no positive connection,
>     direct or indirect -- are blocked by default. I'm talking here
>     about content appearing in search results, news feeds, comments,
>     annotations, etc.  If I ask for something specifically by URL,
>     that's a different matter. Whoever gave me that URL is essentially
>     vouching for the content. If they give a link to bad content, I
>     can push back.
>
>     This general approach subsumes the trust-the-elites model. If
>     someone only says they trust pulitzer.org <http://pulitzer.org>,
>     then they'll get an old-media/elite view of the available
>     content.  If they only say they trust infowars.com
>     <http://infowars.com>, they'll get a very different view.
>
>     My hope is most people have an assortment of sources they find
>     credible and the software can help them flag where the sources
>     disagree.
>
>     (This is what I was prototyping in trustlamp. Many details remain
>     to be solved.)
>
>         -- Sandro
>
>
>
>     On 8/17/21 8:46 PM, Annette Greiner wrote:
>>     I don’t think I have the solution, but I offered my comment to
>>     help better define what would be a reasonable solution. Another
>>     way to think about it is that the signal should not be game-able.
>>     As for what you refer to as “elites” and “hierarchies”,  I have
>>     no problem with harnessing expertise to fight misinformation.
>>     Turning up the volume does not improve the signal/noise ratio.
>>     -Annette
>>
>>>     On Aug 17, 2021, at 2:44 PM, Bob Wyman <bob@wyman.us
>>>     <mailto:bob@wyman.us>> wrote:
>>>
>>>     On Tue, Aug 17, 2021 at 4:37 PM Annette Greiner
>>>     <amgreiner@lbl.gov <mailto:amgreiner@lbl.gov>> wrote:
>>>
>>>         I don’t think this is a wise approach at all.
>>>
>>>     Can you propose an alternative that does not simply formalize
>>>     the status of existing elites and thus strengthen hierarchies
>>>     in public discourse? For instance, the existing Credibility
>>>     Signals <https://credweb.org/reviewed-signals/>
>>>     (date-first-archived, awards-won, ..) would seem to provide
>>>     useful information about only a tiny portion of the many
>>>     speakers on the Web. By focusing on the output of
>>>     awards-granting organizations, while not providing signals
>>>     usable by others, they empower that one group of speakers (those
>>>     who grant awards) over the rest of us. Can you propose a
>>>     mechanism that allows my voice, or yours, to have some influence
>>>     in establishing credibility?
>>>
>>>         We are seeing now that fraudsters and misinformation dealers
>>>         are able to gain traction because there is so little barrier
>>>         to their reaching high numbers of readers.
>>>
>>>     Today, the "bad" folk are able to speak without fear of
>>>     rebuttal. Neither the fact-checking organizations nor the
>>>     platforms for speech seem to have either the resources needed,
>>>     or the motivation required, to usefully remark on the
>>>     credibility of more than an infinitesimal portion of public
>>>     speech. How can we possibly counterbalance the bad-speakers
>>>     without enabling others to rebut their statements?
>>>
>>>     In any case, the methods I sketched concerning Alice's
>>>     statements would empower formal fact checkers as well as
>>>     individuals, For instance, a "climate fact-checking"
>>>     organization would be able to do a Google search for "hydrogen
>>>     'only water-vapor
>>>     <https://www.google.com/search?q=hydrogen+%22only+water-vapor%22>',"
>>>     and then, after minimal checking, annotate each of the hundreds
>>>     of such statements with a common, well formed rebuttal that
>>>     would be easily accessed by readers. Organizations could also
>>>     set up prospective searches, such as a Google Alert, that would
>>>     notify them of new instances of false claims and enable rapid
>>>     response to their proliferation. I think this would be useful.
>>>     Do you disagree?
>>>
>>>         Any real solution must not make it just as easy to spread
>>>         misinformation as good information.
>>>
>>>     I have rarely seen a method for preventing bad things that
>>>     doesn't also prevent some good. The reality is that the most
>>>     useful response to bad speech is more speech. Given more speech,
>>>     we can discover methods to assist in the process of separating
>>>     the good from the bad. But, if we don't provide the means to
>>>     make alternative claims, there is little we can do with the
>>>     resulting silence. False claims will stand if not rebutted.
>>>
>>>         It must yield a signal with much much less noise than the
>>>         currently available signals.
>>>
>>>     What "currently available signals?" Other than platform provided
>>>     moderation and censorship, what is there?
>>>
>>>         Increasing the level of he-said/she-said doesn’t help
>>>         determine what is reliable information. Adding to the
>>>         massive amounts of junk is not the answer.
>>>         -Annette
>>>
>>>>         On Aug 16, 2021, at 11:52 AM, Bob Wyman <bob@wyman.us
>>>>         <mailto:bob@wyman.us>> wrote:
>>>>
>>>>         The thrust of my post is that we should dramatically
>>>>         enlarge the universe of those who make such claims to
>>>>         include all users of the Internet. The result of enabling
>>>>         every user of the Web to produce and discover credibility
>>>>         signals will be massive amounts of junk, but also a great
>>>>         many signals that you'll be able to use to filter, analyze,
>>>>         and reason about claims and the subjects of claims.
>>>
>>
>
>
>
> -- 
> connie moon sehat
> connieimdialog@gmail.com <mailto:connieimdialog@gmail.com>
> https://linkedin.com/in/connieatwork 
> <https://linkedin.com/in/connieatwork>
> PGP Key ID: 0x95DFB60E