Re: credibility networks (was Re: Is Alice, or her post, credible? (A really rough use case for credibility signals.))

Or just stop using email to develop standards.....


I was sharing prior art that did something similar but with a different
stack.

I am agreement.

The signal list matters first and focusing on a trust of the author using
the above pathway outlined makes sense. Focus on the author or a concrete
of data for an MVP of a schema and vocab

(ps yes our signals could be used by journalist and OSINT in general)

On Wed, Aug 18, 2021 at 8:02 PM Subbu Vincent <svincent@scu.edu> wrote:

> Like many others (or all others here), the full-time job makes it hard to
> participate effectively.
>
> I agree with Scott Yates a hundred percent that it would be good to
> reverse the order of seed-to-fruit in our discourse. The novelty of
> discussing a novel idea is itself an immediately satisfying thing
> especially because of this wonderfully diverse group of people. Different
> journeys, solutions, thought leadership, etc. But ultimately it distracts
> us from a sense of rigorous sorting, follow-through, and completion. I'm
> all for developing a common vocabulary doc first and use that as a
> starting point for review of existing and new.
>
> Here's another reason this is urgent: I personally know of startups in
> this space that include credibility labels/signals in their offering. One
> of them simply takes LexisNexis' label for credibility and adds it to their
> site indicator. It's really odd, because L-N uses the label "unknown
> credibility" for local news sources in the US. Some really cool local
> journalism gigs would be upset if they discovered this and that this is
> getting relayed further downstream by an app. [Unintentionally done, btw,
> no real judgment]. Another app company is algorithmically calculating a
> cred score and recently took the word "credibility" out of the label itself
> (rebranding, but algo did not change). The names don't matter because it is
> a bit of a wild-west situation for anyone.
>
> Like Scott says, even publishing something we've come up with will be an
> OUTCOME that we can then use to start a review. I'm also happy to loop such
> app product leaders into the process to even self-vet their stuff against
> it. Some folks would welcome it just because they will a publicly built
> reference.
> ...
> (p.s. One quick addition: Afghanistan is all over the news; Yesterday,
> Sarah Chayes, who reported to the Chairman of the Joint Chiefs in
> Afghanistan '11 (and originally an American NPR reporter who turned a
> Kandahari Pashtun speaking activist for a while) wrote a very credible
> sounding blog article
> <https://www.sarahchayes.org/post/the-ides-of-august> about the
> corruption of the Afghan setup the US was "supporting" for years. She also
> refers to Pakistan-ISI's birthing/support of the Taliban (many from South
> Asia will reckon with this) and the long history there, coming to 2021
> including several claims about Hamid Karzai's own history with the Taliban.
> This is the sort of serious narrative-complicating stuff the industrial
> press is prone to miss in their rush to cover the pain and conflict with
> what I call "immediacy in sourcing". Just using a system to assess the
> credibility of experiential writers such as this would have been really
> helpful. I don't think any major media have called her on their shows yet,
> or given her an opinion journalism space yet. Perhaps she did not ask.
> https://www.sarahchayes.org/post/the-ides-of-august
>
>
>
> On Wed, Aug 18, 2021 at 3:21 PM Scott Yates <scott@journallist.net> wrote:
>
>> *With all due fear of taking over a thread to do something that seems
>> uncomfortably close to campaigning...*
>>
>> In the years since I first met Sandro and then joined this group, I've
>> seen what happened in this email thread happen a number of times. I've also
>> seen it happen in CredCo threads, MisinfoCon discussions, and other groups.
>>
>> What happens is essentially this: Someone proposes a new thing, let's
>> call it a MacGuffin
>> <https://urldefense.com/v3/__https://en.wikipedia.org/wiki/MacGuffin__;!!MLMg-p0Z!QMO909vToEbq61AcMwkLM4rbGVv1b86ehWEG5FeSAhFsSYtbNIoO4z6OyqWm0uI$>.
>> The person who proposes this MacGuffin explains it in great detail, yet is
>> a bit hand-wavy on some of the aspects, especially those about how it will
>> be supported, or how it would get adoption. That doesn't bother me at all.
>> If we didn't have people dreaming up new things we wouldn't have anything
>> new. But the reality is that these are just ideas, not actual initiatives.
>>
>> Then in Act II there is some discussion about the MacGuffin, talking
>> about the pros and the cons, etc.
>>
>> Then there is a pause. This pause comes because the group is made up of
>> people who all have full-time jobs. People who have jobs can't just drop
>> everything and put in the work needed to launch a new MacGuffin. In some
>> cases they can to some degree, which is what Sandro has done with
>> TrustLamp. He would be the first to tell you, I think, how hard that is.
>>
>> After the pause comes Act III, in which some people who are in the group
>> realize that much of the MacGuffin is a lot like what they are already
>> doing, and so they promote their own thing. Greg, you played that part this
>> time. I've played that part many times in the past.
>>
>> After that, the play is over, everyone goes home. And then after a while
>> it happens again.
>>
>> My suggestion (and the reason that I'm running for the chair) is that we
>> reverse the order of this play. Rather than: 1. Idea, 2. Evaluation, 3.
>> Look at related existing initiatives. We do this: 1. Look at existing
>> initiatives, 2. Evaluate them, 3. (with luck) Propose new ideas that fill
>> an existing and yet un-served need.
>>
>> To evaluate the current initiatives effectively, I would propose that
>> first we come up with some guidelines. To do that, we start with the
>> documents we have and that we have all agreed to. Then we turn those into a
>> framework for evaluation. Once we agree to that framework, we publish it.
>> That gives this group the relatively quick win of publishing something that
>> can be used by anyone as they are looking at existing initiatives, or are
>> thinking about starting something new. It essentially puts this group in
>> the middle of many conversations happening about disinformation. It will
>> help everyone to clarify what can actually help, whom it helps, how much it
>> helps, and how much downside there may be.
>>
>> Once we have that document, we can then decide if we want to meet
>> regularly and evaluate initiatives based on that document, or create a new
>> group to do that, or examine the whole landscape and figure out if there's
>> something that would be appropriate for this W3C group to try to do next.
>>
>> I say all this not to say that the original idea doesn't have merit, or
>> that any idea discussed here (including my own trust.txt) is great or
>> sucks. I'm just saying it would help the world, and each of us
>> individually, if we could evaluate ideas based on a common vocabulary.
>>
>> Thank you for reading.
>>
>> -Scott Yates
>> Founder
>> JournalList.net
>> <https://urldefense.com/v3/__http://JournalList.net__;!!MLMg-p0Z!QMO909vToEbq61AcMwkLM4rbGVv1b86ehWEG5FeSAhFsSYtbNIoO4z6ORkvBUNM$>,
>> caretaker of the trust.txt framework
>> 202-742-6842
>> Short Video Explanation of trust.txt
>> <https://urldefense.com/v3/__https://youtu.be/lunOBapQxpU__;!!MLMg-p0Z!QMO909vToEbq61AcMwkLM4rbGVv1b86ehWEG5FeSAhFsSYtbNIoO4z6O321cLn4$>
>>
>>
>> On Wed, Aug 18, 2021 at 11:19 AM Greg Mcverry <jgregmcverry@gmail.com>
>> wrote:
>>
>>> This document was also discussed at the verifiable credentials meeting
>>> this week:
>>> https://docs.google.com/presentation/d/1jn9DjM-wlZT1B9moBP23qhiB2FZc_H8oqXIZN222a9U/edit#slide=id.ge4a5a0fed4_0_18
>>> <https://urldefense.com/v3/__https://docs.google.com/presentation/d/1jn9DjM-wlZT1B9moBP23qhiB2FZc_H8oqXIZN222a9U/edit*slide=id.ge4a5a0fed4_0_18__;Iw!!MLMg-p0Z!QMO909vToEbq61AcMwkLM4rbGVv1b86ehWEG5FeSAhFsSYtbNIoO4z6OmLtT63Q$>
>>>
>>> I know there is a lot of crossover to the verified credential group here
>>> but I think if we are developing trust signals directed at the author we
>>> should develop our spec to align with VC.
>>>
>>> On Wed, Aug 18, 2021 at 12:10 PM Greg Mcverry <jgregmcverry@gmail.com>
>>> wrote:
>>>
>>>> We have been playing with the concept of vouch over in the indieweb
>>>> world: https://indieweb.org/Vouch
>>>> <https://urldefense.com/v3/__https://indieweb.org/Vouch__;!!MLMg-p0Z!QMO909vToEbq61AcMwkLM4rbGVv1b86ehWEG5FeSAhFsSYtbNIoO4z6OloRajg0$>
>>>>
>>>> Different stack since based on webmentions but the workflow pretty much
>>>> the same.
>>>>
>>>> The goal is to create semi-private posts for community members vouched
>>>> by others and as a trust network.
>>>>
>>>> XFN pretty defunct but I use rel="muse" on my poetry follower list as a
>>>> trust signal
>>>>
>>>> https://indieweb.org/XFN
>>>> <https://urldefense.com/v3/__https://indieweb.org/XFN__;!!MLMg-p0Z!QMO909vToEbq61AcMwkLM4rbGVv1b86ehWEG5FeSAhFsSYtbNIoO4z6OPAhhwSs$>
>>>>
>>>>
>>>>
>>>> On Wed, Aug 18, 2021 at 11:35 AM David Karger <karger@mit.edu> wrote:
>>>>
>>>>> We've been working for a few years on this kind of trust network.  I
>>>>> recognize the subject-dependence of trust, but I think that trying to work
>>>>> that into systems being developed now is too ambitious.  Right now the
>>>>> value of a trust network can be demonstrated more effectively by starting
>>>>> with a simpler system that works in terms of generic credibility rather
>>>>> than subject-specific.  What you want are people who know what they know
>>>>> and don't claim to know more.   Yes, you'll lose out on your friend who
>>>>> knows everything about global warming but is anti-vax, but I think there
>>>>> are enough generally trustworthy individuals to drive a network of
>>>>> assessments.
>>>>> On 8/18/2021 9:46 AM, connie im dialog wrote:
>>>>>
>>>>> As an additional thought, perhaps to bridge the exchange between
>>>>> Annette and Bob, and Sandro: one aspect that I see missing in the scenario
>>>>> below is the underlying knowledge/perspective framework or approach that
>>>>> ties signals together: could be understood as a schema or rubric.  This is
>>>>> a different way to tie signals together from trust networks, and is
>>>>> probably underlying those relationships.
>>>>>
>>>>> What I mean by this is: all of the signals proposed are meant to be
>>>>> understood as potential indications of credibility, but they only gain
>>>>> meaning when some of them brought together in a specific interpretive
>>>>> framework.  Implicit in the development of many of the current signals
>>>>> proposed is belief, or trust, in a scientific method of evidence and
>>>>> evaluation of claims using methods such as verifiability. It's also tied to
>>>>> things like expertise and the development of professions.
>>>>>
>>>>> This framework of knowledge is different than a moral order that
>>>>> trusts inherited wisdom, or tradition, for example.  (I'm going to sidestep
>>>>> the elites for now since the power dynamic depends on what kind of elite
>>>>> one is.) Just because they are different does mean that they can't in fact
>>>>> share one or more signals, but the dominance of certain signals over others
>>>>> I think varies.  And because we aren't always consistent, we may hold both
>>>>> of these or more frameworks given a certain context or topic.
>>>>>
>>>>> So I guess I see Bob's suggestion as much in the line of a number of
>>>>> crowdsourced wisdom projects, which can be valuable.  When you think of
>>>>> historical or even current examples, such as genocide reporting, it's very
>>>>> critical to include as many on-the-ground reports as possible, even as
>>>>> those claims also need to be validated as much as possible. In these
>>>>> contexts, there are many indications of what makes for credible witness
>>>>> reports which isn't the same as expertise.
>>>>>
>>>>> But in some cases, on some topics, you can't go with any crowd
>>>>> <https://urldefense.com/v3/__https://wearecommons.us/crowd-wisdom-public-wisdom-regarding-misinformation-at-large/__;!!MLMg-p0Z!QMO909vToEbq61AcMwkLM4rbGVv1b86ehWEG5FeSAhFsSYtbNIoO4z6OFLTC7oE$>.
>>>>> That is at least if you hold to for example a scientific method of
>>>>> evaluation and validation.  As with Annette, I have no problem with
>>>>> deferring to expertise understood in this framework, and think it's even
>>>>> worth being explicit about the theoretical framework: X claim works if you
>>>>> believe or agree with Y approach.
>>>>>
>>>>> My assumption in the cases of when something is complicated, or new to
>>>>> me is to agree with Sandro but to add on a little more: if he tells me
>>>>> someone is good at something, I'll likely think that someone is good, but
>>>>> what's driving this is trust from experience in his knowledge about certain
>>>>> things at certain times at certain topics (back to the framework or
>>>>> approach).
>>>>>
>>>>> Thoughts?
>>>>>
>>>>> One article that I recently came across seems related --  I just
>>>>> started working through it -- is "Beyond subjective and objective in
>>>>> statistics" by Andrew Gelman and Christian Hennig with a number of
>>>>> responses including by L.A. Paul so sharing in case of interest
>>>>> https://www.lapaul.org/papers/objectSubjectPerspectives.pdf
>>>>> <https://urldefense.com/v3/__https://www.lapaul.org/papers/objectSubjectPerspectives.pdf__;!!MLMg-p0Z!QMO909vToEbq61AcMwkLM4rbGVv1b86ehWEG5FeSAhFsSYtbNIoO4z6OWypNDI0$>
>>>>>
>>>>> --connie
>>>>>
>>>>> On Tue, Aug 17, 2021 at 10:53 PM Sandro Hawke <sandro@hawke.org>
>>>>> wrote:
>>>>>
>>>>>> It seems to me we can unify these views using credibility networks.
>>>>>> We can let anybody say anything about anything, as long as we only
>>>>>> propagate that content only along credibility network links. I'll simplify
>>>>>> a bit here, saying a "good" source is one which should be believed or one
>>>>>> which has interesting and non-harmful content.
>>>>>>
>>>>>> So let me see content from sources I've personally assessed as
>>>>>> "good", and also from sources my software predicts will be "good".  If I
>>>>>> say Clarence is good, and Clarence says Darcy is good, and Darcy says
>>>>>> Edward is good, then show me Edward's content, sure.
>>>>>>
>>>>>> On the other hand, if there is no one in my network vouching for
>>>>>> Edward in any way, I'm not going to see his content. Essentially, total
>>>>>> strangers -- people with whom I have no positive connection, direct or
>>>>>> indirect -- are blocked by default. I'm talking here about content
>>>>>> appearing in search results, news feeds, comments, annotations, etc.  If I
>>>>>> ask for something specifically by URL, that's a different matter. Whoever
>>>>>> gave me that URL is essentially vouching for the content. If they give a
>>>>>> link to bad content, I can push back.
>>>>>>
>>>>>> This general approach subsumes the trust-the-elites model. If someone
>>>>>> only says they trust pulitzer.org
>>>>>> <https://urldefense.com/v3/__http://pulitzer.org__;!!MLMg-p0Z!QMO909vToEbq61AcMwkLM4rbGVv1b86ehWEG5FeSAhFsSYtbNIoO4z6OKVqmLEU$>,
>>>>>> then they'll get an old-media/elite view of the available content.  If they
>>>>>> only say they trust infowars.com
>>>>>> <https://urldefense.com/v3/__http://infowars.com__;!!MLMg-p0Z!QMO909vToEbq61AcMwkLM4rbGVv1b86ehWEG5FeSAhFsSYtbNIoO4z6OlYeqvbo$>,
>>>>>> they'll get a very different view.
>>>>>>
>>>>>> My hope is most people have an assortment of sources they find
>>>>>> credible and the software can help them flag where the sources disagree.
>>>>>>
>>>>>> (This is what I was prototyping in trustlamp. Many details remain to
>>>>>> be solved.)
>>>>>>
>>>>>>     -- Sandro
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 8/17/21 8:46 PM, Annette Greiner wrote:
>>>>>>
>>>>>> I don’t think I have the solution, but I offered my comment to help
>>>>>> better define what would be a reasonable solution. Another way to think
>>>>>> about it is that the signal should not be game-able. As for what you refer
>>>>>> to as “elites” and “hierarchies”,  I have no problem with harnessing
>>>>>> expertise to fight misinformation. Turning up the volume does not improve
>>>>>> the signal/noise ratio.
>>>>>> -Annette
>>>>>>
>>>>>> On Aug 17, 2021, at 2:44 PM, Bob Wyman <bob@wyman.us> wrote:
>>>>>>
>>>>>> On Tue, Aug 17, 2021 at 4:37 PM Annette Greiner <amgreiner@lbl.gov>
>>>>>> wrote:
>>>>>>
>>>>>>> I don’t think this is a wise approach at all.
>>>>>>>
>>>>>> Can you propose an alternative that does not simply formalize the
>>>>>> status of existing elites and thus strengthen hierarchies in public
>>>>>> discourse? For instance, the existing Credibility Signals
>>>>>> <https://urldefense.com/v3/__https://credweb.org/reviewed-signals/__;!!MLMg-p0Z!QMO909vToEbq61AcMwkLM4rbGVv1b86ehWEG5FeSAhFsSYtbNIoO4z6OE3oViLk$>
>>>>>> (date-first-archived, awards-won, ..) would seem to provide useful
>>>>>> information about only a tiny portion of the many speakers on the Web. By
>>>>>> focusing on the output of awards-granting organizations, while not
>>>>>> providing signals usable by others, they empower that one group of speakers
>>>>>> (those who grant awards) over the rest of us. Can you propose a mechanism
>>>>>> that allows my voice, or yours, to have some influence in establishing
>>>>>> credibility?
>>>>>>
>>>>>> We are seeing now that fraudsters and misinformation dealers are able
>>>>>>> to gain traction because there is so little barrier to their reaching high
>>>>>>> numbers of readers.
>>>>>>>
>>>>>> Today, the "bad" folk are able to speak without fear of rebuttal.
>>>>>> Neither the fact-checking organizations nor the platforms for speech seem
>>>>>> to have either the resources needed, or the motivation required, to
>>>>>> usefully remark on the credibility of more than an infinitesimal portion of
>>>>>> public speech. How can we possibly counterbalance the bad-speakers without
>>>>>> enabling others to rebut their statements?
>>>>>>
>>>>>> In any case, the methods I sketched concerning Alice's statements
>>>>>> would empower formal fact checkers as well as individuals, For instance, a
>>>>>> "climate fact-checking" organization would be able to do a Google search
>>>>>> for "hydrogen 'only water-vapor
>>>>>> <https://urldefense.com/v3/__https://www.google.com/search?q=hydrogen**A22only*water-vapor*22__;KyUrJQ!!MLMg-p0Z!QMO909vToEbq61AcMwkLM4rbGVv1b86ehWEG5FeSAhFsSYtbNIoO4z6O2OVHpYQ$>',"
>>>>>> and then, after minimal checking, annotate each of the hundreds of such
>>>>>> statements with a common, well formed rebuttal that would be easily
>>>>>> accessed by readers. Organizations could also set up prospective searches,
>>>>>> such as a Google Alert, that would notify them of new instances of false
>>>>>> claims and enable rapid response to their proliferation. I think this would
>>>>>> be useful. Do you disagree?
>>>>>>
>>>>>> Any real solution must not make it just as easy to spread
>>>>>>> misinformation as good information.
>>>>>>>
>>>>>> I have rarely seen a method for preventing bad things that doesn't
>>>>>> also prevent some good. The reality is that the most useful response to bad
>>>>>> speech is more speech. Given more speech, we can discover methods to assist
>>>>>> in the process of separating the good from the bad. But, if we don't
>>>>>> provide the means to make alternative claims, there is little we can do
>>>>>> with the resulting silence. False claims will stand if not rebutted.
>>>>>>
>>>>>> It must yield a signal with much much less noise than the currently
>>>>>>> available signals.
>>>>>>>
>>>>>> What "currently available signals?" Other than platform provided
>>>>>> moderation and censorship, what is there?
>>>>>>
>>>>>> Increasing the level of he-said/she-said doesn’t help determine what
>>>>>>> is reliable information. Adding to the massive amounts of junk is not the
>>>>>>> answer.
>>>>>>> -Annette
>>>>>>>
>>>>>>> On Aug 16, 2021, at 11:52 AM, Bob Wyman <bob@wyman.us> wrote:
>>>>>>>
>>>>>>> The thrust of my post is that we should dramatically enlarge the
>>>>>>> universe of those who make such claims to include all users of the
>>>>>>> Internet. The result of enabling every user of the Web to produce and
>>>>>>> discover credibility signals will be massive amounts of junk, but also a
>>>>>>> great many signals that you'll be able to use to filter, analyze, and
>>>>>>> reason about claims and the subjects of claims.
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> connie moon sehat
>>>>> connieimdialog@gmail.com
>>>>> https://linkedin.com/in/connieatwork
>>>>> <https://urldefense.com/v3/__https://linkedin.com/in/connieatwork__;!!MLMg-p0Z!QMO909vToEbq61AcMwkLM4rbGVv1b86ehWEG5FeSAhFsSYtbNIoO4z6OdWjkdD0$>
>>>>> PGP Key ID: 0x95DFB60E
>>>>>
>>>>>
>>>>
>>>> --
>>>> J. Gregory McVerry, PhD
>>>> Assistant Professor
>>>> Southern Connecticut State University
>>>> twitter: jgmac1106
>>>>
>>>>
>>>>
>>>>
>>>
>>> --
>>> J. Gregory McVerry, PhD
>>> Assistant Professor
>>> Southern Connecticut State University
>>> twitter: jgmac1106
>>>
>>>
>>>
>>>
>
> --
>
> ------------------------------------------------------------------------------
> Subramaniam (Subbu) Vincent
> Director, Journalism and Media Ethics,
> Markkula Center for Applied Ethics, Santa Clara University
> My work
> <https://www.scu.edu/ethics/about-the-center/people/subramaniam-vincent/>
>  | LinkedIn
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.linkedin.com_in_subbuvincent&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=4DXVZEkHbjQ-GSaDRHRSNDk3G2EzTwJo3aSnh8e-ZcY&m=6ogjsD8d8iOY_KNFn9Gig-QVcztiMFMnBT9qyTW-vq0&s=aMnmKpFJDh-tZa7PgTYeiac1ojHgOVxkdwIdPTcN37E&e=>
>  | Twitter
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_subbuvincent&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=4DXVZEkHbjQ-GSaDRHRSNDk3G2EzTwJo3aSnh8e-ZcY&m=6ogjsD8d8iOY_KNFn9Gig-QVcztiMFMnBT9qyTW-vq0&s=_HcPyCH3sWsYYekDm-gobSdCatYz40X49JekeiwKwW0&e=>
>
>


-- 
J. Gregory McVerry, PhD
Assistant Professor
Southern Connecticut State University
twitter: jgmac1106