Real-time Fact-checking Services and Dialogue Systems

Introduction

The topics of real-time fact-checking services and AI dialogue systems are broached and some opinions are shared for purposes of discussion.

Crowdsourced Fact-checking Resources

Are crowdsourced real-time fact-checking services possible? What might the design criteria be for such resources? Can such resources be of use to AI dialogue systems?

Determining the truth value of a given statement – or selecting one statement from a set of mutually exclusive statements – might not always be possible for a crowd at an instant. The instantaneous state of and, in some cases, resolution of, fact-checking processes might be such that multiple mutually-exclusive alternative statements are each supported by evidence and argumentation.

Crowdsourced real-time fact-checking resources should facilitate evidence-based argumentation for elements of sets of mutually exclusive statements as a component of providing groups with processes to determine the factuality of statements. There may be other means of clustering statements beyond mutual exclusivity.

Crowdsourced real-time fact-checking resources should support non-anonymous users as well as verified users, non-anonymous users who, for example, are journalists or fact-checkers.

Crowdsourced real-time fact-checking resources should allow users to opt into receiving notifications whenever a particular statement or statement-cluster is updated and AI dialogue systems should similarly be able to subscribe to such events.

Crowdsourced real-time fact-checking resources should support machine-utilizable output formats, output formats beyond HTML.

Efficiencies and Optimizations Regarding Dialogue Systems and Knowledgebases with Remote Fact-checking Services

At an instant, does a given topic require a dialogue system to poll one or more real-time fact-checking resources? Not every topic that a dialogue system might speak about would seem to require polling the data of one or more fact-checking resources. If dialogue systems could determine which topics were volatile, contentious or dynamic, and if these determinations were cacheable for durations, then dialogue systems could more efficiently provide their services at scale while making use of real-time fact-checking services.

It might be possible to determine the volatility, contentiousness, or dynamicism of a given topic by checking on or receiving notifications with respect to the quantity of recent activity for a topic on one or more fact-checking resources. Dialogue systems could asynchronously seek or receive updates regarding this data to best produce responses for users.

Dialogue System Behaviors

Which behaviors should dialogue systems exhibit if asked about volatile, contentious or dynamic topics? For instance, topics of some unfolding discussion on one or more fact-checking resources?

These scenarios might arise as users ask dialogue systems questions about topics recently discussed or debated, e.g. in the news.

Some options are indicated:

Option 1: I Don’t Know

A dialogue system could indicate that it doesn’t know the answer.

Option 2: I Need More Time to Answer that Question

A dialogue system could indicate that the matter is being reviewed by fact-checkers and somehow provide an estimate of when it should be able to answer the question. A dialogue system could ask a user if they desire a notification, including within specified hours, with respect to updates and could provide notifications to users via one of a number of communication channels.

Option 3: Refer Users to Fact-checking Resources

A dialogue system could refer the user to Web materials, for example to the unfolding discussion on a fact-checking resource. Multimodal dialogue systems could present users with Web content.

Option 4: Explain Alternatives and Possibilities

A dialogue system could, in particular when a fact-checking resolution appears to be disagreement, explain the elements of a set of mutually exclusive alternatives and provide the supporting argumentation for each. “Some are indicating X, while others Y.” Questions include how best to synthesize the output rhetoric in these instances, how best to sort the alternatives and possibilities, how best to specify which parties were supporting which alternatives and possibilities, and so forth.

Option 5: Decide

A dialogue system could review the argumentation and evidence supporting each element of a set of mutually exclusive statements and make a decision.

Conclusion

The topics of real-time fact-checking services and AI dialogue systems were broached and some opinions were shared for purposes of discussion.


Best regards,
Adam Sobieski