Re: ChatGPT, ontologies and SPARQL

Dave,

Thank you and I agree with your points. I’m excited about the near future when scientists will better understand the emergence of abilities in large-scale artificial neural networks. A related topic is that of multi-task learning (https://en.wikipedia.org/wiki/Multi-task_learning).

I'm also excited about artificial neural networks interoperating with external knowledgebases. This interoperation could utilize SPARQL.

With respect to large-scale dialogue systems like ChatGPT, some topics that I previously considered in the contexts of intelligent tutoring systems include dialogue context, user modeling, and A/B testing.

Large-scale systems like ChatGPT are engaging with a large volume of users at an instant, over the course of a day, and with, I imagine, some task redundancy. Large-scale dialogue systems which can measure or infer quality of service and user satisfaction could vary their outputs, e.g., their framings or phrasings, over populations of users to explore whether, which, and why variations result in increased quality of service or user satisfaction.

Beyond providing users with buttons with which to obtain their feedbacks, e.g., a thumbs up button, scientists could also explore more clever means of measuring user comprehension and satisfaction. If a user exits a dialogue quickly after an answer is provided, was their question answered directly and to their satisfaction or, did they, instead, in frustration, seek another system to engage with? If a user responds to an AI system more rapidly in a dialogue, was the AI system’s previous content phrased in a more readable and comprehensible manner for that user and/or was the dialogue more engaging? We can also consider that video-chat-based dialogue systems should have more data to utilize towards ascertaining quality of service and user satisfaction during dialogues.

These techniques would be milestones on the journey to metadialogical capabilities, where dialogue systems could receive feedback about their dialogues in those dialogues.

In addition to the topics of semantic content curation and engineering (which might be otherwise-named subtopics of operating the systems), there are also to consider large-scale AI systems which can perform A/B testing over populations of users to maximize quality of service and user satisfaction. In my opinion, care should be taken to avoid the pitfalls of personalization, e.g., filter bubbles.


Best regards,
Adam

________________________________
From: Dave Raggett <dsr@w3.org>
Sent: Tuesday, January 24, 2023 4:18 AM
To: Adam Sobieski <adamsobieski@hotmail.com>
Cc: Adeel <aahmad1811@gmail.com>; public-aikr@w3.org <public-aikr@w3.org>
Subject: Re: ChatGPT, ontologies and SPARQL

Dropping back to AIKR ...


Scaling up language models has been shown to predictably improve performance and sample efficiency on a wide range of downstream tasks. This paper instead discusses an unpredictable phenomenon that we refer to as emergent abilities of large language models. We consider an ability to be emergent if it is not present in smaller models but is present in larger models. Thus, emergent abilities cannot be predicted simply by extrapolating the performance of smaller models. The existence of such emergence raises the question of whether additional scaling could potentially further expand the range of capabilities of language models.

A deeper understanding of how ChatGPT is able to generate its results should allow us to devise smaller and more climate friendly systems.  Practical applications don’t need the vast breadth of knowledge that ChatGPT got from scraping most of the web.

A deeper understanding will also facilitate research on fixing major limitations of large language models, e.g. continuous learning, integration of explicit domain knowledge, metacognition, introspection and better explanations that cite provenance, etc.

Dave Raggett <dsr@w3.org>