Re: generative KR 2. a step back

Large language models have a means to represent knowledge, but not in forms that are transparent to human inspection. I suspect that the role of transformers and multiple layers is essentially to enable linked chunks (sets of name/value pairs), and crucially to support a fuzzy representation in which symbols crystallise out of noisy statistics as learning proceeds, akin to how detailed images iteratively appear out of the noise in successive rounds of de-noising for text to image generators, as can be seen with the Würstchen demo: 

 https://huggingface.co/spaces/warp-ai/Wuerstchen

This suggests the potential to extract explicit symbolic knowledge, at least where sufficient crystallisation has occurred.  Symbolic knowledge is in my opinion key to ensuring semantic interoperability across systems.  I expect we will be able to create AI tools that lighten the burden for knowledge engineering at scale, changing the job for the human partner to that of a mentor guiding learning, including preparation of datasets and for evaluating competence.  This will involve different kinds of learning.

I expect  that this will be much easier as we enable neural networks to support continual learning and reflective cognition, which are impractical for current systems. That entails some level of sentience.


> On 20 Sep 2023, at 12:14, Paola Di Maio <paoladimaio10@gmail.com> wrote:
> 
> Thank you Dave for picking up the challenge
> Lets hope we can stay friends throughout :-)
> The article was just providing some reading material, it is not representative of the broader topic
> 
> Yes, I need to provide a definition, at the same stime I am still just beginning to formulate the question, so bear with me
> 
> Given the preferred choice of definition for GenAI, and for KR, I would put the two together.  So if we accept that GAI is, say
> 
> Generative artificial intelligence (also generative AI or GenAI) is artificial intelligence capable of generating text, images, or other media, using generative models.(Wikipedia)   
> 
> AND for KR    explicit knowledge used by intelligent algorithms for the purpose of computation (PDM)
> //other definitions welcome//  
> 
> one could say that 
> 
> Generative KR is the explicit knowledge used by intelligents algorithm to run generative models
> 
> Obviously there are a number of areas that need to be tackled further
> 1. if we agree that GenAI is based on ML, it can be argued  that ML does not use KR much
> But others disagree, including myself
> 2. At what stage does the KR for GenAI come into being? before, during or after the generation process?
> 
> takes please

Dave Raggett <dsr@w3.org>