> On 23 Dec 2022, at 15:54, Owen Ambur <owen.ambur@verizon.net> wrote: > > Apparently, ChatGPT, which calls itself "Assistant" ChatGPT is a version of GPT3 that has been further trained to provide more meaningful answers. "Training language models to follow instructions with human feedback" https://arxiv.org/pdf/2203.02155.pdf The underlying technology is based upon training a neural network to generate text that is consistent with the prompt. Lower layers of the network deal with syntactic knowledge, whilst upper layers deal with semantic knowledge, see: Interpretable semantic representations from neural language models and computer vision, Steven Derby, 2022 https://pure.qub.ac.uk/en/studentTheses/interpretable-semantic-representations-from-neural-language-model Once the system is trained, thereafter its knowledge is fixed. Its working memory is the current activation levels of the artificial neurons in its network. This is all System 1, i.e. intuitive, opaque and lacking the ability to reason introspectively. The next evolutionary step will be to integrate support for System 2 by extending the network architecture to support rules that act on working memory (the latent semantics). Related challenges include the need to overcome catastrophic forgetting, and how to enable continuous learning involving a mix of working memory, short term memory and long term memory, mimicking the hippocampus and cortex. Sadly the media reports of ChatGPT, and DALL-E, etc. lack depth and critical understanding of the technical challenges. On the plus side, there are plenty of opportunities for further work, and progress is possible with the modest hardware available to most researchers. Dave Raggett <dsr@w3.org>
Received on Sunday, 25 December 2022 10:16:03 UTC