Re: My Chat with ChatGPT

Hello,

It is basically a dumb mimicry model trained on large amounts of data with
a lot of marketing hype.

Thanks,
Adeel

On Sun, 25 Dec 2022 at 10:16, Dave Raggett <dsr@w3.org> wrote:

>
>
> On 23 Dec 2022, at 15:54, Owen Ambur <owen.ambur@verizon.net> wrote:
>
> Apparently, ChatGPT, which calls itself "Assistant"
>
>
> ChatGPT is a version of GPT3 that has been further trained to provide more
> meaningful answers.
>
> "Training language models to follow instructions with human feedback"
> https://arxiv.org/pdf/2203.02155.pdf
>
> The underlying technology is based upon training a neural network to
> generate text that is consistent with the prompt. Lower layers of the
> network deal with syntactic knowledge, whilst upper layers deal with
> semantic knowledge, see:
>
> Interpretable semantic representations from neural language models and
> computer vision, Steven Derby, 2022
>
> https://pure.qub.ac.uk/en/studentTheses/interpretable-semantic-representations-from-neural-language-model
>
> Once the system is trained, thereafter its knowledge is fixed. Its working
> memory is the current activation levels of the artificial neurons in its
> network.  This is all System 1, i.e. intuitive, opaque and lacking the
> ability to reason introspectively.  The next evolutionary step will be to
> integrate support for System 2 by extending the network architecture to
> support rules that act on working memory (the latent semantics).  Related
> challenges include the need to overcome catastrophic forgetting, and how to
> enable continuous learning involving a mix of working memory, short term
> memory and long term memory, mimicking the hippocampus and cortex.
>
> Sadly the media reports of ChatGPT, and DALL-E, etc. lack depth and
> critical understanding of the technical challenges.  On the plus side,
> there are plenty of opportunities for further work, and progress is
> possible with the modest hardware available to most researchers.
>
> Dave Raggett <dsr@w3.org>
>
>
>
>