Re: Constitutional AI?

Thank you for the analysis

On Tue, Jan 10, 2023 at 7:15 PM Dave Raggett <dsr@w3.org> wrote:

> On 10 Jan 2023, at 04:56, Paola Di Maio <paola.dimaio@gmail.com> wrote:
>
> Not saying anything in favour or against this paper but
> related to AI KR , hence worth a glance
>
> Constitutional AI: Harmlessness from AI Feedback
> https://www.anthropic.com/constitutional.pdf
>
>
> Thanks for the pointer.
>
> The work starts by selecting an initial prompt from a dataset of harmful
> prompts. This is followed by prompts to request a self-critique, to request
> a revision, and then repeating the original harmful prompt to see if the
> response is an improvement on the original response. This process can be
> repeated with variations in the wording for the critique request and
> revision request. The results can later be used to refine the large
> language model to produce less harmful responses without the need for the
> critique request and revision request steps.
>
> This relies on the large language model having sufficient understanding
>  to generate its own critiques and revisions.  However, large language
> model tend to be weak on reasoning. In particular, the deep learning
> architectures used in LLMs, don’t learn to emulate reasoning functions.
> Instead, they find clever ways to learn statistical features that
> inherently exist in the reasoning problems.
>
> See:
> https://bdtechtalks.com/2022/06/27/large-language-models-logical-reasoning/
>
> That isn’t surprising as deep learning struggles with compositional
> generalisability, relying on vast datasets in compensation.
>
> See: https://royalsocietypublishing.org/doi/10.1098/rstb.2019.0307
> And more generally:
> https://link.springer.com/article/10.1007/s13748-021-00239-1
>
> Existing large language models support chain of thought reasoning, relying
> on working memory (the current neural activation values) to hold the
> context. There is no support for continuous learning and the equivalent of
> human short and long term memory. This is an opportunity for experimenting
> with new network architectures and training techniques that would reduce
> the need for vast datasets and better reflect what we know about human
> cognition.
>
> Dave Raggett <dsr@w3.org>
>
>
>
>