Re: How do we learn?

NB / FYI - old doc (not by me, part of an archive i made in 2007, re: related
work)

https://docs.google.com/presentation/d/0B_-AWWDVv3V2U00xel84QjFid0k/edit?usp=sharing&ouid=103119445892247666269&resourcekey=0-SGnRUlClzc9L_wK47Lvn8Q&rtpof=true&sd=true


On Fri, 22 Oct 2021 at 20:01, Dave Raggett <dsr@w3.org> wrote:

> I just read the following article:
>
> A new model of learning centers on bursts of neural activity that act as
> teaching signals — approximating backpropagation, the algorithm behind
> learning in AI.
>
>
>
> https://www.quantamagazine.org/brain-bursts-can-mimic-famous-ai-learning-strategy-20211018/
>
>
> This seems promising and others have noted:
>
> The brain is the perfect place to look for inspiration to develop more
> efficient neural networks. The inner workings of our synapses and neurons
> provide a glimpse at what the future of deep learning might look like.
>
>
> From a paper that surveys potential approaches:
>
>
> https://www.researchgate.net/publication/354889734_Training_Spiking_Neural_Networks_Using_Lessons_From_Deep_Learning/link/6152e7c1d2ebba7be752a26c/download
>
>
> The premise of back propagation is that error signals can be used to
> progressively improve the weights in multi-layer neural networks. However,
> children learn to recognise dogs and cats without being and presented with
> lots and lots of classified examples. Where does the error signal come from?
>
> There is a lot of interest in self-supervised learning, however, this
> often involves techniques such as predicting masked items in a sequence,
> where the software decides what to mask and knows what the masked items
> are, e.g. words from documents found on the web.
>
> Given lots of photos of dogs and cats, where only a few are classified, we
> need to find ways to generalise beyond the statistics in the classified
> examples. Very young children can make mistakes, but rapidly improve when
> their parents or carers correct them in what is essentially a single-shot
> process. That is qualitatively different from back propagation where
> learning takes place gradually over lots of examples.
>
> My guess is that this relates to a notion of salience and constructed
> models rather than the black-box model of deep learning.
>
> For an amusing take on that, see: “Motorist fined after CCTV confuses his
> number plate with woman’s T-shirt”
>
>
> https://www.theguardian.com/uk-news/2021/oct/18/motorist-fined-number-plate-t-shirt
>
>
> The street camera spotted what it thought was a car number plate in a bus
> lane, but in fact was a woman with some text across her clothing.  This is
> an example where the deep learning algorithm was trained to recognise
> letters and digits without any understanding of salient details. In this
> case, a simple fix would be to combine two deep learning solutions one to
> recognise cars and another for number plates.  Bath and North East Somerset
> Council are operating an AI system that is clearly inadequate in respect to
> automated fines.
>
> Other examples for vehicle recognition are good at recognising the
> presence of a vehicle, but lack any understanding of the components that
> make up the vehicle, e.g. wheels, headlights, windows, side mirrors, model
> of car, etc.
>
> We can’t afford to ignore salience and part-whole relationships in scene
> understanding and I think we need AI regulations that prohibit deployment
> of dumb deep learning!
>
> What do you think?
>
> See also: https://ec.europa.eu/commission/presscorner/detail/en/IP_21_1682
>
>
> p.s. Horizon Europe is seeking proposals on improving AI with a submission
> deadline next April for work on a new AI-enabled Cloud-edge framework
> (Cognitive Cloud). Is anyone interested in collaborating on that?  See:
>
>
> https://ec.europa.eu/info/funding-tenders/opportunities/portal/screen/opportunities/topic-details/horizon-cl4-2022-data-01-02
>
>
> Dave Raggett <dsr@w3.org> http://www.w3.org/People/Raggett
> W3C Data Activity Lead & W3C champion for the Web of things
>
>
>
>
>