One tiny step towards AGI ...

I’ve recently spent time on learning Python and the deep learning library PyTorch. I had to overcome many personal misunderstandings, but am making good progress and now have a very crude version of GPT working. It makes it very clear how important large datasets are to the competence in handling language.

Some next steps are to scale up to large datasets, and to experiment with some ideas that might possibly enable smaller language models to do better. Without trying who knows!   After that, I look forward to working on ideas for novel architectures that mimic human cognition as set out in my recent talks.  This is an evolution from Generative AI to human-like AI inspired by how we think.

AGI is usually associated with super human intelligence, but that is largely missing the point, as there will be lots of uses for systems with modest intelligence that can run on everyday computer systems. Nvidia needn’t worry, as there will be plenty of demand for hardware acceleration.

p.s. as I proceed I will upload the software to GitHub, along with any datasets that aren’t already available elsewhere.

Cheers,

Dave Raggett <dsr@w3.org>