Re: diffusion tech

Wrong - image generation uses diffusion to propagate constraints in the latent semantics, e.g. the folds in fabric.

> On 26 Dec 2022, at 12:04, Paola Di Maio <paoladimaio10@gmail.com> wrote:
> 
> Thank you Dave
> this is where we have different views
> 
> diffusion is one thing, and latent semantics is another
> you can study/apply them together, but you can learn about them separately
> 
> since you point out that the article in the media which explains what is diffusion in relation to the discussions on this list and elsewhere (for example re AI generated árt')
>  a primer can simply summarize existing knowledge
> (for example, from new PhD theses as you point to) in an easily readable format
> 
> Luckily,  we do not have to agree on what goes into a primer - it only means
> we may have to work separately on different versions of it :-)
> 
> we can always compare notes if and when we get to talk in person
> until then. thanks for the pointers, and happy reading
> 
> 
> On Mon, Dec 26, 2022 at 8:00 PM Dave Raggett <dsr@w3.org <mailto:dsr@w3.org>> wrote:
>> There are plenty of resources for learning how to work with neural networks, but very little on manipulation of latent semantics, so a primer would be premature.
>> 
>> A likely way forward will be to train a secondary network to generate detailed descriptions of an image composition from the latent semantics, and to then use that in reverse to apply transformations. e.g. to change the colour of a given disc from green to blue and to add a yellow square. One complication is that the latent semantics are spread across layers. There is a loose similarity to natural language translation in which involves a stack of layers than start from syntactic details, transition to semantics then to syntactic details, before the word tokens for the target language.
>> 
>> One of the challenges for working with neural networks is the mathematics, so it is not for the faint hearted!
>> 
>>> On 25 Dec 2022, at 12:40, Paola Di Maio <paoladimaio10@gmail.com <mailto:paoladimaio10@gmail.com>> wrote:
>>> 
>>> Thank you very much Dave
>>> it will be very interesting to learn more in depth about the topic
>>> Having read a few PhD theses, I suspect they are not the easiest of reads 
>>> But maybe, we could put together a primer, with the help of others on this list
>>> on the topic. I d like to do something along those lines 
>>> PDM
>>> 
>>> On Sun, Dec 25, 2022 at 7:52 PM Dave Raggett <dsr@w3.org <mailto:dsr@w3.org>> wrote:
>>>> 
>>>>> On 25 Dec 2022, at 01:45, Paola Di Maio <paola.dimaio@gmail.com <mailto:paola.dimaio@gmail.com>> wrote:
>>>>> 
>>>>> Good read, 
>>>>> would be nice to frame diffusion in the context of KR
>>>>> 
>>>>> https://techcrunch.com/2022/12/22/a-brief-history-of-diffusion-the-tech-at-the-heart-of-modern-image-generating-ai/
>>>> 
>>>> 
>>>> That article doesn’t really explain how image generators actually work, as “de-noising” is little more than a buzzword.  Image generators learn how to decompose images at different levels of abstraction, e.g. at a lower level, using Gabor filters for modelling textures, and at a higher level by knowing something about typical dogs and cats.  Text prompts are mapped to concepts and combined with noise to stochastically generate details in an iterative process that diffuses constraints across the image, working in the space of latent semantics, making design choices all the way, before a final stage which takes latent semantics as instructions to generate the image pixels.
>>>> 
>>>> I recommend reading Steven Derby’s Ph.D thesis as he has done extensive work on determining what knowledge is exposed at different layers in neural networks, see:
>>>> 
>>>>  https://pure.qub.ac.uk/en/studentTheses/interpretable-semantic-representations-from-neural-language-model
>>>> 
>>>> I am hoping to start work next year on ways to directly manipulate latent semantics in artificial neural networks.  In principle, this should pave the way to enabling artists to work collaboratively with image generators, allowing the artist to make suggestions to refine a compositions in an iterative creative process.
>>>> 
>>>> A bigger challenge is to introduce richer knowledge, e.g. to ensure that image compositions embody causal constraints, and that people have four fingers and a thumb on each hand! How can we combine multiple sources of knowledge and ways of reasoning to support that? This is likely to require a paradigm shift that introduces sequential reasoning and continuous learning, and will introduce self-awareness along the way!
>>>> 
>>>> Dave Raggett <dsr@w3.org <mailto:dsr@w3.org>>
>>>> 
>>>> 
>>>> 
>> 
>> Dave Raggett <dsr@w3.org <mailto:dsr@w3.org>>
>> 
>> 
>> 

Dave Raggett <dsr@w3.org>