Re: stable diffusion as a DeepFake machine

> On 29 Oct 2022, at 04:45, Paola Di Maio <paola.dimaio@gmail.com> wrote:
> 
> Dave R shared on the cogai mailing list a link to Stabled Diffusion
> https://huggingface.co/spaces/stabilityai/stable-diffusion <https://huggingface.co/spaces/stabilityai/stable-diffusion> 
> It is one of the many many versions available apparently, 
> I played with it and commented on a related post
> 
> Totally by synchronicity in the very remote village I live in, last night I walked into someone random  (feeding the village cats) and the topic of stable diffusion came up, I was told  Its becoming very popular, in different flavours, although some people are upset  apparently
> 
> It is relevant to KR

I believe that taxonomic, causal and behavioural knowledge will be key to fixing some of the glaring errors that Stable Diffision, and DALL-E and their kind produce.

> From a AI KR perspective, however innocent fun Stable_Diffiusion may be, its output can be misleading, By not exposing the source data and algorithm to generate the image, its outcome can be used to mislead people into thinking this is some kind of original artwork

The training data and the algorithms are open source.  The challenge for attribution is that generated images use data from across a vast number of images in the training data, which makes meaningful attribution very challenging.  I would recommend that the prompt text be embedded as part of the generated image’s metadata. You can get a feeling for which images relate to a given prompt with: https://rom1504.github.io/clip-retrieval/ <https://rom1504.github.io/clip-retrieval/>. This shows that even when the prompt names a single artist, the software considers the work by many other artists with approximately similar styles.

> Only KR can  identify, expose and prevent deepfakes

Really?  Please explain.

Stable Diffusion along with other similar software can take an image along with a text prompt as its input.  Stable Diffusion doesn’t preserve the likeness of someone’s face when used in that way.  However, there is plenty of other work on how to improve on that, e.g. Google’s DreamBooth which takes several images of the same subject as input in order to built a consistent 3D model. Other software can generate cleaned up high resolution images from noisy low resolution images. The technology is likely to improve considerably over the next few years and extend from images to video.

Deep fakes are clearly immoral, but malicious actors will strip out metadata, and seek to deceive people, just as is the case for cyber attackers. We can expect a predator/prey evolutionary cycle as malicious actors seek to overcome the improvements in software for detecting deep fakes.

p.s. generative-adversarial techniques for machine learning are based upon simultaneously training a generator to trick a critic into thinking a machine generated image is human generated, whilst the critic seeks to tell them apart. Both agents get progressively better, given a large enough set of training data, e.g. billion of image/text pairs.

Dave Raggett <dsr@w3.org>