《扩散模型》最新教程

Laura Sánchez GarcíaJulio Antonio Soto Vicente IE University (C4_466671 - Advanced Artificial Intelligence) Outline Denoising Diffusion Probabilistic Models Advancements and improvements 4Large diffusion models Beyond image generation Intro A (certainly not complete) list: •Latent Variable models (incl. VAEs)•Autoregressive models (incl. GPT-style Language Models)•GANs•Flow-based models (incl. Normalizing Flows)•Energy-Based Models (incl. Score-based models)•Diffusion models(kind of mix of all previous points)•Combinations Image generation ”A photo of a Corgi dog riding a bike inTimes Square. It is wearing sunglasses anda beach hat.” Denoising Diffusion Probabilistic Models Denoising Diffusion Probabilistic Models Outline •The forward process•The Nice™ property•The reverse process•Loss function•Training algorithm•The model•Sampling algorithm Denoising Diffusion Probabilistic Models (DDPMs) Denoising Diffusion Probabilistic Models (DDPMs) ”Creating noise from data is easy;creating data from noise is generative modeling.” Song et al., 2020 Denoising Diffusion Probabilistic Models (DDPMs) ”Creating noise from data is easy;creating data from noise is generative modeling.” Song et al., 2020 DDPMs progressively generate images out of noise Denoising Diffusion Probabilistic Models (DDPMs) In order for a model to learn how to do that, we need a process to generatesuitable training data in the form of(noise,image)pairs Denoising Diffusion Probabilistic Models (DDPMs) In order for a model to learn how to do that, we need a process to generatesuitable training data in the form of(noise,image)pairs We will refer to the process of creating the training data as theforwarddiffusionprocess, which will progressively make an image noisier Denoising Diffusion Probabilistic Models (DDPMs) In order for a model to learn how to do that, we need a process to generatesuitable training data in the form of(noise,image)pairs We will refer to the process of creating the training data as theforwarddiffusionprocess, which will progressively make an image noisier Denoising Diffusion Probabilistic Models (DDPMs) Our model will learn how to revert that noising process, through what it is knownas thereversediffusion process → progressively denoising a noisy image Denoising Diffusion Probabilistic Models (DDPMs) Our model will learn how to revert that noising process, through what it is knownas thereversediffusion process → progressively denoising a noisy image Denoising Diffusion Probabilistic Models (DDPMs) Our model will learn how to revert that noising process, through what it is knownas thereversediffusion process → progressively denoising a noisy image Once trained, the model should have therefore learned how to denoise images So we can generate some purely random noise, run it through our model and getback an image generated out of pure noise! Denoising Diffusion Probabilistic Models (DDPMs) More formally, DDPMs work through many steps𝑡which are0, 1, … , 𝑇 •x0is the original image• 𝑞(x𝑡∣x𝑡−1)is theforwarddiffusion process• 𝑝𝜃(x𝑡−1∣x𝑡)will be thereversediffusion process (learned by our model withweights𝜃) Denoising Diffusion Probabilistic Models (DDPMs) More formally, DDPMs work through many steps𝑡which are0, 1, … , 𝑇 •x0is the original image• 𝑞(x𝑡∣x𝑡−1)is theforwarddiffusion process• 𝑝𝜃(x𝑡−1∣x𝑡)will be thereversediffusion process (learned by our model withweights𝜃) During forward diffusion we add Gaussian (Normal) noise to the image in every𝑡,producing noisy imagesx1,x2, …x𝑇 As𝑡becomes higher, the image becomes more and more noisy The forward process The forward process •Take an image at some point𝑡 − 1asx𝑡−1•Generate Gaussian noise from an isotropic multivariate Normal of sizex𝑡,with mean0andvariance𝛽𝑡•Scalex𝑡−1values by√1 − 𝛽𝑡(so data scale does not grow as we add noise)•Add the noise the the scaled image The forward process •Take an image at some point𝑡 − 1asx𝑡−1•Generate Gaussian noise from an isotropic multivariate Normal of sizex𝑡,with mean0andvariance𝛽𝑡•Scalex𝑡−1values by√1 − 𝛽𝑡(so data scale does not grow as we add noise)•Add the noise the the scaled image It can be directly computed as𝑞(x𝑡∣x𝑡−1) ≔ 𝒩(x𝑡; √1 − 𝛽𝑡x𝑡−1, 𝛽𝑡I) The forward process •Take an image at some point𝑡 − 1asx𝑡−1•Generate Gaussian noise from an isotropic multivariate Normal of sizex𝑡,with mean0andvariance𝛽𝑡•Scalex𝑡−1values by√1 − 𝛽𝑡(so data scale does not grow as we add noise)•Add the noise the the scaled image It can be directly computed as𝑞(x𝑡∣x𝑡−1) ≔ 𝒩(x𝑡; √1 − 𝛽𝑡x𝑡−1, 𝛽𝑡I) 𝛽𝑡is called thevariance schedule𝛽1, … , 𝛽𝑇that effectively controls how muchnoise is added in each step𝑡1 The forward process The full forward process is therefore: For a large𝑇, the final image is basically only noise (all original image info isessentially lost), so it becomes roughlyx𝑇∼ 𝒩(0,I) Demo here! The Nice™ property Trick to get anyx𝑡fromx0w