Haribo Blog

• On Distillation of Guided Diffusion Models

  2023-08-07

  Haribo

  논문리뷰

  Diffusion  생성모델  Distillation

  Full Citation: “Meng, Chenlin, et al. “On distillation of guided diffusion models.” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.”
  Link to Paper: https://arxiv.org/pdf/2210.03142.pdf
  Conference Details: CVPR 2023 (우수논문)

  ---

  • 느린 샘플링이라는 기존 diffusion 모델의 한계를 효과적으로 해결한 연구는 있으나, 이는 unconditional diffusion model에 한정된다.

  • High-Resolution conditional image generation 을 위해서는 Classifier-free guidance가 필요하지만, 이 방법은 많은 연산을 필요로 하며 샘플링 속도가 느리다.

  • 이 논문은 Classifier-free guidance를 사용하면서도 빠른 샘플링을 가능하게 하는 새로운 학습 기법을 제안하며, 이 기법은 단 1~4 단계만으로도 기존 모델들과 비교할 수 있는 성능을 보여준다.

    • Pixel-space diffusion: ImageNet 64x64, CIFAR-10 데이터셋에서 단 4 denoising step만에 기존 모델과 비슷한 성능.

    • Latent-space diffusion: LAION 데이터셋에서 1~4 denoising step만에 기존 모델과 비슷한 성능.

    • Text-to-Image diffusion: 2~4 denoising step만에 기존 모델과 비슷한 성능

  Read All

  ---

• Drag Your GAN

  2023-07-17

  Haribo

  논문리뷰

  GAN  생성모델

  Full Citation: “Pan, Xingang, et al. “Drag your gan: Interactive point-based manipulation on the generative image manifold.” ACM SIGGRAPH 2023 Conference Proceedings. 2023.”
  Link to Paper: https://arxiv.org/abs/2305.10973
  Conference Details: ACM SIGGRAPH 2023

  ---

  • 새로운 GAN 모델을 만드는 것이 아닌 기존의 GAN (StyleGAN2)을 컨트롤 하는 연구.
  • Src, Tgt 두 종류의 포인터로 생성 된 이미지의 pose, shape, expression 등등을 변형.
  • GAN을 컨트롤 하는데에 있어 추가적인 인공지능 모델 학습이나 활용 필요없이 내부 featuremap domain에서 연산이 진행됨.
  • GAN의 잠재능력을 극한으로 활용하는 느낌.

  

  Web Demos

  

  

  Read All

  ---

• Make-A-Video

  2023-06-23

  Haribo

  논문리뷰

  Diffusion  생성모델  Meta  Video

  Full Citation: “Singer, Uriel, et al. “Make-a-video: Text-to-video generation without text-video data.” arXiv preprint arXiv:2209.14792 (2022).”
  Link to Paper: https://arxiv.org/abs/2209.14792
  Conference Details: ICLR 2023
  Project Page: Link

  ---

  • 선행 Text-to-Video 연구들은 다수의 video-text pair 데이터셋이 필요했으나, 사전학습 된 Diffusion 모델의 능력을 활용해 video-text 데이터셋 없이 video 데이터셋만을 활용해 고퀄리티 text-to-video 생성모델 학습 방식을 선보임.
  • 4D 입력인 video 처리를 위해 Spatial/Temporal Convolution + Attention 연산을 활용.

  Read All

  ---

• Auto-encoding variational bayes

  2023-04-28

  Haribo

  논문리뷰

  VAE  생성모델

  Full Citation: “Kingma, Diederik P., and Max Welling. “Auto-encoding variational bayes.” arXiv preprint arXiv:1312.6114 (2013).”
  Link to Paper: https://arxiv.org/abs/1312.6114
  Conference Details: arXiv 2013 \

  ---

  Read All

  ---

• High-resolution image synthesis with latent diffusion models.

  2023-03-31

  Haribo

  논문리뷰

  Diffusion  생성모델

  Full Citation: “Rombach, Robin, et al. “High-resolution image synthesis with latent diffusion models.” Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022.”
  Link to Paper: https://arxiv.org/abs/2112.10752
  Conference Details: CVPR 2022 \

  ---

  • Inference, Training에 매우 큰 자원과 시간이 소모되던 기존의 Diffusion 모델의 한계를 극복하고 안정된 공간에서 diffusion process를 학습시키기 때문에 다양한 conditional에 대해 prior를 학습할수 있게함.
  • 실험적인 해석이 매우 필요함. 직접적인 코드 해석이 필요함.

  Read All

  ---

• DDIM (Denoising Diffusion Implicit Models)

  2023-03-20

  Haribo

  논문리뷰

  Diffusion  생성모델

  Full Citation: “Song, Jiaming, Chenlin Meng, and Stefano Ermon. “Denoising diffusion implicit models.” arXiv preprint arXiv:2010.02502 (2020).”
  Link to Paper: https://arxiv.org/abs/2010.02502
  Conference Details: arXiv 2020 \

  ---

  Read All

  ---

• Classifier-Free Diffusion Guidance

  2023-03-05

  Haribo

  논문리뷰

  Diffusion  생성모델

  Full Citation: “Ho, Jonathan, and Tim Salimans. “Classifier-free diffusion guidance.” arXiv preprint arXiv:2207.12598 (2022).”
  Link to Paper: https://arxiv.org/abs/2207.12598
  Conference Details: arXiv 2022 \

  ---

  Read All

  ---

• ControlNet: Adding Conditional Control to Text-to-Image Diffusion Models

  2023-02-20

  Haribo

  논문리뷰

  Diffusion  생성모델

  Full Citation: “Zhang, Lvmin, Anyi Rao, and Maneesh Agrawala. “Adding conditional control to text-to-image diffusion models.” Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.”
  Link to Paper: https://arxiv.org/abs/2302.05543
  Conference Details: CVPR 2023 \

  ---

  Read All

  ---

• Improved Denoising Diffusion Probabilistic Models

  2023-02-10

  Haribo

  논문리뷰

  Diffusion  생성모델

  Full Citation: “Nichol, Alexander Quinn, and Prafulla Dhariwal. “Improved denoising diffusion probabilistic models.” International Conference on Machine Learning. PMLR, 2021.”
  Link to Paper: https://arxiv.org/abs/2102.09672
  Conference Details: PMLR 2021 \

  ---

  Read All

  ---

• Person Image Synthesis via Denoising Diffusion Model

  2023-01-25

  Haribo

  논문리뷰

  Diffusion  생성모델

  Full Citation: “Bhunia, Ankan Kumar, et al. “Person image synthesis via denoising diffusion model.” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.”
  Link to Paper: https://arxiv.org/abs/2211.12500
  Conference Details: CVPR 2023 \

  ---

  Read All

  ---