Diffusers

A Unity package to run pretrained diffusion models with Unity Sentis

OpenUPM · Documentation (coming soon) · Feedback/Questions

About

This is essentially a port of Hugging Face’s diffusers library.

It is still very early though, so as of today only a limited number of pipelines and schedulers are supported (see below).

Installation

via OpenUPM

In Edit -> Project Settings -> Package Manager, add a new scoped registry:

Name: Doji
URL: https://package.openupm.com
Scope(s): com.doji

In the Package Manager install com.doji.diffusers either by name or select it in the list under Package Manager -> My Registries

Roadmap

Some things that might be worked on next are:

• add/fix classifier-free guidance
• support more models
  • SD 1.5
  • SD 2.1
  • SDXL
  • SD-Turbo
  • SDXL-Turbo
  • LoRA models
• more scheduler implementations
  • PNDM
  • DDIM
  • EulerDiscrete
  • EulerAncestralDiscrete
  • LCM
  • DDPM
  • KDPM2Discrete
  • KDPM2AncestralDiscrete
• more pipeline/model types
  • txt2img
  • img2img,
  • inpaint,
  • upscale,
  • depth2img
  • marigold
• async pipeline methods (sliced inference)
• support multiple images per prompt
• write some documentation

Dependencies

• com.doji.transformers
• com.doji.pngcs