madebyollin/taesd-x4-upscaler

🍰 Tiny AutoEncoder for Stable Diffusion X4 Upscaler

taesd-x4-upscaler is very tiny autoencoder which uses the same "latent API" as stable-diffusion-x4-upscaler's VAE. taesd-x4-upscaler is useful for real-time previewing of the upsampling process.

This repo contains .safetensors versions of the taesd-x4-upscaler weights.

Using in 🧨 diffusers

import requests
from PIL import Image
from io import BytesIO

url = "https://huggingface.co/datasets/hf-internal-testing/diffusers-images/resolve/main/sd2-upscale/low_res_cat.png"
low_res_img = Image.open(BytesIO(requests.get(url).content)).convert("RGB").resize((128, 128))

import torch
from diffusers import StableDiffusionUpscalePipeline, AutoencoderTiny

pipe = StableDiffusionUpscalePipeline.from_pretrained("stabilityai/stable-diffusion-x4-upscaler", torch_dtype=torch.float16)
pipe.vae = AutoencoderTiny.from_pretrained("madebyollin/taesd-x4-upscaler", torch_dtype=torch.float16)
pipe = pipe.to("cuda")

image = pipe("a white cat", image=low_res_img, num_inference_steps=25).images[0]
image.save("upsampled.png")