Categories: Image

Stable Diffusion v2-1-unCLIP model released

Information taken from the GitHub page: https://github.com/Stability-AI/stablediffusion/blob/main/doc/UNCLIP.MD

HuggingFace checkpoints and diffusers integration: https://huggingface.co/stabilityai/stable-diffusion-2-1-unclip

Public web-demo: https://clipdrop.co/stable-diffusion-reimagine


unCLIP is the approach behind OpenAI’s DALL·E 2, trained to invert CLIP image embeddings. We finetuned SD 2.1 to accept a CLIP ViT-L/14 image embedding in addition to the text encodings. This means that the model can be used to produce image variations, but can also be combined with a text-to-image embedding prior to yield a full text-to-image model at 768×768 resolution.

If you would like to try a demo of this model on the web, please visit https://clipdrop.co/stable-diffusion-reimagine

This model essentially uses an input image as the ‘prompt’ rather than require a text prompt. It does this by first converting the input image into a ‘CLIP embedding’, and then feeds this into a stable diffusion 2.1-768 model fine-tuned to produce an image from such CLIP embeddings, enabling a users to generate multiple variations of a single image this way. Note that this is distinct from how img2img does it (the structure of the original image is generally not kept).

Blog post: https://stability.ai/blog/stable-diffusion-reimagine

submitted by /u/hardmaru
[link] [comments]

AI Generated Robotic Content

Share
Published by
AI Generated Robotic Content
Tags: ai images

Recent Posts

Tried longer videos with WAN 2.2 Animate

I altered the workflow a little bit from my previous post (using Hearmeman's Animate v2…

4 hours ago

10 Python One-Liners for Generating Time Series Features

Time series data normally requires an in-depth understanding in order to build effective and insightful…

4 hours ago

Evaluating Evaluation Metrics — The Mirage of Hallucination Detection

Hallucinations pose a significant obstacle to the reliability and widespread adoption of language models, yet…

4 hours ago

Announcing new capabilities in Vertex AI Training for large-scale training

Building and scaling generative AI models demands enormous resources, but this process can get tedious.…

4 hours ago

MiniMax-M2 is the new king of open source LLMs (especially for agentic tool calling)

Watch out, DeepSeek and Qwen! There's a new king of open source large language models…

5 hours ago

Elon Musk’s Grokipedia Pushes Far-Right Talking Points

The new AI-powered Wikipedia competitor falsely claims that pornography worsened the AIDS epidemic and that…

5 hours ago