Testing Stable Diffusion on Apple Silicon.

Background

Stable Diffusion is a text-to-image diffusion model that can create an AI image from a given prompt. I wanted to test it on my M1 Macbook.

Setup

Setup was fairly simple. I followed the instructions on Lincoln Stein’s fork of Stable Diffusion for M1 Macs, and despite all the disclaimers, the process worked like a charm. There is rapid development across Stable Diffusion and pytorch nightly, so my experience could be attributed to lucky timing, YMMV.

I do experience some of the bugs already reported as issues – occasionally my generated output is an black square.

Getting any output image was slow since I couldn’t take advantage of CUDA cores, and was running everything off CPU. I am lucky enough to have a friend with a beefy Nvidia GPU though – I’ve attached some of the interesting images generated at the end.

Images

“ocean”. First photo was created with only one pass.

ocean1

ocean2

ocean3

“game of thrones anime”

game-of-throne

“thanos eating popcorn”

thanos-popcorn

“dog king”

dog-king

“cat in armor high def”

cat-knight

I also tried out the img2img command, turning the last photo into a “drawing”.

cat-knight-drawing

And, a profile photo for myself.

me

More Images

These photos were generated with my friend’s desktop – he has an AMD 5950x, RTX 3080 FE. Super fast – much easier to iterate on good prompts and search for better seeds.

cityscape

The following is the original generated from txt2img, the rest are alternatives/improvements using img2img.

cyborg

cyborg1

cyborg2

nicolas_cage_party