Tutorial – How to use Stable Diffusion for Architecture students
From Alexander Sanning
views
Related Media
Timestamps:
0:00 [Intro] Preview of generated images
0:19 [Intro] Introduction to scope of tutorial and hardware prerequisites
1:02 [Intro] Overview of software, models, extensions, etc.
5:16 [Installation] Installing dependencies (Python 3.10.6, Git)
6:54 [Installation] Installing AUTOMATIC1111 Web UI
9:24 [AUTOMATIC1111] First start-up, installing extensions
13:00 [File Explorer] Pasting the models, embeddings and extension data from the extracted Google Drive archive
13:58 [AUTOMATIC1111] Changing default output file type and more in UI settings
15:37 [Blender] Setting up a linear, raw-colour data mist pass render to be used as depth map
20:21 [AUTOMATIC1111] Customizing generation settings and prompts for generation from depth map
27:44 [AUTOMATIC1111] Using wildcards to increase the level of variation for the outputs
32:14 [AUTOMATIC1111] Setting the current generation settings as default values and organizing the outputs
32:58 [AUTOMATIC1111] Showcasing alternative workflow, with digitized sketch instead of depth map
38:19 [FFmpeg] Stitching the outputs into an animated video with FFmpeg
39:47 [Outro] Demonstration of reference outputs
.........................................
Link to tutorial assets:
Stable diffusion models, embeddings and extension data, as well as reference data and tutorial assets:
https://drive.google.com/drive/folders/1Zly_hhGlyM8A_Q5JA1p1Q9x1-D0v5_gi?usp=sharing
Other links:
AUTOMATIC1111 SD Web UI:
https://github.com/AUTOMATIC1111/stable-diffusion-webui
Python:
https://www.python.org/downloads/release/python-3106/
Git:
https://git-scm.com/download/win
FFmpeg:
https://www.gyan.dev/ffmpeg/builds/
ControlNet:
https://github.com/Mikubill/sd-webui-controlnet
Dynamic Prompts:
https://github.com/adieyal/sd-dynamic-prompts
DreamShaper 4:
https://civitai.com/models/4384?modelVersionId=25067
Realistic Vision 2.0:
https://civitai.com/models/4201?modelVersionId=29460
ControlNet models:
https://huggingface.co/lllyasviel/ControlNet-v1-1/tree/main
epi_noiseoffset:
https://civitai.com/models/13941?modelVersionId=18445
Detail Tweaker LoRA:
https://civitai.com/models/58390/detail-tweaker-lora-lora
Style Asian Less:
https://civitai.com/models/50755?modelVersionId=57451
Bad Prompt:
https://civitai.com/models/55700/badprompt-negative-embedding
Bad Artist:
https://huggingface.co/nick-x-hacker/bad-artist/tree/main
Easy Negative:
https://huggingface.co/embed/EasyNegative/tree/main
Deep Negative:
https://civitai.com/models/4629/deep-negative-v1x
.........................................
PowerShell FFmpeg command (make sure to change the command so that the first part reflects where you personally have FFmpeg installed):
D:\FFmpeg\ffmpeg-2023-05-18-git-01d9a84ef5-full_build\bin\ffmpeg.exe -f image2 -framerate 1 -i "output (%d).jpg" -r 24 Animation.mp4
.........................................
Hardware requirements:
NVIDIA dedicated GPU with minimum 6 GB of VRAM
.........................................
This tutorial demonstrates to KTH Architecture students how to use Stable Diffusion to generate images from either a depth map of a massing study 3D model, or a digital/digitized sketch.
Stable Diffusion is presented as yet another tool for architecture students to use in their design process, not to replace the active designing part of it, but to enrichen the process by offering a multitude of generated variations – all stemming from the design work you already have done, either in the form of a volumetric massing study in a 3D program, a digital sketch, or a digitized analogue sketch.
With the variations generated, you are free to use the images as references, as you would any other non-machine-learning-created reference image, and decide yourself what elements you would like to work further with, include, dismiss, or take a stance against.
The operating system used for this tutorial is Windows 10. Running another operating system (e.g. Linux, MacOS) might amount to a different user experience or software incompatibility.
If you want to check in which computer room(s) any relevant software might be installed in – CADLAB, DFL, and ARCPLAN are in the KTH School of Architecture building – please check the following link for current status:
https://www.kth.se/en/student/it/campus/computer-rooms/lista
Disclaimer:
This tutorial aims to increase student awareness of the versatility of the digital tools available for use within the context of the architectural education offered by KTH. As such, it might not be generally applicable, but on the other hand, if at least one architecture student is helped by it in fulfilling deliverables requirements or being taught something of interest, the purpose of this tutorial is satisfactorily achieved. There might be inaccuracies in this tutorial – if you identify any significant one, please tell us in the comments.
0:00 [Intro] Preview of generated images
0:19 [Intro] Introduction to scope of tutorial and hardware prerequisites
1:02 [Intro] Overview of software, models, extensions, etc.
5:16 [Installation] Installing dependencies (Python 3.10.6, Git)
6:54 [Installation] Installing AUTOMATIC1111 Web UI
9:24 [AUTOMATIC1111] First start-up, installing extensions
13:00 [File Explorer] Pasting the models, embeddings and extension data from the extracted Google Drive archive
13:58 [AUTOMATIC1111] Changing default output file type and more in UI settings
15:37 [Blender] Setting up a linear, raw-colour data mist pass render to be used as depth map
20:21 [AUTOMATIC1111] Customizing generation settings and prompts for generation from depth map
27:44 [AUTOMATIC1111] Using wildcards to increase the level of variation for the outputs
32:14 [AUTOMATIC1111] Setting the current generation settings as default values and organizing the outputs
32:58 [AUTOMATIC1111] Showcasing alternative workflow, with digitized sketch instead of depth map
38:19 [FFmpeg] Stitching the outputs into an animated video with FFmpeg
39:47 [Outro] Demonstration of reference outputs
.........................................
Link to tutorial assets:
Stable diffusion models, embeddings and extension data, as well as reference data and tutorial assets:
https://drive.google.com/drive/folders/1Zly_hhGlyM8A_Q5JA1p1Q9x1-D0v5_gi?usp=sharing
Other links:
AUTOMATIC1111 SD Web UI:
https://github.com/AUTOMATIC1111/stable-diffusion-webui
Python:
https://www.python.org/downloads/release/python-3106/
Git:
https://git-scm.com/download/win
FFmpeg:
https://www.gyan.dev/ffmpeg/builds/
ControlNet:
https://github.com/Mikubill/sd-webui-controlnet
Dynamic Prompts:
https://github.com/adieyal/sd-dynamic-prompts
DreamShaper 4:
https://civitai.com/models/4384?modelVersionId=25067
Realistic Vision 2.0:
https://civitai.com/models/4201?modelVersionId=29460
ControlNet models:
https://huggingface.co/lllyasviel/ControlNet-v1-1/tree/main
epi_noiseoffset:
https://civitai.com/models/13941?modelVersionId=18445
Detail Tweaker LoRA:
https://civitai.com/models/58390/detail-tweaker-lora-lora
Style Asian Less:
https://civitai.com/models/50755?modelVersionId=57451
Bad Prompt:
https://civitai.com/models/55700/badprompt-negative-embedding
Bad Artist:
https://huggingface.co/nick-x-hacker/bad-artist/tree/main
Easy Negative:
https://huggingface.co/embed/EasyNegative/tree/main
Deep Negative:
https://civitai.com/models/4629/deep-negative-v1x
.........................................
PowerShell FFmpeg command (make sure to change the command so that the first part reflects where you personally have FFmpeg installed):
D:\FFmpeg\ffmpeg-2023-05-18-git-01d9a84ef5-full_build\bin\ffmpeg.exe -f image2 -framerate 1 -i "output (%d).jpg" -r 24 Animation.mp4
.........................................
Hardware requirements:
NVIDIA dedicated GPU with minimum 6 GB of VRAM
.........................................
This tutorial demonstrates to KTH Architecture students how to use Stable Diffusion to generate images from either a depth map of a massing study 3D model, or a digital/digitized sketch.
Stable Diffusion is presented as yet another tool for architecture students to use in their design process, not to replace the active designing part of it, but to enrichen the process by offering a multitude of generated variations – all stemming from the design work you already have done, either in the form of a volumetric massing study in a 3D program, a digital sketch, or a digitized analogue sketch.
With the variations generated, you are free to use the images as references, as you would any other non-machine-learning-created reference image, and decide yourself what elements you would like to work further with, include, dismiss, or take a stance against.
The operating system used for this tutorial is Windows 10. Running another operating system (e.g. Linux, MacOS) might amount to a different user experience or software incompatibility.
If you want to check in which computer room(s) any relevant software might be installed in – CADLAB, DFL, and ARCPLAN are in the KTH School of Architecture building – please check the following link for current status:
https://www.kth.se/en/student/it/campus/computer-rooms/lista
Disclaimer:
This tutorial aims to increase student awareness of the versatility of the digital tools available for use within the context of the architectural education offered by KTH. As such, it might not be generally applicable, but on the other hand, if at least one architecture student is helped by it in fulfilling deliverables requirements or being taught something of interest, the purpose of this tutorial is satisfactorily achieved. There might be inaccuracies in this tutorial – if you identify any significant one, please tell us in the comments.
- Tags
-