What is Deforum extension for Automatic1111?
Deforum is an open-source community-driven project to generate images and videos using Stable Diffusion. The Deforum extension for Automatic1111 lets you run Deforum in Stable Diffusion Webui.
How to replace characters in videos using Deforum extension in Stable Diffusion Webui?
1. If you don’t have stable diffusion Webui installed, go to Install stable diffusion webui on Windows.
2. If you haven’t installed ControlNet, go to install ControlNet in Stable Diffusion Webui.
3. Prepare a video with a character is actioning or dancing. Render it as a 576 x 1024 mp4.
4. Run “webui-user.bat” to open Stable Diffusion Webui. Click Extension tab, search for deforum, and install deforum-for-automatic1111-webui.
5. Close and reopen your Webui, you will see Deforum tab at the top area of the screen. Select a Stable Diffusion checkpoint, such as protogenV22.
6. Click Deforum tab, in the Run tab, Set Sampler to “Euler a”. Set step to “15”, Set Width to “576,” set Height to “1024.”
7. Click Keyframes tab. Set Animation mode to “3D”, Set Border mode to “wrap”. Keep Cadence to be “2”. Keep Max frames to “120”.
8. In the below area, click Strength tab, Set Strength schedule to “0:(0).”
8. Click CFG tab, Set CFG scale schedule to “0:(3).”
9. Click Seed tab, set Seed behavior to “fixed”.
10. In the area below, click Motion tab. Change Translation Z to “0:(0).”
11. Click Noise tab, set Noise schedule to 0:(0). Set Noise multiplier schedule to “0:(0.0).”
12. Click Coherence tab, change Color coherence to “None”.
13. Click Anti Blur tab, change Amount schedule to “0:(0.05).”
14. Click Prompts tab at the top. Change the sample prompts to your prompts. Note the last prompt has no comma at the end.
15. Click Init tab. In Image Init tab, set strength to “0.”
In Video Init tab, copy the path of your video mp4 file, and paste to Video init path/URL.
16. Click ControlNet tab. In CN Model1, check “Enable.” Set Preprocessor to None, set Model to “control_v11f1e_sd15_tile.” Past the path of the video file in ControlNet Input Video/Image Path. Check “ControlNet is more important.”
17. In CN Model2 tab, check “Enable”. Set Preprocessor to “openpose_full”, set Model to “control_v11p_sd15_openpose.” Past the path of the video file in ControlNet Input Video/Image Path. Check “ControlNet is more important.”
18. In Hybrid Video tab, Set Hybrid composite to “Before Motion.” Set Hybrid motion to “Optical Flow”. Set Flow method to “DIS Medium”.
19. In Output tab, change FPS to a value which is the same as your input video.
20. At right bottom area, set the path to save your settings as a text file and click “Save Settings.” You can load the same settings next time.
21. Click Generate button. Monitor the progress in webui. When it finishes, the result is saved at “outputs\img2img-images\” directory.
22. Continue tuning prompts, the parameters in Keyframes, such as CFG or Noise to improve the quality and consistency of your results.
