face  animation using ai

What is Thin-Plate Spline Motion Model?

Thin-Plate Spline Motion Mode is an Ai model to add movement to a still image based on the driving video. The paper is here. One of its usage is to do face animation using Ai.

How to create face animation using Ai model from GitHub?

1. First download the code from Thin-Plate-Spline-Motion-Model GitHub repository. Open a dos prompt, in the directory where you want to install, type the command:
>git clone https://github.com/yoyo-nb/Thin-Plate-Spline-Motion-Model.git
2. Still in the dos prompt, run the following commands
>cd Thin-Plate-Spline-Motion-Model
>mkdir checkpoints
>pip3 install wldhx.yadisk-direct
>curl -L $(yadisk-direct https://disk.yandex.com/d/i08z-kCuDGLuYA) -o checkpoints/vox.pth.tar
This will download vox.path.tar in the “checkpoints” directory.
3. You should setup a conda environment as this guide. If you run into problems when you setup, you can download a pytorch_env at Gumroad. Unzip and put it at your anaconda3 installation under “envs” directory.
4. Now prepare your input files. The first one is a short video mp4 file with a face movement. The dimension is 512 x 512. Name it as “driving.mp4.” Put it at “assets” directory.
5. The second file is a face that you want to animate. Note the mouth of the character should be open to show a little bit upper teeth. The dimension is 512 x 512. Name it as “source.jpg” and put it at “assets” directory.
6. Download main.py and put it at “Thin-Plate-Spline-Motion-Model” directory.
7. Open an Anaconda Prompt. Run command:
>conda activate pytorch_env
8. Still in Anaconda prompt, go to the directory “Thin-Plate-Spline-Motion-Model” and run command:
>python main.py
9. When it finishes, the new video is called “generated.mp4” under “Thin-Plate-Spline-Motion-Model” directory.
10. (Optional) You can enhance resolution of the animation using Real-ESRGan.