Anime2Sketch

Anime2Sketch: A sketch extractor for illustration, anime art, manga

By Xiaoyu Xiang

Updates

• 2022.1.14: Add Docker environment by kitoria
• 2021.12.25: Update README. Merry Christmas!
• 2021.5.24: Fix an interpolation error and a GPU inference error.
• 2021.5.12: Web Demo by AK391
• 2021.5.2: Upload more example results of anime video.
• 2021.4.30: Upload the test scripts. Now our repo is ready to run!
• 2021.4.11: Upload the pretrained weights, and more test results.
• 2021.4.8: Create the repo.

Introduction

The repository contains the testing codes and pretrained weights for Anime2Sketch.

Anime2Sketch is a sketch extractor that works well on illustration, anime art, and manga. It is an application based on the paper "Adversarial Open Domain Adaption for Sketch-to-Photo Synthesis".

Prerequisites

• Linux, macOS, Docker
• Python 3 (Recommend to use Anaconda)
• CPU or NVIDIA GPU + CUDA CuDNN
• Pillow, PyTorch

Get Started

Installation

Install the required packages: pip install -r requirements.txt

Download Pretrained Weights

Please download the weights from GoogleDrive, and put it into the weights/ folder.

Test

Shell python3 test.py --dataroot /your_input/dir --load_size 512 --output_dir /your_output/dir The above command includes three arguments: - dataroot: your test file or directory - load_size: due to the memory limit, we need to resize the input image before processing. By default, we resize it to 512x512. - output_dir: path of the output directory

Run our example: Shell python3 test.py --dataroot test_samples/madoka.jpg --load_size 512 --output_dir results/

Docker

If you want to run on Docker, you can easily do so by customizing the input/output images directory.
Build docker image
Shell make docker-build

Setting input/output directory
You can customize mount volumes for input/output images by Makefile. Please setting your target directory.
docker run -it --rm --gpus all -v `pwd`:/workspace -v {your_input_dir}:/input -v {your_output_dir}:/output anime2sketch

example:
docker run -it --rm --gpus all -v `pwd`:/workspace -v `pwd`/test_samples:/input -v `pwd`/output:/output anime2sketch

Run
Shell make docker-run

if you want to run cpu only, you will need to fix two things (remove gpu options). - Dockerfile CMD line to CMD [ "python", "test.py", "--dataroot", "/input", "--load_size", "512", "--output_dir", "/output" ] - Makefile docker-run line to docker run -it --rm -v `pwd`:/workspace -v `pwd`/images/input:/input -v `pwd`/images/output:/output anime2sketch

Train

This project is a sub-branch of AODA. Please check it for the training instructions.

More Results

Our model works well on illustration arts: Turn handrawn photos to clean linearts: Simplify freehand sketches: And more anime results:

Contact

Xiaoyu Xiang.

You can also leave your questions as issues in the repository. I will be glad to answer them!

License

This project is released under the MIT License.

Citations

```BibTex @misc{Anime2Sketch, author = {Xiaoyu Xiang, Ding Liu, Xiao Yang, Yiheng Zhu, Xiaohui Shen}, title = {Anime2Sketch: A Sketch Extractor for Anime Arts with Deep Networks}, year = {2021}, publisher = {GitHub}, journal = {GitHub repository}, howpublished = {\url{https://github.com/Mukosame/Anime2Sketch}} }

@inproceedings{xiang2022adversarial, title={Adversarial Open Domain Adaptation for Sketch-to-Photo Synthesis}, author={Xiang, Xiaoyu and Liu, Ding and Yang, Xiao and Zhu, Yiheng and Shen, Xiaohui and Allebach, Jan P}, booktitle={Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision}, year={2022} } ```

Issues

Fixed error when applying a resampling filter.

opened on 2022-10-24 16:53:27 by virtualramblas

This fix prevent the following error happening when resizing input image(s) before generating sketches: ValueError: Unknown resampling filter (InterpolationMode.BICUBIC). Use Image.Resampling.NEAREST (0), Image.Resampling.LANCZOS (1), Image.Resampling.BILINEAR (2), Image.Resampling.BICUBIC (3), Image.Resampling.BOX (4) or Image.Resampling.HAMMING (5)

ValueError: Unknown resampling filter (InterpolationMode.BICUBIC).

opened on 2022-10-18 10:17:30 by geng-lee

ValueError: Unknown resampling filter (InterpolationMode.BICUBIC). Use Image.Resampling.NEAREST (0), Image.Resampling.LANCZOS (1), Image.Resampling.BILINEAR (2), Image.Resampling.BICUBIC (3), Image.Resampling.BOX (4) or Image.Resampling.HAMMING (5)

一个关于使用AODA训练这个模型时遇到的问题

opened on 2022-10-11 11:15:27 by cnyvfang

请问一下作者大大，在使用AODA训练Anime2Sketch时是否有使用GC，是将类别数设置为了0还是1呢。

ValueError: Expected more than 1 spatial element when training, got input size torch.Size([1, 512, 1, 1])

opened on 2021-11-30 15:20:33 by xcfstudio

raise ValueError("Expected more than 1 spatial element when training, got input size {}".format(size)) ValueError: Expected more than 1 spatial element when training, got input size torch.Size([1, 512, 1, 1])

analytics-python 1.4.0 backoff 1.10.0 bcrypt 3.2.0 certifi 2021.10.8 cffi 1.15.0 charset-normalizer 2.0.8 click 8.0.3 colorama 0.4.4 cryptography 36.0.0 cycler 0.11.0 ffmpy 0.3.0 Flask 2.0.2 Flask-CacheBuster 1.0.0 Flask-Cors 3.0.10 Flask-Login 0.5.0 fonttools 4.28.2 gradio 2.4.6 idna 3.3 itsdangerous 2.0.1 Jinja2 3.0.3 kiwisolver 1.3.2 markdown2 2.4.1 MarkupSafe 2.0.1 matplotlib 3.5.0 monotonic 1.6 numpy 1.21.4 packaging 21.3 pandas 1.3.4 paramiko 2.8.1 pep517 0.12.0 Pillow 8.4.0 pip 21.3.1 pycparser 2.21 pycryptodome 3.11.0 pydub 0.25.1 PyNaCl 1.4.0 pyparsing 3.0.6 python-dateutil 2.8.2 pytz 2021.3 requests 2.26.0 setuptools 41.2.0 setuptools-scm 6.3.2 six 1.16.0 tomli 1.2.2 torch 1.10.0 torchtext 0.11.0 torchvision 0.11.1 tqdm 4.62.3 typing_extensions 4.0.0 urllib3 1.26.7 Werkzeug 2.0.2

Why dont you add a simple bat file for windows to go through all files in folder ?

opened on 2021-09-21 10:18:48 by 2blackbar

heres what works for me and converts all files from in folder to out folder with 2048 tex quality :

@echo off for %%F in (in*.*) do test.py --dataroot in --load_size 2048 --output_dir out/

enhancement : add a dockerfile

opened on 2021-05-10 21:52:17 by survivant

could be useful to have a dockerfile that contains all the programs needed to run it. We could just mount a volume for the input/output.