Kolors-IP-Adapter-FaceID-Plus weights and inference code

โ€‚ โ€‚ โ€‚

๐Ÿ“– Introduction

We provide Kolors-IP-Adapter-FaceID-Plus module weights and inference code based on Kolors-Basemodel. Examples of Kolors-IP-Adapter-FaceID-Plus results are as follows:

  • Our Kolors-IP-Adapter-FaceID-Plus module is trained on a large-scale and high-quality face dataset. We use the face ID embeddings generated by insightface and the CLIP features of face area to keep the face ID and structure information.

๐Ÿ“Š Evaluation

For evaluation, we constructed a test set consisting of over 200 reference images and text prompts. We invited several image experts to provide fair ratings for the generated results of different models. The experts assessed the generated images based on five criteria: visual appeal, text faithfulness, face similarity, facial aesthetics and overall satisfaction. Visual appeal and text faithfulness are used to measure the text-to-image generation capability, adhering to the evaluation standards of BaseModel. Meanwhile, face similarity and facial aesthetics are used to evaluate the performance of the proposed Kolors-IP-Adapter-FaceID-Plus. The results are summarized in the table below, where Kolors-IP-Adapter-FaceID-Plus outperforms SDXL-IP-Adapter-FaceID-Plus across all metrics.

Model Average Text Faithfulness Average Visual Appeal Average Face Similarity Average Facial Aesthetics Average Overall Satisfaction
SDXL-IP-Adapter-FaceID-Plus 4.014 3.455 3.05 2.584 2.448
Kolors-IP-Adapter-FaceID-Plus 4.235 4.374 4.415 3.887 3.561

Kolors-IP-Adapter-FaceID-Plus employs chinese prompts, while SDXL-IP-Adapter-FaceID-Plus use english prompts.

๐Ÿ› ๏ธ Usage

Requirements

The dependencies and installation are basically the same as the Kolors-BaseModel.


  1. Repository Cloning and Dependency Installation
apt-get install git-lfs
git clone https://github.com/Kwai-Kolors/Kolors
cd Kolors
conda create --name kolors python=3.8
conda activate kolors
pip install -r requirements.txt
pip install insightface onnxruntime-gpu
python3 setup.py install
  1. Weights download link๏ผš
huggingface-cli download --resume-download Kwai-Kolors/Kolors-IP-Adapter-FaceID-Plus --local-dir weights/Kolors-IP-Adapter-FaceID-Plus

or

git lfs clone https://huggingface.co/Kwai-Kolors/Kolors-IP-Adapter-FaceID-Plus weights/Kolors-IP-Adapter-FaceID-Plus
  1. Inference๏ผš
python ipadapter_FaceID/sample_ipadapter_faceid_plus.py ./ipadapter_FaceID/assets/image1.png "็ฉฟ็€ๆ™š็คผๆœ๏ผŒๅœจๆ˜Ÿๅ…‰ไธ‹็š„ๆ™šๅฎดๅœบๆ™ฏไธญ๏ผŒ็ƒ›ๅ…‰้—ช้—ช๏ผŒๆ•ดไธชๅœบๆ™ฏๆด‹ๆบข็€ๆตชๆผซ่€ŒๅฅขๅŽ็š„ๆฐ›ๅ›ด"

python ipadapter_FaceID/sample_ipadapter_faceid_plus.py ./ipadapter_FaceID/assets/image2.png "่ฅฟ้ƒจ็‰›ไป”๏ผŒ็‰›ไป”ๅธฝ๏ผŒ่’้‡Žๅคง้•–ๅฎข๏ผŒ่ƒŒๆ™ฏๆ˜ฏ่ฅฟ้ƒจๅฐ้•‡๏ผŒไป™ไบบๆŽŒ๏ผŒ,ๆ—ฅ่ฝไฝ™ๆ™–, ๆš–่‰ฒ่ฐƒ, ไฝฟ็”จXT4่ƒถ็‰‡ๆ‹ๆ‘„, ๅ™ช็‚น, ๆ™•ๅฝฑ, ๆŸฏ่พพ่ƒถๅท๏ผŒๅคๅค"

Acknowledgments

Downloads last month
2,823
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Spaces using Kwai-Kolors/Kolors-IP-Adapter-FaceID-Plus 3