PuLID-FLUX: ID Customization Solution for Flux AI

Introduction to PuLID-FLUX

PuLID-FLUX is a new tuning-free ID customization solution for the FLUX.1-dev model. It maintains the original model behavior while allowing for high-fidelity ID customization.

What is PuLID-FLUX?

PuLID-FLUX-v0.9.0 is a model that lets you customize identification without tuning for the FLUX.1-dev model. It's highly flexible and maintains the original model's performance while adding unique IDs to images.

Setup and Integration

Installing PuLID-FLUX

Clone the Repository: Head to the PuLID GitHub page at PuLID GitHub.
Set Up Environment: Follow the guide to set up the required environment (Python version, dependencies, etc.).
```
git clone https://github.com/ToTheBeginning/PuLID.git
cd PuLID
pip install -r requirements.txt
```
Download Necessary Models:
- Download flux1-dev.safetensors and ae.safetensors from black-forest-labs/FLUX.1-dev.
- The PuLID-FLUX model will be automatically downloaded from Hugging Face.
Run the Application:
```
python app_flux.py
```

Using Different Memory Configurations

Naive bf16

Run with bf16 directly:
```
python app_flux.py
```
- Peak memory is under 45GB.

bf16 + Offload

Use offloading to save memory:
```
python app_flux.py --offload
```
- Peak memory is under 30GB.

fp8 + Offload (For Consumer-Grade GPUs)

Ensure additional requirements are installed:
```
pip install -r requirements-fp8.txt
```

Use flux-dev-fp8 checkpoint:

python app_flux.py --offload --fp8 --onnx_provider cpu
- Peak memory is under 15GB (for GPUs with 16GB memory).
- For 24GB graphic memory users:
python app_flux.py --offload --fp8
- Peak memory is under 17GB.

Note: fp8 has quality degradation in face details compared to bf16.

bf16 + Aggressive Offload

Use more aggressive offloading:
```
python app_flux.py --aggressive_offload
```
- Peak memory is around 23GB, but the process will be very slow.

Online Demo

Try it out on Hugging Face: PuLID-FLUX Demo

ComfyUI

Stay tuned for the community implementation of ComfyUI support.

Visual Results

Useful Tips

Parameter Tuning

1. Timestep to Start Inserting ID

Controls the timing of ID insertion. Sets ID fidelity and editability trade-off.
Usage:
- Realistic images: Start at timestep 4.
- Higher ID similarity: Lower the value.
- Stylized images: Set it to 0-1.

2. True CFG Scale

FLUX.1-dev uses guidance distillation with a modulated guidance scale to simulate CFG.
Compare true CFG with fake CFG for photorealistic and stylized scenes.

Technical Details

Switched ID encoder from MLP to Transformer.
Inspired by Flamingo, additional cross-attention blocks are inserted every few DIT blocks to handle ID features interaction with DIT image features.
Optional acceleration method similar to SDXL-Lightning, yet not essential for training PuLID.

License

As long as you use FLUX.1-dev model, you should follow the FLUX.1-dev model license.

Contact

For queries or suggestions, contact Yanze Wu or open an issue/discussion on the GitHub page.

Additional FAQs

Q: How can I try PuLID-FLUX? A: You can start by visiting their GitHub page at PuLID-FLUX GitHub and following the setup instructions.

Q: Does PuLID-FLUX support commercial use? A: The license on the PuLID-FLUX GitHub suggests commercial use is allowed, but integrated models like Insightface have commercial restrictions.

Q: What's the recommended VRAM for PuLID-FLUX? A: Initially, 45GB was required, but optimizations have made it feasible to run on 16GB now.

Q: What if I need a ComfyUI node? A: A compatible node isn't available yet, but developers like Cubiq are expected to release one soon.

Q: How do I deal with blurriness in output images? A: Upscaling techniques can often mitigate blurriness effectively.

Q: What's the main advantage of using PuLID-FLUX over other models? A: PuLID-FLUX offers a tuning-free, high-fidelity ID customization option, making it a robust tool for precise image customization.

Q: Can PuLID-FLUX work with non-human images? A: While primarily designed for human ID customization, experimentation with non-human images can yield interesting results. Visit the demo page to try.

By paying close attention to the latest updates and optimizations, utilizing PuLID-FLUX can be both manageable and highly rewarding for serious image generation enthusiasts.