logo
pub

PuLID-FLUX: ID Customization Solution for Flux AI

Introduction to PuLID-FLUX

PuLID-FLUX is a new tuning-free ID customization solution for the FLUX.1-dev model. It maintains the original model behavior while allowing for high-fidelity ID customization.

What is PuLID-FLUX?

PuLID-FLUX-v0.9.0 is a model that lets you customize identification without tuning for the FLUX.1-dev model. It's highly flexible and maintains the original model's performance while adding unique IDs to images.

Setup and Integration

Installing PuLID-FLUX

  1. Clone the Repository: Head to the PuLID GitHub page at PuLID GitHub.
  2. Set Up Environment: Follow the guide to set up the required environment (Python version, dependencies, etc.).
    git clone https://github.com/ToTheBeginning/PuLID.git
cd PuLID
pip install -r requirements.txt
  3. Download Necessary Models:
  4. Run the Application:
    python app_flux.py

Using Different Memory Configurations

Naive bf16

  • Run with bf16 directly:
    python app_flux.py
    • Peak memory is under 45GB.

bf16 + Offload

  • Use offloading to save memory:
    python app_flux.py --offload
    • Peak memory is under 30GB.

fp8 + Offload (For Consumer-Grade GPUs)

  • Ensure additional requirements are installed:
    pip install -r requirements-fp8.txt
  • Use flux-dev-fp8 checkpoint:
    python app_flux.py --offload --fp8 --onnx_provider cpu
- Peak memory is under 15GB (for GPUs with 16GB memory).
- For 24GB graphic memory users:
python app_flux.py --offload --fp8
- Peak memory is under 17GB.
    • Note: fp8 has quality degradation in face details compared to bf16.

bf16 + Aggressive Offload

  • Use more aggressive offloading:
    python app_flux.py --aggressive_offload
    • Peak memory is around 23GB, but the process will be very slow.

Online Demo

ComfyUI

  • Stay tuned for the community implementation of ComfyUI support.

Visual Results

puLID_flux_results

Useful Tips

Parameter Tuning

1. Timestep to Start Inserting ID

  • Controls the timing of ID insertion. Sets ID fidelity and editability trade-off.
  • Usage:
    • Realistic images: Start at timestep 4.
    • Higher ID similarity: Lower the value.
    • Stylized images: Set it to 0-1.
start_id

2. True CFG Scale

  • FLUX.1-dev uses guidance distillation with a modulated guidance scale to simulate CFG.
  • Compare true CFG with fake CFG for photorealistic and stylized scenes.
fake_cfg_vs_true_cfg_fidelity

Technical Details

  • Switched ID encoder from MLP to Transformer.
  • Inspired by Flamingo, additional cross-attention blocks are inserted every few DIT blocks to handle ID features interaction with DIT image features.
  • Optional acceleration method similar to SDXL-Lightning, yet not essential for training PuLID.

License

As long as you use FLUX.1-dev model, you should follow the FLUX.1-dev model license.

Contact

For queries or suggestions, contact Yanze Wu or open an issue/discussion on the GitHub page.

Additional FAQs

Q: How can I try PuLID-FLUX? A: You can start by visiting their GitHub page at PuLID-FLUX GitHub and following the setup instructions.

Q: Does PuLID-FLUX support commercial use? A: The license on the PuLID-FLUX GitHub suggests commercial use is allowed, but integrated models like Insightface have commercial restrictions.

Q: What's the recommended VRAM for PuLID-FLUX? A: Initially, 45GB was required, but optimizations have made it feasible to run on 16GB now.

Q: What if I need a ComfyUI node? A: A compatible node isn't available yet, but developers like Cubiq are expected to release one soon.

Q: How do I deal with blurriness in output images? A: Upscaling techniques can often mitigate blurriness effectively.

Q: What's the main advantage of using PuLID-FLUX over other models? A: PuLID-FLUX offers a tuning-free, high-fidelity ID customization option, making it a robust tool for precise image customization.

Q: Can PuLID-FLUX work with non-human images? A: While primarily designed for human ID customization, experimentation with non-human images can yield interesting results. Visit the demo page to try.

By paying close attention to the latest updates and optimizations, utilizing PuLID-FLUX can be both manageable and highly rewarding for serious image generation enthusiasts.

SHARE