Content-Style Composition with RF-Inversion+LoRA

[DexiangHong]

I wonder how to realize the "Content-Style Composition with RF-Inversion+LoRA" showed in the prjiect page

[LituRout]

You can follow these steps:
(1) LoRA finetuning for "a sks dog" or "a sks cat".
(2) Take a reference style image $(Y_0)$ and invert it using RF-Inversion Algorithm-1 (https://openreview.net/pdf?id=Hu0FSOSEyS) to get $y_1$
(3) Run Algorithm-2 (https://openreview.net/pdf?id=Hu0FSOSEyS) initialized at $y_1$. Instead of using the standard base model (unconditional vector field) in line 7 of Algorithm-2, use the base model (unconditional vector field) + LoRA weights.