Reconstruction and editing with gamma=0 and eta=0

[sashapff]

Can you please explain why, when using the Euler scheme without modifications with the controllers from the article (the case when gamma=0 and eta=0), the reconstruction and the edited image do not fall into the data distribution? Is it because such a large computational error accumulates or why?

I attach an example for prompts 'a cat' for the reconstruction (guidance_scale=1.) and 'a tiger' for the editing (guidance_scale=3.5).

[LituRout]

Hi Alexandra, thanks for your interest in our paper. Are you using the prompt 'a cat' with guidance scale 1.0 while inverting the cat image with gamma=0? If yes, then this is wrong because you are removing the catness along with inverting the image. The catness comes from the text conditional score for nonzero guidance. Ideally, you would want to invert with zero guidance scale (no text conditioning), which would preserve catness in the structured noise y_1. Then, if you initialize the reverse flow with y_1, you should get more cat looking image in reconstruction. Please let me know if this resolves your concern. Happy to provide any further clarifications.

[EthanLiang99]

Great work! I’m curious about the parameter settings used for Figure 3 in the paper. Was it reconstructed using stop_timestep = 28? Additionally, could you share the values for eta and gamma?

I would appreciate it if you could provide more details about the reconstruction process. Thanks! 😊

[sashapff]

Thank you for your earlier reply. It has become a little clearer. However, I still have a question, can you tell us what eta and gamma parameters should be used to get the most accurate reconstruction of the original image?

[ohayonguy]

Did anyone manage to obtain a good image reconstruction with $$\gamma=\eta=0$$ (via Proposition 3.1 in the paper)? @LituRout should it work?

[LituRout]

Note that Proposition 3.1 in the paper says perfect reconstruction under ideal conditions. In practice, you don't want to run the inversion process all the way till end when setting $\gamma=\eta=0$ for a good reconstruction. There is a trade-off between reconstruction quality and the number of inversion steps. If you want to understand this better, please look at our numerical simulations in Appendix D.3 (https://openreview.net/pdf?id=Hu0FSOSEyS), where we study this behavior in a more controlled experimental setting. We also have the exact setting ($\gamma=\eta=0$) in Table 5 in Appendix D.3.

[LituRout]

Thank you for your earlier reply. It has become a little clearer. However, I still have a question, can you tell us what eta and gamma parameters should be used to get the most accurate reconstruction of the original image?

@EthanLiang99 and @sashapff : It is important to note that RF-Inversion is completely training-free, which means you need to tune these two hyper-parameters according to your original image. Images that seem natural for the underlying model (e.g. Flux) can be easily reconstructed without much hyper-parameter tuning. If you are starting from a typical image, then you can set $\gamma=0$, then use a higher $\eta$ value for better (training-free) reconstruction. You can achieve this by increasing the start and stopping time for $\eta$ in our Algorithm 2. Please refer to Table 4 (https://openreview.net/pdf?id=Hu0FSOSEyS) for a list of hyper-parameters used in different settings. You can always tune these for better results in your applications.