Generated image pixels were very different from the original image

[DongxiaLiu15]

hi!Thanks for your work.

When I edited with RF-inversion, the generated image pixels were very different from the original image. Here are my codes and results. I would like to know how to improve it.

  import torch
  import requests
  import PIL
  from io import BytesIO
  import yaml
  import os
  from PIL import Image
  from pipeline_flux_rf_inversion import RFInversionFluxPipeline
  from diffusers import FluxImg2ImgPipeline
  
  with open("/mnt/zhangjy80/liudx15/FlowEdit_multiobject/edits.yaml", "r") as f:
      config_list = yaml.safe_load(f)
  
  
  for idx, config in enumerate(config_list[1:]):
      input_img_path = config["input_img"]
      target_prompt = config["target_prompts"][0]
  
    
      image = Image.open(input_img_path).convert("RGB")
      image = image.resize((1024, 1024))
      pipe_invert = RFInversionFluxPipeline.from_pretrained(
      "../modelzoo_jiayu/FLUX.1-dev", torch_dtype=torch.float16).to('cuda:7')
      pipe_invert.enable_model_cpu_offload()
  
      # Inversion
      inverted_latents, image_latents, latent_image_ids = pipe_invert.invert(
          image=image, 
          num_inversion_steps=28, 
          gamma=0.5
)

# edit
edited_image = pipe_invert(
    prompt=target_prompt,
    inverted_latents=inverted_latents,
    image_latents=image_latents,
    latent_image_ids=latent_image_ids,
    start_timestep=0,
    stop_timestep=0.25,
    num_inference_steps=28,
    eta=0.9
).images[0]

# save
output_path = f"results/edited_image_{idx}.png"
os.makedirs(os.path.dirname(output_path), exist_ok=True)
edited_image.save(output_path)

print(f"✅ Saved edited image to {output_path}")`

[LituRout]

Hi @DongxiaLiu15
Thanks for your question. If you check Appendix D.2 of our paper: https://openreview.net/pdf?id=Hu0FSOSEyS, you will see that there is a trade-off between faithfulness to original image and realism, which you can control by tuning the hyperparamters. If you need the generated image to be very similar to the original image, you can try to increase the starting time (currently $0$) or stopping time (currently $0.25$) or both while keeping $\eta=0.9$ as you have chosen above.

I hope this resolves your issue.

[LituRout]

with start step=5, stop step=16, $\eta = 1$, you can already get very close to the original image. Feel free to tune these parameters further to get your desired results.

[DongxiaLiu15]

Thank you very much for your reply, which is very helpful to me. I have another question, which is equivalent to the null prompt inversion. Do you try add the original prompt? Will there be any difference in the result?

[LituRout]

If you want to subtract some characteristics from the original image, then you can use the prompt corresponding to that feature during inversion.