[RELEASE] Community reimplementation using TRL SFTTrainer

[MilkClouds]

Hi, great work on VLA-0! The simplicity of representing actions as text tokens is really elegant.

I made a minimal reimplementation using Hugging Face TRL's SFTTrainer for my own learning:
https://github.com/MilkClouds/vla0-trl

A few notes:

• ~1,200 lines (relies heavily on TRL abstractions)
• Flash Attention enabled by default
• Tested on LIBERO, gets ~90% avg (slightly below paper results, likely due to hyperparameter differences)

Not intended as a replacement—just a simpler entry point for those already familiar with the HF/TRL ecosystem. All credit goes to the original authors.

If there's any concern about this, please let me know.

Thanks again for open-sourcing the code and weights!

[imankgoyal]

Nice work @MilkClouds!

Quick question, why don’t you use Qwen2.5_VL? That might affect performance.

Another reason for the gap might be data augmentation. Does your implementation take care of it?

[MilkClouds]

@imankgoyal Thanks for the quick reply! Mention of Qwen2-VL instead of Qwen2.5-VL was from my simple mistake. I checked that code is actually using Qwen2.5-VL and going to fix the docs. I also took care of image augmentation and currently I'm suspecting add_vision_id on processor's argument made some difference. (transformers v4.51.3 ignore this argument and does NOT insert vision id, where transformers v4.57.3 on vla0-trl respecting this argument) I'm going to run another train with fixed version.

[vovw]

have you tried doing lora?

[MilkClouds]

Update: After MilkClouds/vla0-trl#1, LIBERO performance on vla0-trl increased to 92.2, but still bit lower than vla0.

@vovw Nope, I haven't, but it would be easy to apply LoRA to vla0-trl since vla0-trl is closely related to HuggingFace ecosystem.

[vovw]

interesting thanks!

[JiaH-w]

Hi, @MilkClouds, thanks for sharing your reproduction results!

I also encountered a similar issue #29 and got ~92.5 after retraining. Additionally, I tested with the official released weights and got 92.7.

Could you also try evaluating with the official released weights and share your results? It would be helpful to see if we get similar numbers, which might help narrow down whether the gap is related to training or evaluation.

Also, may I ask what GPU you used for the evaluation? The author mentioned that the performance difference might be related to GPU configurations.

Thanks!