The issue of the time ratio between pre-training and fine-tuning

[dajiebi]

Why does Univla set the pre-train time particularly short while the fine-tuning time particularly long compared to other VLAs?

[retsuh-bqw]

This is actually not the case. UniVLA is very compute efficient in trianing is due to we are learning at a compact latent action space, and there was evidences that UniVLA after 20k steps of pretraining already give us a decent result on LIBERO while further pre-training bring marginal improvement. So we've decided to pre-train UniVLA for only 20k steps whatsoever. （But further pre-training might be beneficial for other benchmarks).

In terms of fine-tuning, UniVLA is more efficient or equvalent to other VLAs as well. For example, OpenVLA-OFT need 150k steps of training on LIBERO-Spatial, while UniVLA needs 30k. We trained UniVLA on CALVIN about 2 days (with 8 A100) while we trained OpenVLA for a week to get this far.

Another issue that may cause the UniVLA is trained slower is that we are running latent action model on-the-fly to generate pseudo labels. This part of latency can be deminished if we processed the dataset offline with the latent action model before VLA training.

[dajiebi]

Thank you for answering your questions and providing guidance! So the reason why the training time for each step has increased is that LAM is extracting latent action and how can I process my dataset with LAM model before training?