skip training and directly evaluate

[jinzhuoran]

Hi there,
Thanks for your work on this.
How can I skip training and directly evaluate?

[jinzhuoran]

By the way, it seems that setting CUDA_VISIBLE_DEVICES=4,5,6,7 does not work.
@lkevinzc

[lkevinzc]

Hi @jinzhuoran , thank you for your question and sorry for the delayed response.

1/ Yes, for evaluation, you could run script which loads model using vllm and do the inference as you want. An example can be found here for math evaluation: https://github.com/sail-sg/understand-r1-zero/blob/main/evaluate_model.py. Please modify as you want for your own evaluation!

2/ Yea this is an issue because we will directly manage CUDA_VISIBLE_DEVICES given the number of GPUs specified. I am doing a fix and will be ready soon!

[lkevinzc]

Hi @jinzhuoran , the GPU issue should be fixed. Now you can set CUDA_VISIBLE_DEVICES=4,5,6,7 and set --gpus 4 to make it work properly.