Skip to content

Fix: Properly Respect CUDA_VISIBLE_DEVICES Setting#40

Open
LeonGuertler wants to merge 1 commit intosail-sg:mainfrom
LeonGuertler:cuda_visible_fix
Open

Fix: Properly Respect CUDA_VISIBLE_DEVICES Setting#40
LeonGuertler wants to merge 1 commit intosail-sg:mainfrom
LeonGuertler:cuda_visible_fix

Conversation

@LeonGuertler
Copy link
Copy Markdown

This ensures that setting CUDA_VISIBLE_DEVICES correctly restricts execution to the specified GPU. Previously, even when assigning a specific GPU ID (e.g., CUDA_VISIBLE_DEVICES=1), the code would still run on gpu:0. This fix ensures that the selected GPU is properly honored.

✅ Tested on a single-GPU setup – this allows running multiple experiments in parallel across different GPUs.
⚠️ Multi-GPU setups haven't been tested yet.

@lkevinzc
Copy link
Copy Markdown
Collaborator

@LeonGuertler Thanks for spotting the issue and suggest the fix!

  1. Could you also consider multi-gpu cases, where e.g., the user specify CUDA_VISIBLE_DEVICES="4,5,6,7", so the actors should take devices 4,5 and learners take devices 6,7 (without --collocate), or both actors and learners take devices 4,5,6,7 if --collocate is set.
  2. Please run make format to align the code format so that it looks nicer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants