CUDA out of memory, multiple gpu seem not working.

[jacob-kang]

Hi.
Thanks for your great work.

I tried the demo but CUDA out of memory was occred.
I have 4 RTX3090 gpus. and I used CUDA_VISIBLE_DEIVES=0,1,2
But one gpu works only.

Any suggestion for multi gpu and for light gpu user?

Thanks.

[tsunghan-wu]

Hi,

Thank you for reaching out and showing interest in our work. Currently, our support is limited to single GPU configurations.

For users with lighter GPU resources, here are some suggestions that might help:

1. Model Parallelization: Our framework comprises several modular components, including the object detector, base diffusion models, SAM, and the SDXL refinement. We recommend distributing these modules across different GPUs to manage memory consumption more effectively. BTW, If you're able to setup this distribution via elegant config files, please consider submitting a pull request. We're very keen on incorporating this capability and would greatly appreciate your contribution!
2. Opt for Lightweight Models: Consider using more resource-efficient models. For example, switching from the SAM-ViT-huge module to SAM-ViT-base here could reduce the memory consumption while minimizing performance drop. Additionally, you can skip the SDXL refinement step or replace the OWLv2 model with its predecessor, OWL-ViT v1. However, please be aware that altering the object detection model may necessitate adjustments to the detection threshold for optimal results.

I hope these tricks helps. LMK if you need further assistance or have any more questions!