Ray loader to support mini-batch training

[ziyi-zhang]

This PR introduces a ray loader that wraps a list of sensors to enable mini-batch optimization.
The sampling strategy aims to uniformly cover all pixels before reshuffling (i.e., permutation-based sampling), but this is not strictly guaranteed when a tile size is specified.

TODO

• Tiling for GPU coherence.
• Support ptracer & projective: don't sample pixels, sample a batched K source sensors, where K is the smallest number to saturate pixels_per_batch.
• FP16 support (FP16 will not be supported as mi.render by default forces float32)
• Include ray loader in one of the tutorials
• Add tests

[merlinND]

Hi @ziyi-zhang,

I haven't ready the implementation so maybe this is already supported, but one feature that could be quite helpful: ensure that rays can be sampled as e.g. 2x2, 4x4, etc tiles from the same sensor. The user could select the tile size to get more or less coherence.

[ziyi-zhang]

Hi @merlinND,

Good point.

What do you think is the best solution if the film size is not divisible by the patch size, e.g., 4x4 patch for a 12x10 film?
A related question is the division of pixels_per_batch vs. total_pixels. I currently pad total reference pixels with random pixels (so some pixels may appear twice) to make sure it's a multiple. But it may be better to just pad with dummy pixels?

[merlinND]

Hi @ziyi-zhang,

I haven't checked the implementation, but it sounds like you are trying to exhaustively enumerate all batches (and maybe now all NxN ray tiles) out of the total pixels, without overlap.
I believe it would be fine to instead sample batches / tiles with overlap allowed, which means the tiles / batches just need to fit in the ref image dimensions, rather than being divisible.

[ziyi-zhang]

Set tile size to 4 by default.

• Very large tile sizes are bad.
• Tile size not a multiple of 2 is bad.