How did you handle the missing geometry in the compromised scenes?

[Yonggie]

Hi, as we all know that HM3D has many incomplete scenes, many of which leave black holes resulting the depth missing and even spatial miss alignment, such as:

How did you walk around these flaws of the compromised scenes while fully explore the 3D space smoothly?

[BoySun045]

Hi @Yonggie , thanks for raising this point.

You are absolutely right that imperfect 3D assets in HM3D can introduce issues in exploration experiments. When testing on HM3D, we designed heuristic to reduce the impact of incomplete geometry. For example, black holes (missing geometry) can be detected from invalid depth values, and we remove any frontiers registered nearby to prevent them from affecting exploration. This filtering is also supported internally by our frontier manager implementation (see https://github.com/cvg/FrontierNet/blob/main/frontier/manager.py#L671).

Regarding spatial misalignment, I am not fully certain about the exact scenario you are referring to. However, based on our experimental observations, minor misalignments, such as small height differences, did not lead to a significant drop in exploration performance.

Overall, imperfect geometry will inevitably create challenges and can impact overall system performance. We cannot fully explore every scene in HM3D, so there may still be edge cases we have not fully studied. Hope this helps and we would be very happy to hear any advice or experience from your side.

[Yonggie]

Hi @Yonggie , thanks for raising this point.

You are absolutely right that imperfect 3D assets in HM3D can introduce issues in exploration experiments. When testing on HM3D, we designed heuristic to reduce the impact of incomplete geometry. For example, black holes (missing geometry) can be detected from invalid depth values, and we remove any frontiers registered nearby to prevent them from affecting exploration. This filtering is also supported internally by our frontier manager implementation (see https://github.com/cvg/FrontierNet/blob/main/frontier/manager.py#L671).

Regarding spatial misalignment, I am not fully certain about the exact scenario you are referring to. However, based on our experimental observations, minor misalignments, such as small height differences, did not lead to a significant drop in exploration performance.

Overall, imperfect geometry will inevitably create challenges and can impact overall system performance. We cannot fully explore every scene in HM3D, so there may still be edge cases we have not fully studied. Hope this helps and we would be very happy to hear any advice or experience from your side.

Thank you for the quick replies and inspiration!