Updating child node to a shape group always triggers kd-tree rebuild

[leroyvn]

System information (I'm using a custom build of v3.5 with the built-in kd-tree; I haven't been able to try with v3.6 yet):

System information:

  OS: Ubuntu 22.04.5 LTS
  CPU: AMD Ryzen 9 5900X 12-Core Processor
  GPU: NVIDIA GeForce GT 730
  Python: 3.10.13 (main, Feb 25 2024, 04:16:53) [Clang 17.0.6 ]
  NVidia driver: 470.256.02
  LLVM: 15.0.7

  Dr.Jit: 0.4.6
  Mitsuba: 3.5.2
     Is custom build? True
     Compiled with: Clang 11.1.0
     Variants:
        scalar_mono
        scalar_mono_double
        scalar_mono_polarized
        scalar_mono_polarized_double
        scalar_rgb
        scalar_spectral
        llvm_ad_rgb

This issue is related to scene tree traversal (#508 is useful to understand what is going on when changing scene dictionary keys).

Let us consider a simple script like this one:

import mitsuba as mi

mi.set_variant("scalar_rgb")

mi.Thread.thread().logger().set_log_level(mi.LogLevel.Debug)

scene_dict = {
    "type": "scene",
    "mat_sphere": {"type": "diffuse", "id": "mat_sphere"},
    "group": {
        "type": "shapegroup",
        "sphere": {"type": "sphere", "bsdf": {"type": "ref", "id": "mat_sphere"}},
        "id": "group",
    },
    "instance": {"type": "instance", "shapegroup": {"type": "ref", "id": "group"}},
    "sphere": {"type": "sphere", "bsdf": {"type": "ref", "id": "mat_sphere"}},
    "integrator": {"type": "path"},
    "sensor": {"type": "perspective"},
}

scene = mi.load_dict(scene_dict)
params = mi.traverse(scene)
print(params)

params["group.sphere.bsdf.reflectance.value"] = 1.0
params.update()

mi.render(scene)

If you run it just like this, you will see that the scene parameter update triggers a kd-tree rebuild.

• This does not occur if you change the key mat_sphere to a_mat_sphere. In that case, the material has no parent node in the update tree.
• Neither does it if you change the key sphere to a_sphere. In that case, the material is child to the sphere shape.

So my conclusion is that the kd-tree rebuild is only triggered upon traversal when updating a node that has a shape group as its parent. A careful definition of scene dictionary keys is a valid workaround, but it's not very reliable.

I ran into this issue while rendering a scene with instances of large meshes (a few 100s MB of PLY data). The kd-tree rebuilds end up dominating the processing time in that case.

There is another issue, also (maybe unrelated): upon kd-tree rebuild, no memory is released, and my process eventually runs out of memory after a dozen scene updates.