- We present the first method capable of photorealistically reconstructing a non-rigidly - deforming scene using photos/videos captured casually from mobile phones. -
-- Our approach augments neural radiance fields - (NeRF) by optimizing an - additional continuous volumetric deformation field that warps each observed point into a - canonical 5D NeRF. - We observe that these NeRF-like deformation fields are prone to local minima, and - propose a coarse-to-fine optimization method for coordinate-based models that allows for - more robust optimization. - By adapting principles from geometry processing and physical simulation to NeRF-like - models, we propose an elastic regularization of the deformation field that further - improves robustness. -
-- We show that Nerfies can turn casually captured selfie - photos/videos into deformable NeRF - models that allow for photorealistic renderings of the subject from arbitrary - viewpoints, which we dub "nerfies". We evaluate our method by collecting data - using a - rig with two mobile phones that take time-synchronized photos, yielding train/validation - images of the same pose at different viewpoints. We show that our method faithfully - reconstructs non-rigidly deforming scenes and reproduces unseen views with high - fidelity. + In Gasket Assembly, a deformable gasket must be + aligned and pressed into a narrow channel. This task is common + for sealing surfaces in the manufacturing of automobiles, + appliances, electronics, and other products. Gasket Assembly is + a long-horizon, high-precision task and the gasket must align + with the channel and be fully pressed in to achieve a secure + fit. We present and compare 4 methods for Gasket Assembly: + one policy from deep imitation learning and three procedural + algorithms. We evaluate each using 3D printed channels with + 100 physical trials. Results suggest that deep imitation learning + can fail (lowest quartile of alignment and insertion performance) + on a straight channel in 2 of 10 trials, whereas a hybrid + procedural algorithm achieves highest quartile performance in + all 10 trials. The procedural algorithm also performs reliably + on a curved channel but poorly on a closed trapezoidal channel.
- There's a lot of excellent work that was introduced around the same time as ours. -
-- Progressive Encoding for Neural Optimization introduces an idea similar to our windowed position encoding for coarse-to-fine optimization. -
-- D-NeRF and NR-NeRF - both use deformation fields to model non-rigid scenes. -
-- Some works model videos with a NeRF by directly modulating the density, such as Video-NeRF, NSFF, and DyNeRF -
-- There are probably many more by the time you are reading this. Check out Frank Dellart's survey on recent NeRF papers, and Yen-Chen Lin's curated list of NeRF papers. -
-