Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

3dgs背景图有点不清晰 #53

Open
fcdml opened this issue Oct 28, 2024 · 4 comments
Open

3dgs背景图有点不清晰 #53

fcdml opened this issue Oct 28, 2024 · 4 comments

Comments

@fcdml
Copy link

fcdml commented Oct 28, 2024

00000

@yifanlu0227
Copy link
Owner

This is possible. 3DGS has different reconstruction performance in different scenes, and some scene can be particularly difficult. You can adjust some hyperparameters to optimize the results, but note that ChatSim just use the vanilla 3DGS, without any advanced techniques tailored for autonomous driving data (e.g. modeling the sky separately).

It is recommended to check the latest literatures and borrow their methods to achieve a better reconstruction.

@fcdml
Copy link
Author

fcdml commented Oct 29, 2024

00000 (2)
我用原版的3dgs跑了一下 比原来好些 想问下chatsim单独做了相机参数的转换 将waymo的坐标转成colmap 跟这个有关系么

@yifanlu0227
Copy link
Owner

确实也不排除这个scene里colmap标出来的位姿比原来的效果要差的可能。可以尝试在config里位姿直接用cams_meta_waymo.npy,这是从waymo原始数据集里提取出来的位姿

data
`-- waymo_multi_view
    |-- ...
    `-- segment-1172406780360799916_1660_000_1680_000_with_camera_labels
        |-- 3d_boxes.npy                # 3d bounding boxes of the first frame
        |-- images                      # a clip of waymo images used in chatsim (typically 40 frames)
        |-- images_all                  # full waymo images (typically 198 frames)
        |-- map.pkl                     # map data of this scene
        |-- point_cloud                 # point cloud file of the first frame
        |-- cams_meta.npy               # Camera ext&int calibrated by metashape and transformed to waymo coordinate system.
        |-- cams_meta_metashape.npy     # Camera ext&int calibrated by metashape (intermediate file, relative scale, not required by simulation inference)
        |-- cams_meta_colmap.npy        # Camera ext&int calibrated by colmap (intermediate file, relative scale, not required by simulation inference)
        |-- cams_meta_waymo.npy         # Camera ext&int from original waymo dataset (intermediate file, not required by simulation inference)
        |-- shutters                    # normalized exposure time (mean=0 std=1)
        |-- tracking_info.pkl           # tracking data
        |-- vehi2veh0.npy               # transformation matrix from i-th frame's vehicle coordinate to the first frame's vehicle 
        |-- camera.xml                  # calibration file from Metashape (intermediate file, not required by simulation inference)
        `-- colmap/sparse_undistorted/[images/cams_meta.npy/points3D_waymo.ply]   # calibration files from COLMAP (intermediate file, only required when using 3dgs rendering)
        

@fcdml
Copy link
Author

fcdml commented Oct 29, 2024

我好像知道问题在哪了 如果自己安装colmap 提取点云等信息 跑出来的效果就是第一张图 如果用你矫正后的colmap数据就没问题 生成的类似用于之前的第二张图 效果贴近3dgs生成的 可能自己安装colmap还不够 还需要调colmap 想问下 你大概是怎么调的

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants