Rendering results of nerf_synthetic lego validation dataset by provided single-scene model

[leo-frank]

Description:

I use the pretrained model that you provided at lego synthetic by provided single-scene model at https://drive.google.com/file/d/1IbhbBr5XfxQz0jSQM3nLX_htTbvc59kj/view?usp=share_link
the command is:
CUDA_VISIBLE_DEVICES=2 python3 eval.py --config configs/gnt_blender.txt \ --eval_dataset nerf_synthetic \ --eval_scenes lego --run_val\ --expname gnt_author_pretrained_single_lego \ --ckpt_path out/gnt_lego_from_single_ckpt/pretrained_lego_model_435000.pth \
(gnt_blender.txt is not modified)

Results:

1. rendering results on validation set seems not correct, since the color has some green

2. but the metrics seems resonable as this screenshot shows:

Questions:

I wonder the cause of such unexpected rendering results.Is it caused by my command or anything else?

[YiftachEde]

Have you solved this issue? it happens all the same using gnt_full model as a checkpoint model.

[leo-frank]

No, I can't solve it. I tried 3 types of model:

1. single-scene model provided by author
2. cross-scene model provided by author
3. single-scene model trained by myself
   all of these have the same issues.

[alaneze123]

Hi, have you solved this problem?

[lan-creator]

It also happened to me. You can check the RGB value range here:

GNT/eval.py

Lines 187 to 189 in 53e0fc3

	rgb_coarse = rgb_coarse.permute(1, 2, 0).detach().cpu().numpy()
	filename = os.path.join(out_folder, prefix[:-1] + "_{:03d}_coarse.png".format(global_step))
	imageio.imwrite(filename, rgb_coarse)

In the original code, there is no value range changed as the rgb_coarse may have a distribution between around [0, 1] and you need to change it to [0, 255] and clip outliers. There is function in utils and we can directly use:

GNT/utils.py

Line 28 in 53e0fc3

to8b = lambda x: (255 * np.clip(x, 0, 1)).astype(np.uint8)