vqnsp training batch size

[zeydabadi]

Hi,

According to Table 3 of the paper, the batch size should be 1024, while in the example command line you provided, it is set to 128.
Could you clarify which one is correct?

OMP_NUM_THREADS=1 torchrun --nnodes=1 --nproc_per_node=8 run_vqnsp_training.py
--output_dir ./checkpoints/vqnsp/
--log_dir ./log/vqnsp/
--model vqnsp_encoder_base_decoder_3x200x12
--codebook_n_emd 8192
--codebook_emd_dim 64
--quantize_kmeans_init
--batch_size 128
--opt adamw
--opt_betas 0.9 0.99
--weight_decay 1e-4
--warmup_epochs 10
--epochs 100
--save_ckpt_freq 20

[935963004]

We use 8 GPUs so the total batch size is 8 * 128 = 1024.