New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

您好，我在运行make_h5dataset_for_pretrain.py时遇到了问题，想请教一下您 #51

Open

zxc1216 opened this issue Nov 29, 2024 · 1 comment

zxc1216 commented Nov 29, 2024

当将数据保存为 HDF5 数据集时，chunks 参数定义的块大小在某些维度上超出了数据本身的形状（data.shape），导致不兼容。具体信息是：chunk shape 为 (62, 200)，而数据的实际形状为 (30, 251400)。

ValueError: Chunk shape must not be greater than data shape in any dimension. (62, 200) is not compatible with (30, 251400)

Owner

935963004 commented Dec 13, 2024

你需要更改chunk shape，让其符合你的数据集，代码中的chunk shape是为62导的数据设置的

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment