Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ParquetDataset support configuration with default value #156

Open
Markz2z opened this issue Jan 22, 2024 · 0 comments · Fixed by #157 · May be fixed by #160
Open

ParquetDataset support configuration with default value #156

Markz2z opened this issue Jan 22, 2024 · 0 comments · Fixed by #157 · May be fixed by #160
Assignees
Labels
enhancement New feature or request

Comments

@Markz2z
Copy link
Collaborator

Markz2z commented Jan 22, 2024

User Story

The fixed-length features in TFRecord support configuration with default values(https://www.tensorflow.org/api_docs/python/tf/io/FixedLenFeature), but currently, Parquet does not support this feature. If encountering a non-existent feature, an error will be thrown, such as 'No Column called xx found in yy'.

So when we transition from training with TFRecord to Parquet, there may be some functionality alignment issues. We hope that Parquet can also support configuration with default values.

@2sin18 2sin18 self-assigned this Feb 19, 2024
@2sin18 2sin18 added the enhancement New feature or request label Feb 19, 2024
@2sin18 2sin18 linked a pull request Apr 19, 2024 that will close this issue
@2sin18 2sin18 reopened this Apr 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
2 participants