Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Remove row group filter for map and list (facebookincubator#10510)
Summary: There is a fast path for filter all null columns in `ScanSpec` based on `statistics` by assuming all null happens when `NumberOfValues` equals `0`. But when do `filterRowGroups` against one line file with null `Map`, this rule will apply to the `Key` column reader, which has `testNull()` as `false` and finally decide this rowgroup should be skipped. It means this rule doesn't apply for the columns like `Map::Key` . But limited to the info that statistics can provide, we don't have a better way to refine this fast path, so we remove the `filterRowGroups` in `MapColumnReader` in this PR. Pull Request resolved: facebookincubator#10510 Reviewed By: Yuhta Differential Revision: D60835181 Pulled By: kagamiori fbshipit-source-id: 9284ec94800ebe038a6ec1bbc99be0b994fd38cd
- Loading branch information