You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
A query has parquet scan with complex data type fallbacked.
If I set spark.sql.parquet.enableVectorizedReader = False and run the query twice, the first one is correct, but the second one wrongly add ColumnartoRow, caused error UnsafeRow cannot be cast to org.apache.spark.sql.vectorized.ColumnarBatch
Backend
VL (Velox)
Bug description
A query has parquet scan with complex data type fallbacked.
If I set spark.sql.parquet.enableVectorizedReader = False and run the query twice, the first one is correct, but the second one wrongly add ColumnartoRow, caused error
UnsafeRow cannot be cast to org.apache.spark.sql.vectorized.ColumnarBatch
First query plan:
The second query plan:
Similarly if I set spark.sql.parquet.enableVectorizedReader = True, the first run reports error and the second run successed.
Spark version
Spark-3.2.x
Spark configurations
No response
System information
No response
Relevant logs
No response
The text was updated successfully, but these errors were encountered: