[VL] ANSI MODE

[yonger001]

Description

Any plan to add the support of ANSI mode for Velox backend???

[PHILO-HE]

@zhouyuan, @weiting-chen

[zhouyuan]

@yonger001
Hi, is this a real requirement from Spark production env? As far as I know ANSI is not very commonly used in Spark production env due to historical reasons.

We did a quick investigation ANIS support in Spark and find the support would be into two parts:

• spark.sql.ansi.enabled
• spark.sql.storeAssignmentPolicy

The first part covers Arithmetic and Cast mostly, which contains many functions. The second part is mostly for data write, which maybe a bit easier. For both of these parts, it may need to discuss with Velox community to align with their ANSI impl as it's also used in Presto.

Please also note Gluten is in the process of migrating to Apache project. We are hoping to have more support from the community.

Thanks,
-yuan

[mbasmanova]

We are also interested in ANSI SQL support. We are migrating our Spark workloads from HQL into ANSI SQL and would like to evaluate Gluten to achieve better performance and efficiency.

[zhouyuan]

related Velox discussion: facebookincubator/velox#3869