23.0.0.md

23.0.0 (2023-04-21)

Full Changelog

Breaking changes:

• Add new physical rule CombinePartialFinalAggregate #5837 (mingmwang)
• feat: Remove compact row since it's no longer used #6021 (yjshen)

Implemented enhancements:

• feat: extend substrait type support, including type variations #5775 (waynexia)
• feat: hyperbolic functions #5884 (izveigor)
• feat: Support SQL filter clause for aggregate expressions, add SQL dialect support #5868 (yjshen)
• feat: Implement the bitwise_not in NotExpr #5902 (RTEnzyme)
• feat: implementation of the constant "Pi" #5965 (izveigor)
• feat: allow the customization of analyzer rules #5963 (waynexia)
• feat: extra sqllogictests for scalar functions #5887 (izveigor)
• feat: degrees and radians #6023 (izveigor)
• feat: allow TableScan without projection #6032 (waynexia)
• feat: support month and year interval for date_bin on constant data #5982 (NGA-TRAN)

Fixed bugs:

• fix: type coercion for expr/subquery in InSubquery #5883 (jackwener)
• fix: binaryExpr not supported for LargeUtf8 #5896 (jackwener)
• fix: match boolean null for substrait #5947 (waynexia)
• fix: return NotImplemented when execute SELECT INTO syntax #5945 (r4ntix)
• fix: support date - timestamp and timestamp - date #5960 (jackwener)
• fix: do not depend on time 0.1 #5972 (crepererum)
• fix: largeUtf8 don't support is distinct from #5993 (jackwener)
• fix: Do not panic on invalid placeholders #5998 (Weijun-H)
• fix: Allow values in subqueries #6018 (scsmithr)
• fix: split "union" and "interleave" #6045 (crepererum)

Documentation updates:

• docs: improve expressions.md #5978 (izveigor)
• minor: add developer's guide link in README.md #6055 (jackwener)

Merged pull requests:

• fix: type coercion for expr/subquery in InSubquery #5883 (jackwener)
• Minor: fix the architecture slide url #5920 (gaoxinge)
• Count agg support multiple expressions #5908 (allenma)
• feat: extend substrait type support, including type variations #5775 (waynexia)
• fix: binaryExpr not supported for LargeUtf8 #5896 (jackwener)
• Clean up SortExec creation and add doc comments #5889 (alamb)
• Improve file scan time opening metric to include start_next_file #5900 (alamb)
• Remove unsafe code (transmute) in datafusion-proto #5946 (alamb)
• support date_trunc with millisecond and microsecond and keep the same type as the input #5818 (Weijun-H)
• fix: match boolean null for substrait #5947 (waynexia)
• feat: hyperbolic functions #5884 (izveigor)
• Planner: normalize_ident only when enable_ident_normalization is enabled #5785 (ayushdg)
• Improve contributor guide and main API landing page #5921 (alamb)
• minor: Refactor row_hash implementation #5936 (mustafasrepo)
• Specialize Primitive Cursor -- make sorts / merges on a single primitive column faster #5897 (tustvold)
• just match for BinaryExpr Date/Time +/- Interval #5932 (jackwener)
• Improve avg/sum Aggregator performance for Decimal #5866 (mingmwang)
• fix: return NotImplemented when execute SELECT INTO syntax #5945 (r4ntix)
• Update arrow 37 #5782 (tustvold)
• Scalar arithmetic should return error when overflows. #5811 (zhzy0077)
• feat: Support SQL filter clause for aggregate expressions, add SQL dialect support #5868 (yjshen)
• test: add Between UT for type_coercion #5929 (jackwener)
• feat: Implement the bitwise_not in NotExpr #5902 (RTEnzyme)
• Minor: Add DFField.with_nullable(bool) #5966 (comphead)
• Move error check from pipeline fixer to pipeline checker #5938 (mustafasrepo)
• when inferring the schema of compressed CSV, decompress before newline-delimited chunking #5860 (jiangzhx)
• Update sqlparser requirement from 0.32 to 0.33 #5957 (dependabot[bot])
• fix: support date - timestamp and timestamp - date #5960 (jackwener)
• minor: Add Expr::between to clean up boilerplate #5967 (alamb)
• Add new physical rule CombinePartialFinalAggregate #5837 (mingmwang)
• Specialized Cursor for StringArray and BinaryArray #5964 (tustvold)
• add an example of using DataFrame to create a subquery #5961 (jiangzhx)
• feat: implementation of the constant "Pi" #5965 (izveigor)
• fix: do not depend on time 0.1 #5972 (crepererum)
• Minor: fix wrong code comment #5979 (viirya)
• [DOCS]: consolidate doc site content simplify navbar #5962 (alamb)
• minor: port some expr tests to sqllogictests, improve error message #5968 (alamb)
• minor: fix doctest that runs accidentally #5989 (yjshen)
• Row AVG accumulator support Decimal type #5973 (mingmwang)
• Treat Partition by columns as set for window functions #5951 (mustafasrepo)
• feat: allow the customization of analyzer rules #5963 (waynexia)
• feat: extra sqllogictests for scalar functions #5887 (izveigor)
• docs: improve expressions.md #5978 (izveigor)
• Minor: Improve doc comments in FileStream #5898 (alamb)
• Update prost-build requirement from =0.11.8 to =0.11.9 #5987 (dependabot[bot])
• Don't use parquet file offset for file range pruning #5997 (tustvold)
• Streaming Memory Reservation in SHJ #5937 (metesynnada)
• Temporal datatype support for interval arithmetic #5971 (berkaysynnada)
• Remove optimize_children and replace with map_children #5984 (2010YOUY01)
• doc: fix site sidebar logo position #5990 (Jefffrey)
• fix: largeUtf8 don't support is distinct from #5993 (jackwener)
• Proper resolution for old name in with_column_renamed #5992 (Jefffrey)
• minor:update subquery example #6011 (jiangzhx)
• fix: Do not panic on invalid placeholders #5998 (Weijun-H)
• fix: Allow values in subqueries #6018 (scsmithr)
• chore: make JsonOpener and CsvOpener public #6004 (WenyXu)
• update count_wildcard_rule for more scenario #6010 (jiangzhx)
• Add analyzer output to verbose explain #6020 (Jefffrey)
• Minor: Fix compilation error #6029 (viirya)
• Open/Closed bounds for interval arithmetic #6007 (berkaysynnada)
• minor:move log_plan to utils #6030 (jiangzhx)
• Use OwnedTableReference for subquery aliases #6022 (scsmithr)
• Update sql doc #6025 (Jefffrey)
• feat: Remove compact row since it's no longer used #6021 (yjshen)
• Add test for handling precision overflow when casting from integer to decimal #6041 (viirya)
• feat: degrees and radians #6023 (izveigor)
• feat: allow TableScan without projection #6032 (waynexia)
• refactor: make FirstSelector not to have update and merge #6038 (Weijun-H)
• Port math.rs to sqllogictest #6037 (2010YOUY01)
• Row accumulator support update Scalar values #6003 (mingmwang)
• Minor: tweak docuement of sort enforcement optimizer rule #6054 (waynexia)
• Decimal multiply kernel should not cause precision loss #5980 (viirya)
• Clean up rustdoc and add doc lint #6044 (alamb)
• Minor: port more create_drop table tests to sqllogictests #6031 (jiangzhx)
• minor feat: impl FromStr for JoinType enum #6033 (nkarpov)
• Float support on interval artihmetics #6048 (metesynnada)
• minor: add developer's guide link in README.md #6055 (jackwener)
• feat: support month and year interval for date_bin on constant data #5982 (NGA-TRAN)
• Update rust_lint.sh to include newly added cargo doc check #6070 (alamb)
• Add support for UDAF in physical plan serialization #6063 (thinkharderdev)
• fix: split "union" and "interleave" #6045 (crepererum)
• Minor: assorted schema handling debug improvements #6076 (crepererum)