Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add sqlite test files into sqllogictests #13935

Closed
wants to merge 11 commits into from

Conversation

Omega359
Copy link
Contributor

Which issue does this PR close?

Closes #13812

Rationale for this change

Add most of the sqlite test suite to Datafusion sqllogictests. Note: THESE TESTS DO NOT CURRENTLY PASS! Any test results where Datafusion returns a result that does not match sqlite nor match Postgresql was left as-is.

What changes are included in this PR?

This PR includes a number of changes many of which are part of the test files in the datafusion-testing repo (5,711,125 select statements of which 78,437 fail outright in Datafusion). The list below includes both the changes in this direct PR as well as the process to generate the files in datafusion-testing/data/sqlite/

  • All the .test files in the sqlite test suite are includes except the contents of the evidence and the index/delete folders
  • All .test files have had mysql and mssql specific tests removed. All other references to mysql and mssql have been removed.
  • All .test files were renamed to .slt
  • All files had control resultmode valuewise added to the beginning to allow the sqllogictest runner to properly be able to compare the results from Datafusion (and Postgresql) to the results in the .slt file
  • All queries have been run through both Datafusion and Postgres and any queries that failed with an error have had comments added explaining the failure and a skipif Datafusion and/or skipif postgres. For example:
# Datafusion - DataFusion error: SQL error: RecursionLimitExceeded
skipif Datafusion
  • Datatypes have been updated to reflect data types from Datafusion/Postgresql as the sqlite datatypes are very limited. Comments reflecting the change have been added. For example:
# Datafusion - Types were automatically converted from:
# Datafusion - [Expected] [T]
# Datafusion - [Actual  ] [I]
  • Results have been updated if the Datafusion results differ from the sqlite results AND the Datafusion results are the same as what the results from Postgresql are. There are queries where the results differs especially around floating point (Real results in slt terms). floating point results were deemed equivalent between Datafusion and Postgresql if the result was the same to 4 decimal places. Comments reflecting the change have been added. For example:
# Datafusion - Data was automatically updated based on comparison db results
# Datafusion - Previous results:
# Datafusion - 54
# Datafusion - 9
  • The sqllogictest and runners have been updated to include progress information
  • A datafusion-testing git submodule has been added. You may need to run git submodule update --init --remote --recursive to get it added to an existing checkout of datafusion.
  • Added the ability to start a postgres docker container automatically if no PG_URI is set.
  • Readme updates

Are these changes tested?

Indeed, yes. To run the tests locally checkout this branch, update the git submodules then run INCLUDE_SQLITE=true cargo test --profile release-nonlto --test sqllogictests. Be aware that the tests can take quite a long time to run, especially if you do not run with release or release-nonlto profiles.

Are there any user-facing changes?

No.

Omega359 and others added 11 commits November 20, 2024 18:00
* Update sqllogictest requirement from 0.24.0 to 0.25.0

Updates the requirements on [sqllogictest](https://github.com/risinglightdb/sqllogictest-rs) to permit the latest version.
- [Release notes](https://github.com/risinglightdb/sqllogictest-rs/releases)
- [Changelog](https://github.com/risinglightdb/sqllogictest-rs/blob/main/CHANGELOG.md)
- [Commits](risinglightdb/sqllogictest-rs@v0.24.0...v0.25.0)

---
updated-dependencies:
- dependency-name: sqllogictest
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>

* Remove labels

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: jonahgao <[email protected]>
@github-actions github-actions bot added sqllogictest SQL Logic Tests (.slt) functions labels Dec 28, 2024
@Omega359 Omega359 closed this Dec 28, 2024
@Omega359 Omega359 deleted the sqllogictest_add_sqlite2 branch December 28, 2024 22:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
functions sqllogictest SQL Logic Tests (.slt)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Complete / integrate sqlite sqllogictest test scripts integrattion
3 participants