Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[VL] Gluten-it: Improve test report table format for parameterized test #6052

Merged
merged 4 commits into from
Jun 12, 2024

Conversation

zhztheplayer
Copy link
Member

@zhztheplayer zhztheplayer commented Jun 12, 2024

Just changes the table format to a compacter view for gluten-it option parameterized for easier analysis on the test result.

Before:

| Query ID | Succeeded | SHUFFLE_PARTITIONS | GLUTEN | Row Count | Planning Time (Millis) | Query Time (Millis) |
|----------|-----------|--------------------|--------|-----------|------------------------|---------------------|
|        q1|       true|                  30|     OFF|          4|                     427|                 7945|
|        q2|       true|                  30|     OFF|        100|                     356|                 1265|
|        q1|       true|                  30|      ON|          4|                      42|                 1504|
|        q2|       true|                  30|      ON|        100|                     134|                  812|
|        q1|       true|                  20|     OFF|          4|                      25|                 5406|
|        q2|       true|                  20|     OFF|        100|                      90|                  751|
|        q1|       true|                  20|      ON|          4|                      24|                  756|
|        q2|       true|                  20|      ON|        100|                      87|                  615|

After:

Configurations:
1. SHUFFLE_PARTITIONS -> 30, GLUTEN -> OFF
2. SHUFFLE_PARTITIONS -> 30, GLUTEN -> ON
3. SHUFFLE_PARTITIONS -> 20, GLUTEN -> OFF
4. SHUFFLE_PARTITIONS -> 20, GLUTEN -> ON

| Query ID |         Succeeded         |       Row Count       |  Planning Time (Millis)   |    Query Time (Millis)     |
|          |  1   |  2   |  3   |  4   |  1  |  2  |  3  |  4  |  1   |  2   |  3   |  4   |   1   |  2   |  3   |  4   |
|----------|------|------|------|------|-----|-----|-----|-----|------|------|------|------|-------|------|------|------|
|        q1|  true|  true|  true|  true|    4|    4|    4|    4|   427|    48|    34|    32|   9342|  1505|  5975|   719|
|        q2|  true|  true|  true|  true|  100|  100|  100|  100|   312|   125|   108|   102|   1116|   850|  1005|   609|
|       all|  true|  true|  true|  true|  104|  104|  104|  104|   739|   173|   142|   134|  10458|  2355|  6980|  1328|

Copy link

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/apache/incubator-gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:

Copy link

Run Gluten Clickhouse CI

@zhztheplayer zhztheplayer marked this pull request as ready for review June 12, 2024 03:35
Copy link

Run Gluten Clickhouse CI

@zhztheplayer zhztheplayer changed the title [VL] Gluten-it: Improve test report representation for parameterized test [VL] Gluten-it: Improve test report table format for parameterized test Jun 12, 2024
Copy link

Run Gluten Clickhouse CI

1 similar comment
Copy link

Run Gluten Clickhouse CI

Copy link
Contributor

@zhouyuan zhouyuan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@zhztheplayer zhztheplayer merged commit 1c505df into apache:main Jun 12, 2024
39 of 40 checks passed
@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_6052_time.csv log/native_master_06_08_2024_31c384f0d1_time.csv difference percentage
q1 37.34 32.86 -4.485 87.99%
q2 23.94 23.75 -0.194 99.19%
q3 38.50 36.30 -2.206 94.27%
q4 32.20 32.11 -0.082 99.75%
q5 71.25 68.89 -2.356 96.69%
q6 6.12 7.29 1.176 119.22%
q7 80.42 79.65 -0.769 99.04%
q8 85.74 82.98 -2.763 96.78%
q9 119.39 122.29 2.906 102.43%
q10 45.26 46.66 1.394 103.08%
q11 20.20 20.64 0.437 102.16%
q12 26.46 27.87 1.411 105.33%
q13 38.97 37.78 -1.188 96.95%
q14 21.71 19.38 -2.331 89.27%
q15 32.73 30.47 -2.260 93.09%
q16 14.02 14.26 0.240 101.71%
q17 105.10 102.90 -2.198 97.91%
q18 147.47 144.26 -3.206 97.83%
q19 13.53 13.59 0.065 100.48%
q20 28.15 28.18 0.020 100.07%
q21 256.80 257.39 0.592 100.23%
q22 12.14 12.17 0.034 100.28%
total 1257.43 1241.66 -15.763 98.75%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants