Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Shorten the error message in orphaned-objects monrun check #265

Merged
merged 12 commits into from
Dec 9, 2024
Merged

Shorten the error message in orphaned-objects monrun check #265

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
4b72bed
Added regular pattern for handle errors in orphaned-objects monrun check
Dec 5, 2024
6617254
Update tests
Dec 5, 2024
437b88c
lint
Dec 5, 2024
4040ce6
Refactoring and add spare_pattern
Dec 5, 2024
79540d2
Fix test
Dec 5, 2024
8661cc8
Fix
Dec 5, 2024
5aac2eb
Fix tests
Dec 5, 2024
b8f23b8
Create consts
Dec 5, 2024
164054b
Delete SPARE_ERROR_MESSAGE_PATTERN
Dec 5, 2024
4b1fd6c
Fixes
Dec 5, 2024
6229422
Refactoring, update ch_tools/common/config.py
Dec 6, 2024
af66fd8
Update Check Geobase
Dec 6, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions ch_tools/common/config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,10 @@
"pattern": "\n",
"replacement": " ",
},
{
"pattern": r"(Code:\s\d+\.\sDB::Exception:\s).*(\s\([A-Z_]*\)\s\(version\s.*\s\(official build\)\)\s).*",
"replacement": r"\1...\2...",
},
],
},
},
Expand Down
13 changes: 12 additions & 1 deletion tests/features/monrun.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ Feature: ch-monitoring tool
"""
Then we get response contains
"""
1;Unknown error: Code: 156. DB::Exception: Embedded dictionaries were not loaded.
1;Unknown error: Code: 156. DB::Exception: ... (DICTIONARIES_WAS_NOT_LOADED)
"""
When we execute command on clickhouse01
"""
Expand Down Expand Up @@ -513,3 +513,14 @@ Feature: ch-monitoring tool
"""
2;ERROR
"""

Scenario: Check clickhouse orphaned objects with long error_msg
When we create file /tmp/object_storage_cleanup_state.json with data "{ \"orphaned_objects_size\": 0, \"error_msg\": \"Code: 27. DB::Exception: Cannot parse: input:: expected '\\\\t' before: 'klg%2D1acvr8hmq0n16qm5%2Edb%2Eyandex%2Enet\\\\ndefault\\\\n6736d483-516a-4892-87d4-084d5c1f6d3c\\\\n': While executing SystemRemoteDataPaths. (CANNOT_PARSE_INPUT_ASSERTION_FAILED) (version 24.8.5.115 (official build)) Query: SELECT obj_path, obj_size FROM _system.listing_objects_from_object_storage AS object_storage LEFT ANTI JOIN remoteSecure('klg-1acvr8hmq0n16qm5.db.yandex.net', system.remote_data_paths) AS object_table ON object_table.remote_path = object_storage.obj_path AND object_table.disk_name = 'object_storage' SETTINGS traverse_shadow_remote_data_paths=1 FORMAT TabSeparated (klg-1acvr8hmq0n16qm5.mdb.yandex.net)\" }"
And we execute command on clickhouse01
"""
ch-monitoring orphaned-objects --state-local
"""
Then we get response
"""
2;Code: 27. DB::Exception: ... (CANNOT_PARSE_INPUT_ASSERTION_FAILED) (version 24.8.5.115 (official build)) ...
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's use we get response matches for checking response without specific versions

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The purpose of this test is to show the final error message, we are parsing an error that already contains "(version 24.8.5.115 (official build))", that is, this result of this test does not depend on the version of clickhouse.

Maybe I didn’t understand why to change this test.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I got confused with the Scenario: Check Geobase test. But it is already fixed

"""
Loading