Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix bug which should delete node from store when node is not found #980

Merged
merged 1 commit into from
Jan 2, 2025

Conversation

zhengxiexie
Copy link
Contributor

8153:2024-12-24 13:20:56.232	INFO	node/node_controller.go:40	reconciling node	{"node": {"name":"192-163-127-8"}}
8154:2024-12-24 13:20:56.234	INFO	node/node_controller.go:46	node not found	{"req": {"name":"192-163-127-8"}}
8596:2024-12-24 13:48:43.606	INFO	node/node_controller.go:40	reconciling node	{"node": {"name":"192-163-127-8"}}
8597:2024-12-24 13:48:43.607	INFO	node/node.go:78	node alreay cached	{"node.Fqdn": "192-163-127-8", "node.UniqueId": "78dac851-95a4-4b78-84f0-acd69c7244e1"}
9179:2024-12-24 13:58:41.968	ERROR	pod/pod_controller.go:86	failed to get node ID for pod	{"pod.Name": {"name":"test-0","namespace":"ns-2"}, "pod.UID": "a84b008a-230c-4d5a-a088-0e1c5d230962", "node": "192-163-127-8", "error": "multiple node IDs found for node 192-163-127-8: [78dac851-95a4-4b78-84f0-acd69c7244e1 d90046e7-4992-4739-822e-663c95a9dfaf]"}

From the log, we can see when node is deleted, there is node not found error, and there are still two node ids in store.
The phase node.ObjectMeta.DeletionTimestamp being not zero is intermittent, and it is more possible the node is not found, we should delete the node from store.

@zhengxiexie zhengxiexie force-pushed the topic/zhengxie/main/fix_node branch 5 times, most recently from 5f9e3a0 to 5c774da Compare December 30, 2024 06:26
@codecov-commenter
Copy link

codecov-commenter commented Dec 30, 2024

Codecov Report

Attention: Patch coverage is 85.71429% with 1 line in your changes missing coverage. Please review.

Project coverage is 73.37%. Comparing base (c729a64) to head (5a770d7).

Files with missing lines Patch % Lines
pkg/controllers/node/node_controller.go 85.71% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #980      +/-   ##
==========================================
+ Coverage   73.29%   73.37%   +0.07%     
==========================================
  Files         118      118              
  Lines       16397    16401       +4     
==========================================
+ Hits        12019    12035      +16     
+ Misses       3591     3578      -13     
- Partials      787      788       +1     
Flag Coverage Δ
unit-tests 73.37% <85.71%> (+0.07%) ⬆️
Files with missing lines Coverage Δ
pkg/controllers/node/node_controller.go 51.89% <85.71%> (+10.56%) ⬆️

... and 1 file with indirect coverage changes

@zhengxiexie zhengxiexie force-pushed the topic/zhengxie/main/fix_node branch 5 times, most recently from 28cc7ca to 2ece25f Compare December 31, 2024 02:56
@zhengxiexie
Copy link
Contributor Author

/e2e

1 similar comment
@zhengxiexie
Copy link
Contributor Author

/e2e

pkg/controllers/node/node_controller.go Outdated Show resolved Hide resolved
pkg/controllers/node/node_controller.go Outdated Show resolved Hide resolved
@zhengxiexie zhengxiexie force-pushed the topic/zhengxie/main/fix_node branch from 2ece25f to defd8df Compare December 31, 2024 07:01
…und error

When node is deleted, the phase node.ObjectMeta.DeletionTimestamp being not zero is intermittent,
and it is more possible the node is not found, we should delete the node from store.
@zhengxiexie zhengxiexie force-pushed the topic/zhengxie/main/fix_node branch from defd8df to 5a770d7 Compare January 2, 2025 03:53
Copy link
Contributor

@yanjunz97 yanjunz97 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@lxiaopei lxiaopei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zhengxiexie zhengxiexie merged commit 1c4c7cd into main Jan 2, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants