feat(controller): Log expression context for conditional retries. #7181
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Josh Carp <[email protected]>
|
By the way, I'm still confused about why preempted pods have no outputs or exit code. Here's an example from my testing: retry-test-preempt-01-3853357172:
displayName: retry-test-preempt-01(1)
finishedAt: null
hostNodeName: ...
id: retry-test-preempt-01-3853357172
message: Node is shutting, evicting pods
name: retry-test-preempt-01(1)
phase: Failed
progress: 1/1
resourcesDuration:
cpu: 0
memory: 0
startedAt: "2021-11-07T07:13:40Z"
templateName: retry-crash
templateScope: local/retry-test-preempt-01
type: Pod |
|
Can you please add the output of this change? |
sarabala1979
approved these changes
Dec 10, 2021
|
I attempted to generate log output for this issue, but it appears that using Until that is resolved, I do not know of a good way to show this output as requested. |
alexec
reviewed
Jan 19, 2022
| @@ -791,6 +791,7 @@ func (woc *wfOperationCtx) processNodeRetries(node *wfv1.NodeStatus, retryStrate | |||
| if retryStrategy.Expression != "" && len(node.Children) > 0 { | |||
| localScope := buildRetryStrategyLocalScope(node, woc.wf.Status.Nodes) | |||
| scope := env.GetFuncMap(localScope) | |||
| woc.log.WithField("node", node.Name).Infof("evaluating retry conditional with scope %s", localScope) | |||
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
|
where @jmcarp |
agilgur5
added
area/templating
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I was debugging an issue with conditional retries, and it took a while to figure out that a certain failure mode was associated with an exit code of
-1. Logging the expression context would help with this kind of task in the future. Let me know if there's a better place to put this than the operator logs!cc @jli
Signed-off-by: Josh Carp [email protected]
Don't bother creating a PR until you've done this:
make pre-commit -Bto fix codegen, lint, and commit message problems.