feat: add timeoutSeconds to ManagedChart

[FrankYang0529]

Problem:

Solution:

Related Issue:
harvester/harvester#7375

Test plan:

[tserong]

How does adding a timeout help to fix the related issue? I'm sure I'm personally missing some knowledge here about managed charts, so please feel free to tell me to RTFM ;-)

[albinsun]

A Try w/ Wordaround, upgrade can be triggered and complete.

1. Cluster Setup

   3 nodes witness, hits this issue as expected.

   • chart
     
   • managedChart
     

2. Apply workaround (Edit by k9s)

   Add timeoutSeconds: 600 in spec like this PR

   

   • chart
     
   • managedChart
     

3. Create storage class sc2 and set as default
   

4. Start an airgapped upgrade to v1.4.1-rc1
   

   

[FrankYang0529]

How does adding a timeout help to fix the related issue?

The error message is from fleet bundle and fleet is used to run helm command. The default timeout in helm command is 5 minutes, so increasing timeout maybe helpful. I think that this approach may not solve root cause, but it may be a good workaround to reduce the probability of problems occurring.

[innobead]

LGTM

[w13915984028]

I can confirm this PR and setting is essential.

With debug on fleet-agent, if managedchart does not set a timeout, it uses 0, when there are any hooks, it will fail in below function.

// execHook executes all of the hooks for the given hook event.
func (cfg *Configuration) execHook(rl *release.Release, hook release.HookEvent, timeout time.Duration) error {

We had below PRs on upgrade path, now with #929, it will solve finally.

harvester/harvester#6608
harvester/harvester#7386

ErrApplied(1) [Cluster fleet-local/local: post-upgrade hooks failed, debugging, timeout:0s : context deadline exceeded]

-> with a force param 5

NotReady(1) [Cluster fleet-local/local: not installed: post-upgrade hooks failed, debugging, timeout:5ns : warning: Hook longhorn-post-upgrade Job harvester/charts/longhorn/templates/postupgrade-job.yaml failed with timeout 5ns: 1 error occurred:...

harv21:/home/rancher # kk get bundle -A
NAMESPACE     NAME                                          BUNDLEDEPLOYMENTS-READY   STATUS
fleet-local   fleet-agent-local                             1/1                       
fleet-local   local-managed-system-agent                    1/1                       
fleet-local   mcc-harvester                                 0/1                       ErrApplied(1) [Cluster fleet-local/local: post-upgrade hooks failed, debugging, timeout:0s : context deadline exceeded]
fleet-local   mcc-harvester-crd                             1/1                       
fleet-local   mcc-local-managed-system-upgrade-controller   1/1                       
fleet-local   mcc-rancher-logging-crd                       1/1                       
fleet-local   mcc-rancher-monitoring-crd                    1/1   



harv21:/home/rancher # kk get bundle -A
NAMESPACE     NAME                                          BUNDLEDEPLOYMENTS-READY   STATUS
fleet-local   fleet-agent-local                             1/1                       
fleet-local   local-managed-system-agent                    1/1                       
fleet-local   mcc-harvester                                 0/1                       NotReady(1) [Cluster fleet-local/local: not installed: post-upgrade hooks failed, debugging, timeout:5ns : warning: Hook longhorn-post-upgrade Job harvester/charts/longhorn/templates/postupgrade-job.yaml failed with timeout 5ns: 1 error occurred:...
fleet-local   mcc-harvester-crd                             1/1                       
fleet-local   mcc-local-managed-system-upgrade-controller   1/1                       
fleet-local   mcc-rancher-logging-crd                       1/1                       
fleet-local   mcc-rancher-monitoring-crd                    1/1