Skip to content

Commit

Permalink
fix!: manager track prev executed directives, prevent duplicate runs
Browse files Browse the repository at this point in the history
`directives` are supposed to only be executed _once_ IFF the file has
been executed within the last 24h. The existing code would, during the
course of the normal scheduled runs, attempt to reload and run all valid
directives found in the inventory. Because we are not tracking
directives that have already been executed, this means that if a
directive is added to an inventory, then during the course of it's
normal scheduled runs, _each run_ of the reload-and-run-all code would
attempt to execute the directive during that 24h validity period. This
means that a directive would not be executed once as intended, but
rather would be executed `24h / $inventory.reload_interval` number of
times.

This adds a cache map to the manager to keep track of which directives
have already been executed.

NOTE: the cache is not persistent beyond the lifetime of the process in
which mango is running. This means that if a new directive is added and
executed by mango, and then mango itself subsequently gets restarted
during the 24h validity period of that directive, the cache will be
reset and mango will re-execute the directive. Given that mango is
intended to be run as a long lived daemon, I find this reasonable.
  • Loading branch information
tjhop committed May 8, 2024
1 parent 189759d commit 81d9166
Show file tree
Hide file tree
Showing 2 changed files with 23 additions and 9 deletions.
15 changes: 14 additions & 1 deletion internal/manager/directive.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,19 @@ func (mgr *Manager) ReloadDirectives(ctx context.Context) {
dirScripts[i] = Directive{d: ds}
}

mgr.directives = dirScripts
// check newly loaded directives against the already executed
// directives. if a directive has already been executed, we do not want
// to add it to the directives list, as this is the feed that
// `RunDirectives()` works off of; rather we only want to add it to the
// manager's list of directives if it has _not_ been executed
var dirScriptsToExecute []Directive
for _, d := range dirScripts {
if _, found := mgr.executedDirectives[d.String()]; !found {
dirScriptsToExecute = append(dirScriptsToExecute, d)
}
}

mgr.directives = dirScriptsToExecute
}

// RunDirective is responsible for actually executing a directive, using the `shell`
Expand Down Expand Up @@ -58,6 +70,7 @@ func (mgr *Manager) RunDirective(ctx context.Context, ds Directive) error {
}

err = shell.Run(ctx, runID, ds.String(), renderedScript, nil)
mgr.executedDirectives[ds.String()] = struct{}{} // mark directive as executed

// update metrics regardless of error, so do them before handling error
applyEnd := time.Since(applyStart)
Expand Down
17 changes: 9 additions & 8 deletions internal/manager/manager.go
Original file line number Diff line number Diff line change
Expand Up @@ -33,14 +33,15 @@ var (

// Manager contains fields related to track and execute runnable modules and statistics.
type Manager struct {
id string
inv inventory.Store // TODO: move this interface to be defined consumer-side in manager vs in inventory
modules graph.Graph[string, Module]
directives []Directive
hostVariables VariableSlice
runLock sync.Mutex
funcMap template.FuncMap
tmplData templateData
id string
inv inventory.Store // TODO: move this interface to be defined consumer-side in manager vs in inventory
modules graph.Graph[string, Module]
directives []Directive
executedDirectives map[string]struct{} // stores the ID of the directive as key
hostVariables VariableSlice
runLock sync.Mutex
funcMap template.FuncMap
tmplData templateData
}

func (mgr *Manager) String() string { return mgr.id }
Expand Down

0 comments on commit 81d9166

Please sign in to comment.