Skip to content

Commit

Permalink
Fix spurious use of indexResolver preventing idling (#589)
Browse files Browse the repository at this point in the history
We have noticed that Quesma still has problems with properly idling (not
making queries to ClickHouse Cloud to allow it to idle).

The current idling logic relies on checking if table discovery was
recently used. It turns out that idle Kibana very often does inserts to
an internal `.kibana` index and to `/_monitoring/bulk` endpoint.

Even though those requests are routed to Elastic, we still checked if
`.kibana` or `_monitoring` are names of enabled indices using
indexResolver - this use bumped the LastAccessTime, preventing idling.

Fix the issue by avoiding a use of indexResolver if it's an endpoint
(`_endpoint`) or if it's an internal index (starting with `.`).
  • Loading branch information
avelanarius authored Aug 1, 2024
1 parent 248c280 commit aa2bb5b
Showing 1 changed file with 12 additions and 0 deletions.
12 changes: 12 additions & 0 deletions quesma/feature/not_supported.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,9 @@ package feature
import (
"context"
"fmt"
"quesma/elasticsearch"
"regexp"
"strings"
)

// https://www.elastic.co/guide/en/elasticsearch/reference/current/search-template.html
Expand Down Expand Up @@ -80,6 +82,16 @@ func checkIfOurIndex(ctx context.Context, method string, path string, indexResol
if len(match) > 1 {
indexNamePart := match[1]
var matched bool

if strings.HasPrefix(indexNamePart, "_") {
// This is not actually an index, this is some endpoint (for example /_monitoring/bulk)
return false
}
// Optimization: avoid using indexResolver for internal indexes
if elasticsearch.IsInternalIndex(indexNamePart) {
return false
}

indexes, err := indexResolver(ctx, indexNamePart)
if err != nil {
logMessage("Error resolving index: %v", err)
Expand Down

0 comments on commit aa2bb5b

Please sign in to comment.