Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Define value for all constant_keyword fields in apmpackage #12219

Merged
merged 3 commits into from
Jan 24, 2024

Conversation

carsonip
Copy link
Member

@carsonip carsonip commented Dec 13, 2023

Motivation/summary

Define value for all constant_keyword fields in apmpackage to workaround Fleet upgrade bug. Under certain conditions, it is possible that apmpackage upgrade creates a lot of indices if constant_keyword does not have a value.

Targeting 8.12 branch because apmpackage has been moved to https://github.com/elastic/integrations/tree/main/packages/apm/data_stream on main.

Checklist

For functional changes, consider:

  • Is it observable through the addition of either logging or metrics?
  • Is its use being published in telemetry to enable product improvement?
  • Have system tests been added to avoid regression?

How to test these changes

  • Ingest APM data, upgrade package to a version that contains the fix, and verify that DS rollover is successful and mapping contains a value for constant keyword fields.

Related issues

Closes #12159

Copy link
Contributor

mergify bot commented Dec 13, 2023

This pull request does not have a backport label. Could you fix it @carsonip? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-7.17 is the label to automatically backport to the 7.17 branch.
  • backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit.

NOTE: backport-skip has been added to this pull request.

@mergify mergify bot added the backport-skip Skip notification from the automated backport with mergify label Dec 13, 2023
@carsonip carsonip added backport-8.12 Automated backport with mergify and removed backport-skip Skip notification from the automated backport with mergify labels Dec 13, 2023
@carsonip carsonip force-pushed the constant-keyword-value branch from 7e88ed6 to 5dfe45a Compare December 13, 2023 16:56
Copy link
Contributor

mergify bot commented Jan 12, 2024

This pull request is now in conflicts. Could you fix it @carsonip? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b constant-keyword-value upstream/constant-keyword-value
git merge upstream/main
git push upstream constant-keyword-value

@carsonip
Copy link
Member Author

@simitt now that apmpackage no longer lives in apm-server repo, should I split this PR into 2 parts, 1 targeting apm-server 8.12 branch, another changing https://github.com/elastic/integrations/tree/main/packages/apm/data_stream ?

@simitt
Copy link
Contributor

simitt commented Jan 18, 2024

Yeah I fear we need to put this into both places now unfortunately.

@carsonip carsonip force-pushed the constant-keyword-value branch from 6747d45 to b204358 Compare January 18, 2024 17:29
@carsonip carsonip changed the base branch from main to 8.12 January 18, 2024 17:29
@carsonip carsonip added backport-skip Skip notification from the automated backport with mergify and removed backport-8.12 Automated backport with mergify labels Jan 18, 2024
Copy link
Member Author

@carsonip carsonip left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once this is merged I'll port the change to https://github.com/elastic/integrations/tree/main/packages/apm

@carsonip carsonip marked this pull request as ready for review January 18, 2024 17:57
@carsonip carsonip requested a review from a team as a code owner January 18, 2024 17:57
@kruskall kruskall closed this Jan 23, 2024
@kruskall kruskall reopened this Jan 23, 2024
@bmorelli25
Copy link
Member

run docs-build

@carsonip carsonip merged commit 5d0de3a into elastic:8.12 Jan 24, 2024
12 checks passed
@carsonip
Copy link
Member Author

carsonip commented Feb 6, 2024

This change did not go into 8.12.1. Now the plan is to land this fix in 8.12.2.

carsonip added a commit to elastic/integrations that referenced this pull request Feb 7, 2024
carsonip added a commit to carsonip/apm-server that referenced this pull request Feb 21, 2024
The fix did not go into 8.12.1 due to automation issues. See elastic#12219
carsonip added a commit that referenced this pull request Feb 21, 2024
The fix did not go into 8.12.1 due to automation issues. See #12219
mergify bot pushed a commit that referenced this pull request Feb 21, 2024
The fix did not go into 8.12.1 due to automation issues. See #12219

(cherry picked from commit 405663f)
mergify bot pushed a commit that referenced this pull request Feb 21, 2024
The fix did not go into 8.12.1 due to automation issues. See #12219

(cherry picked from commit 405663f)
mergify bot added a commit that referenced this pull request Feb 21, 2024
#12697)

The fix did not go into 8.12.1 due to automation issues. See #12219

(cherry picked from commit 405663f)

Co-authored-by: Carson Ip <[email protected]>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
mergify bot added a commit that referenced this pull request Feb 21, 2024
#12698)

The fix did not go into 8.12.1 due to automation issues. See #12219

(cherry picked from commit 405663f)

Co-authored-by: Carson Ip <[email protected]>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
@carsonip
Copy link
Member Author

testing notes

✔️ test-plan-ok

In 8.12.2, in component template metrics-apm.transaction.1m@package:

    "metricset": {
      "properties": {
        "name": {
          "type": "constant_keyword",
          "value": "transaction"
        },
        "interval": {
          "type": "constant_keyword",
          "value": "1m"
        }
      }
    },

the value of constant keyword is now set.

@carsonip
Copy link
Member Author

testing notes

Upgraded from a 8.12.1 deployment with existing data in metrics-apm.service_destination.1m-default etc to 8.12.2. Upgrade went well and data streams rolled over successfully.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-skip Skip notification from the automated backport with mergify test-plan test-plan-ok v8.12.2
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Define value for all constant-keyword fields in the apm package
4 participants