hostmetricsreceiver semantic conventions transition

[braydonk]

This issue will be used to track the work for making hostmetricsreceiver capable of transitioning to Semantic Conventions compliance.

Task list:

• Add feature to mdatagen to allow it to generate packages with names other than metadata (Allow mdatagen to generate packages with names other than metadata opentelemetry-collector#11231)
• Introduce a semantic conventions featuregate in the hostmetrics receiver
• Add a semconv spec to each receiver and allow it to produce either existing or semconv metrics
  • processscraper
  • cpuscraper
  • diskscraper
  • filesystemscraper
  • loadscraper
  • memoryscraper
  • networkscraper
  • pagingscraper
• Unify the processscraper and processesscraper since the processes.* metrics are being moved to process.* ([receiver/hostmetrics] Deprecate processesscraper #30894) (Blocked on mdatagen: add wildcard name matching for configs opentelemetry-collector#10065)

[github-actions]

Pinging code owners for receiver/hostmetrics: @dmitryax @braydonk. See Adding Labels via Comments if you do not have permissions to add labels yourself.

[crobert-1]

Removing needs triage as this makes sense to me, and was filed by a code owner.

[rogercoll]

Introduce a semantic conventions featuregate in the hostmetrics receiver

Do we plan to support the two metrics schemas during the transition? Or new naming features will be only applied to the semconv schema?

[braydonk]

Do we plan to support the two metrics schemas during the transition? Or new naming features will be only applied to the semconv schema?

I think the plan at the moment is to support two schemas, but leaving it up to implementer whether a new metric or attribute change should be added to both schemas at the same time or only the semconv schema. Given that we are getting closer and closer to stabilization before I'm able to initiate this plan, I'm hoping we won't have to support two schemas for long. I think the two schema support will be necessary for a short while though, given that some of the metrics/attributes are a major lift that users need time to prepare for.

[ChrsMark]

Other areas like messaging SemConv define a specific migration plan for instrumentations, ie: https://github.com/open-telemetry/semantic-conventions/pull/1198/files#diff-bbe463329a83f81ca6c90d49d1998d69929432ba28bb6eed991a03cd110f74e6R12-R27

I'm preparing an early stage plan for k8s SemConv as well: open-telemetry/semantic-conventions#1597.

What I'm missing though is what would be the plan to follow in the Collector.

1. Should that be an environment variable or a feature gate?
2. Should we support changing to only new SemConv but also new+old(dup)? See add k8s semconv migration non-normative doc semantic-conventions#1597.
3. What versioning plan are we going to follow since the Collector has only one Major version so far and it's basically released on Minor version basis?

I would like to collect feedback here since answering the above questions will help us making our migration plans more concrete.

@open-telemetry/semconv-system-approvers @open-telemetry/collector-contrib-approvers could you please share your thoughts?

[mx-psi]

Should that be an environment variable or a feature gate?

A feature gate is the idiomatic approach in the Collector, I think this is what our users would expect and what makes the most sense here.

Should we support changing to only new SemConv but also new+old(dup)

Do we have data on what users preferred for this on the HTTP migration?

What versioning plan are we going to follow since the Collector has only one Major version so far and it's basically released on Minor version basis?

Not sure I understand this question: what is the concern here? IMO doing this migration (at least until we have the beta stage) should be a requirement for the hosmetrics receiver or the k8s components to be marked as 1.0. Since we are at 0.x, breaking users is okay, but we want to do it slowly and allowing them to adopt this change at their own pace through the feature gate approach.

[ChrsMark]

Not sure I understand this question: what is the concern here? IMO doing this migration (at least until we have the beta stage) should be a requirement for the hosmetrics receiver or the k8s components to be marked as 1.0. Since we are at 0.x, breaking users is okay, but we want to do it slowly and allowing them to adopt this change at their own pace through the feature gate approach.

The question is mostly about deciding for how long the transition phase will be in place and how this will be expressed in our versioning strategy. Using the v1.0 of these components makes sense and a plan like this would also apply here 👍🏼 .

That would mean that we can introduce the feature gate in v0.1xx.0 we keep it for at least 6 minor versions and once we feel comfortable we switch to v1.0.0 keeping only the new behavior. Would that make sense?

[mx-psi]

That would mean that we can introduce the feature gate in v0.1xx.0 we keep it for at least 6 minor versions and once we feel comfortable we switch to v1.0.0 keeping only the new behavior. Would that make sense?

I would block 1.0 on semconvs being stable. I would not block 1.0 on the feature gate being stable (I feel like what matters is default behavior). There will be other requirements to prepare this component for 1.0, but provided we have all of those then yes, that makes sense

[braydonk]

Should we support changing to only new SemConv but also new+old(dup)? See open-telemetry/semantic-conventions#1597.

On first glance I struggle to see the usecase for hostmetrics to produce both schemas at the same time, but we can certainly find a way to build that in if users would be able to make use of it.

[mx-psi]

Discussed on 2024-11-21 meeting. We probably want two feature gates: one for enabling the new schema, and one for disabling the old schema. Then:

• On alpha old is enabled, new is disabled
• On beta/stable old is disabled, new is enabled
• It is an error to disable both