[Process Suggestion] Add documentation for each usability issue created

[crobert-1]

Component(s)

No response

Describe the issue you're reporting

Problem
Configuring the collector can be confusing, and understanding how to use different options can often be unclear. From my experience as a triager in the contrib repository, we see lots of issues created by users who don't understand how to configure a component to do what they want. This is a challenge for practically all of the collector's components. I've included some common themes and examples I've found below, but there are many more examples if we wanted to compile a more thorough list.

Common themes (Issue numbers in parenthesis):

• Can X component do Y? (Route the logs to loki exporter or Splunk exporter based on log attributes. #26720)
• Unaware of required config options or environment variables. (Jaeger UI SPM spanmetrics not working in 0.85.0 #26712)
• Filters aren't matching data the way the user expected. (Tranform processor- expected string but got pcommon.Map #26689, Metric Datapoint attribute isn't getting filtered out via otelcollector #24277)
• General config issues. (Loki exporter is not setting X-scoppe-OrgID(tenant id) based on log attribute (loki.tenant) #26557, K8sevents receiver panics  #26528, Unable to scrape metrics through kubeletstats receiver #26481, Region information is AKS and EKS #26688, Document/refine all k8sattributes config fields and their functionality #26110, health_check extension: liveliness is failing, not sure. #24655)

The easiest answer to these kinds of questions and issues is to point to the component's README with an explicit configuration example relevant to their use case, or statement of the component's use cases.

If there's no obvious answer in documentation the alternatives are usually one of the following:

1. Try to set up a demo manually in your local environment to reproduce the issue and point the user in the right direction.
2. Ping code owners.
3. Leave issues alone to go stale.

None of these are good long term options. It's hard to try to help someone on an issue without context, especially trying to get a demo up and running. Code owners are responsible for their components, but understandably very busy, so it's hard to set strict requirements about how quickly they should respond. Issues going stale is very clearly poor user experience, and not sustainable for repo hygiene.

Proposed Solution:
From my perspective, the best option here is to add some piece of documentation that directly addresses any usability issue. Whether it's adding a sentence about "X is supported by Y", or adding a configuration example for what someone is trying to do, this is extremely helpful for everyone. This will help users get demos up and running more quickly, as well as enabling triagers (and any interested party) to have more examples and references to turn to with this kind of problem.

My proposal is to add a label (usability) to any issue that falls under this umbrella. For any issue with the usability label, a documentation PR would be required to close/resolve the issue.

Acknowledging concerns
I fully understand that this is not going to solve configuration challenges. Having lots of examples can be intimidating to users and also hard to maintain as components evolve. That being said, I think we're much further on the side of too little information than too much. Also, to the point of documentation being hard to maintain, the alternatives are don't document, don't develop. I also believe if we are making more consistent small updates we'll be more likely to keep documentation up to date.

New processes can be annoying and cumbersome, but this is relatively light weight. The work required is done in triaging an issue and responding to the issue author. The process is simply opening a PR that moves the result of the discussion into a component's README.

[crobert-1]

Discussed in collector sig on 10/11/23.

From collector sig (directly related to this issue):

1. Re-use the documentation label instead of adding a new usability label. The documentation label doesn't have a definition in the contributing readme, so we should add a definition there.
2. Don't make a documentation PR required to close an issue with its label. There are times where something is too specific to be useful in official documentation, and would likely just make readmes too bloated.
3. Direct users to official collector documentation for more general usability "how tos". We should take advantage of these landing pages more often in responses.

Other takeaways/goals from collector sig:

• Add more automation to GitHub discussions to be able to add labels and ping code owners, as discussions are probably the correct place to have these questions and answers. Once automation is added, add a new issue template in the repository where the question is converted to a discussion rather than an issue. As @djaglowski suggested: New Issue => Question => Automatically convert to Discussion. Once this is in place we'll work on proactively converting these types of issues to discussions if they're logged as bugs/enhancements.
• Stack overflow is probably a better avenue for questions and answers than the CNCF slack. The main concern with slack is that it isn't searchable by the general public.