Use ERF error codes for troubleshooting

[ekohl]

We used to maintain https://projects.theforeman.org/projects/foreman/wiki/ErrorCodes to make troubleshooting easier. Foreman automatically generates these based on the class and message (see https://github.com/theforeman/foreman/blob/b4bd8e2f248e6bba92b53c1a9264edb8164c84cd/lib/foreman/exception.rb#L8-L14) and there's a command to extract them.

While today it's not well maintained, it can be a solid base for user troubleshooting. Perhaps developers need to improve their code a bit more to make it usable.

Historically it was a wiki so users could also add solutions. These days the wiki isn't used that much anymore so I'm considering replacements.

Upstream we have Discourse and can also consider that. Can also be interesting to work with downstream customer support to see which errors come in.

Originally posted by @ekohl in #3549 (review)

[maximiliankolb]

I just tested foreman-rake exception:codes on Foreman/Katello 3.11. ✅

Partial output:

 * [[ERF42-8590]] - unable to sign a non pending certificate
 * [[ERF42-8739]] - Setting '%s' is already defined, please avoid collisions
 * [[ERF42-8800]] - Cannot register compute resource, wrong type supplied
 * [[ERF42-8966]] - Cache refreshing is not supported for %s
 * [[ERF42-9046]] - Unsupported search operators :and / :or
 * [[ERF42-9186]] - Absolute command path must be specified: %s

I think using these error codes is a neat idea for the docs to make it easier to find troubleshooting instructions.

Perhaps developers need to improve their code a bit more to make it usable.

Do you think we should add possible remediations to the docs as part of troubleshooting sections or directly to the code?

• 👍 [[ERF42-6165]] - :foreman_url is not set, please configure in the Foreman Web UI (Administer -> Settings -> General)
• less helpful: unknown permission %s

[asteflova]

Do you think we should add possible remediations to the docs as part of troubleshooting sections or directly to the code?

* 👍 `[[ERF42-6165]] - :foreman_url is not set, please configure in the Foreman Web UI (Administer -> Settings -> General)`

* less helpful: `unknown permission %s`

#3549 (comment) suggests that the same error might show in multiple scenarios. So adding remediations to code might be impractical because too many might need to be listed.

https://github.com/theforeman/foreman-documentation/pull/3549/files was to some extent about adding this as a troubleshooting section within a procedure. Another approach might be to create a troubleshooting guide that would list all errors. Yet another approach would be to leverage modularity and do both a separate guide and then also reuse the individual modules in related procedures.

Historically it was a wiki so users could also add solutions. These days the wiki isn't used that much anymore so I'm considering replacements.

I like the idea of a wiki where users can add their solutions. That would probably be easier with docs than code.

[ekohl]

I just realized from another conversation that we no longer show the exception message in our error pages since theforeman/foreman@a7874f5 and because of that, these ERF codes aren't shown as much anymore. Instead, users have to go through an admin.

[asteflova]

Notes from my research:

• ERF codes aren't referenced in customer cases and solutions very often.
• The same error might have multiple solutions.
• It's not just about ERF errors, any error/problem might be worth documenting in a troubleshooting guide.
• We have existing infrastructure to document troubleshooting errors (.Troubleshooting sections as part of procedures, = Troubleshooting appendices as part of guides).
• Any troubleshooting guide/section should include: what the error means, which log files to check further, which config files to inspect, which objects (subnet, domain, host, repo, etc. ) to inspect in foreman/katello. The idea shouldn't be just to copy-paste whatever is already shown in the error message or logs but provide additional value.

[asteflova]

I'm a big fan of documenting errors and their solutions but the idea of creating a troubleshooting guide based on ERF codes didn't seem to cause a wave of enthusiasm downstream. So I'm wondering whether we really should add so much new content when noone seems to feel a real need for it.

Then again we do have #45 which requests adding a troubleshooting guide for provisioning.

So what do you think @maximiliankolb and @ekohl: Based on your findings and the comments above, do you want to keep this issue open and migrate https://projects.theforeman.org/projects/foreman/wiki/ErrorCodes to foreman-documentation? Or should we close it?