Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor validation of generic_resolver rules to startup #694

Merged
merged 3 commits into from
Nov 11, 2024

Conversation

dtrai2
Copy link
Collaborator

@dtrai2 dtrai2 commented Nov 5, 2024

  • Move validation logic to __attrs_post_init__ in GenericResolverRule.
  • Remove redundant error handling in GenericResolver processor.
  • Update corresponding tests to reflect changes.

- Move validation logic to `__attrs_post_init__` in GenericResolverRule.
- Remove redundant error handling in GenericResolver processor.
- Update corresponding tests to reflect changes.
@dtrai2 dtrai2 requested a review from djkhl November 5, 2024 12:04
@dtrai2 dtrai2 added the enhancement New feature or request label Nov 5, 2024
@dtrai2 dtrai2 marked this pull request as ready for review November 5, 2024 13:48
Copy link
Collaborator

@djkhl djkhl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good and works as expected :)

@ppcad
Copy link
Collaborator

ppcad commented Nov 8, 2024

Getters need to be updated on the fly in the generic resolver. They are going to be refreshed periodically to get updated data from the HTTP getter.
Therefore, raising a configuration error leads to problems, since this can happen when Logprep is already running.
I'm going to create a pull request for this.

@dtrai2
Copy link
Collaborator Author

dtrai2 commented Nov 8, 2024

Getters need to be updated on the fly in the generic resolver. They are going to be refreshed periodically to get updated data from the HTTP getter.
Therefore, raising a configuration error leads to problems, since this can happen when Logprep is already running.
I'm going to create a pull request for this.

but isn't it already working with the general config refresh we have in Logprep? That way the configuration is also reloaded periodically and in case an exception is thrown, Logprep will continue running with the previous configuration.

@ppcad
Copy link
Collaborator

ppcad commented Nov 8, 2024

Getters need to be updated on the fly in the generic resolver. They are going to be refreshed periodically to get updated data from the HTTP getter.
Therefore, raising a configuration error leads to problems, since this can happen when Logprep is already running.
I'm going to create a pull request for this.

but isn't it already working with the general config refresh we have in Logprep? That way the configuration is also reloaded periodically and in case an exception is thrown, Logprep will continue running with the previous configuration.

Oh, you are right, it does that. Last time I checked I've gotten an error when I raised a configuration error. I must have made a mistake then.

However, as far as I know, a config refresh can't detect changes in HTTP getters, so it won't reload.
Furthermore, relying on a reload would require to reaload the whole config.
What I have done now is make the resolver refresh getters only.
This would require some form of _ensure_rules_from_file again.
I suppose I could make it raise a warning and retain the old getter values on failure.
What do you think about this?

@ekneg54
Copy link
Collaborator

ekneg54 commented Nov 8, 2024

What I have done now is make the resolver refresh getters only.

We should avoid implementing special mechanics in favor of get a solution for the config refresh. I think it is better to solve the problem in the config refresh that configs behind getters gets evaluated too on config refresh.

yes I see, that the fix here could be very easy. but if we have a lot of these easy fixes again it will be hard to implement global mechanics again.

@dtrai2
Copy link
Collaborator Author

dtrai2 commented Nov 11, 2024

as discussed offline we can merge this pull request and then find a solution for loading external resources (that are not rule configurations) later.

@ekneg54 ekneg54 merged commit 382848f into main Nov 11, 2024
13 checks passed
@ekneg54 ekneg54 deleted the dev-refactor-generic-resolver-rule-validation branch November 11, 2024 07:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants