Tracking: Reworking includes/excludes

[simonsan]

Open questions

• What are the pain points we are trying to fix?
  • main pain point is that we do rely on ignore's logic, which currently is preventing us from implementing other sources easily... But it may be that in order to implement some include/exclude logic, we may have to modify the source trait
• when we decide to add backup possibilities from other sources (opendal, rclone,..), we also need options to include/exclude which should be general and not depending on the specific backend. Currently, the includes/excludes are handled by the ignore crate which only works for local FS

Features to add

• add something like exclude-fn using rhai: Feature discussion: Programatically defined file-filtering. rustic#1317

TODO

• implement testing first for the current way that includes and excludes work
  • we should also implement a more high-level test in rustic itself, to make sure, we check if the overall behaviours will stay the same or not (breaking changes): Implement more testing for including/excluding rustic#1318
• adjust the tests in the same PR that we implement the new way how includes and excludes should work
• ignore::LocalSource::new and tree::NodeStreamer::new_with_glob share the same logic for initializing an OverrideBuilder
  • we should probably refactor that to a module that contains only logic for glob handling
    will also make it easier to test, I guess
  • this should be source-independent (or at least we should have a source-independent glob handling for include/exclude and maybe additional some source-dependent include/exclude like one-filesystem) so that we can work on other sources and fix the open issues about the ignore things

Related

• Milestone: https://github.com/rustic-rs/rustic/milestone/3