This repository has been archived by the owner on Nov 16, 2023. It is now read-only.
Add sample configuration loaders for inMemory and ciab #106
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Sketched out implementations of two different config loaders that go with the PR from @patrick-steele-idem for refactoring the crawler factory. That PR should end up in this branch eventually.
This change takes a poor man's dependency injection approach (thanks @willbar). The basic idea is that a config loader creates effectively a dependency injection container full of services that make up the crawler (including the crawler itself). Each service is created by a factory (or whatever you want). The standard pattern is for the factory function to get a container of services previously created, an optional name (if provided it should add itself to the container) and an optional set of default settings.
There is also a philosophical change in that the provider code should have core defaults built in so the user does not have to spec any/many options. We'll have to document the available options for each provider but that's a good thing anyway.