Use type=error to identify error datasets

[dcesari]

I am testing an archive setup with multiple dataset groups for scanning, but I cannot force them to use an error dataset with a name different from error, if I create a dataset error1 with the config:

name = error1
step = daily
type = error
remote import = yes
delete age = 3

arki-merge the configs and so on, arki-scan --dispatch says:

Traceback (most recent call last):
  File "/usr/bin/arki-scan", line 11, in <module>
    main()
  File "/usr/bin/arki-scan", line 7, in main
    sys.exit(Scan.main())
  File "/usr/lib/python3.6/site-packages/arkimet/cmdline/base.py", line 85, in main
    return cmd.run()
  File "/usr/lib/python3.6/site-packages/arkimet/cmdline/scan.py", line 156, in run
    arki_scan.set_dispatcher(**kw)
RuntimeError: configuration of dataset 'error1' does not have a 'filter' directive

apparently it interprets it as an ordinary dataset.

[spanezz]

I confirm: arkimet looks for a dataset called error, and type=error is only used to choose the dataset on-disk format (error is currently an alias for simple, but this way it can be changed to a different format in the future)

[dcesari]

So the only way to have multiple error datasets for different dataset groups is to keep them in separate directories?

[spanezz]

So the only way to have multiple error datasets for different dataset groups is to keep them in separate directories?

If you mean, you create different scanning configurations for different groups of datasets, then yes, at the moment one has to have datasets in different directories all called error, and pick one or the other in arki-mergeconf as needed.

It's an interesting use case, though. I can change arkimet to use a dataset of type=error regardless of the name as an error dataset when dispatching, and complain if the dispatching configuration contains more than one dataset with type=error.

Shall I do that?

[dcesari]

Well, if that's not hard and does not break other assumptions, it would be useful.