Unused tags

[nvpstr]

In mlp_compliance/configs/v0.5.0_level2.yaml
- EXACTLY_ONE: run_clear_caches
No reference is using this currently.

- AT_LEAST_ONE: train_learn_rate
Should be opt_learning_rate.
train_learn_rate is not used and not even defined in tags.py

[bitfort]

Change train_learn_rate to opt_learning_rate

I think a reference to run_clear_caches was added to the reference repo... this should be printed when caches are cleared (or printed if you know caches are cleared). Are you okay with leaving this here and having submissions print it?

I'll change it to AT_LEAST_ONE, though. Since I think there may be multiple points caches can be cleared.

[nvpstr]

My point is that the ref implementation should have run_clear_caches if we require it. Currently sample logs are missing that.

Besides according to the policy system restart before the training can be an alternative to the cache clearing...and there is no tag needed for that.

[nvpstr]

To be honest I am strongly supporting using this tag, as it enforces the 6.1 policy point.

[nvpstr]

On the other hand I am not sure if you anybody is able to clear cache without root privileges.

[bitfort]

True - I think this is a good point. The references don't have them because it is non trivial to make the changes (and it is difficult to do within docker too).

Maybe we can require the tag to be present (for people to acknowledge the rule) but people can be flexible in how they choose to implement? e.g. you can print the tag if you've manually checked this is done? Or add it with a script later from the control harness?

[nvpstr]

I agree. It could be the first tag issued in the training script if the preprocessing is done separately.