IndexError in lrp.utils::LayerRelevance.scatter(-1).size(1) caused in lrp.common::prop_C3 using Custom YOLOv5 v6.1/6.2

[Timmimim]

Hello there,

First off, thanks for your work on this repo!

I am trying to run your explainer on a custom trained YOLOv5 under late v6.1 (pulled from master shortly ahead of v6.2 release). After some tinkering, I got the code framework to run with my setup. The showcase example set by the default parameter values for the explain.py script works just fine.

I am then calling the explain.py script with custom paths for the --weightsand --source flags. All other params are still the default.

Unfortunately, I get the following error message (I am logging relevance ahead of the message):

LayerRelevance(0.0, cache=(17, 0.65351) (20, 0.2363) (23, 0.11019), contrastive=False)

Traceback (most recent call last):
  File "/nn_training/explainer/yolo-heatmaps/explain.py", line 391, in <module>
    main(opt)
  File "/nn_training/explainer/yolo-heatmaps/explain.py", line 387, in main
    run(**vars(opt))
  File "/.conda/envs/YOLOexplainer/lib/python3.10/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/nn_training/explainer/yolo-heatmaps/explain.py", line 282, in run
    get_explanation(inn_model, init, contrastive, b1, b2, int(cls),
  File "/nn_training/explainer/yolo-heatmaps/explain.py", line 54, in get_explanation
    lrp_out = inn_model.innvestigate(in_tensor=None, initializer=init)
  File "/nn_training/explainer/yolo-heatmaps/innvestigator.py", line 351, in innvestigate
    relevance = self.inverter(layer, relevance)
  File "/nn_training/explainer/yolo-heatmaps/inverter.py", line 206, in __call__
    return self.invert(layer, relevance, **kwargs)
  File "/nn_training/explainer/yolo-heatmaps/inverter.py", line 199, in invert
    return self.inv_funcs[type(layer)](self, layer, relevance, **kwargs) 
  File "/nn_training/explainer/yolo-heatmaps/lrp/common.py", line 98, in prop_C3
    c_ = msg.size(1)

IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)

relevance.scatter() returns several cuda tensors, though looking different from the Tensors for the default yolov5s.pt net provided by you. I found that
But then, relevance.scatter(which=-1) returns an empty tensor([]) object.

The explain.py fails after this point. I don't know where to even start fixing the issue at this point.

Can you give me advice as to how to fix this? Am I missing something in the parameter flags maybe?
Do you need more info on the issue?

Thanks in advance and have a wonderful day!
Best, Timm

[Timmimim]

Oh, I didn't see your note in README before that "Current version only supports YOLOv5s-x models".
This turns out to be the source of my errors. I trained a yolov5m6, and was feeding it into your explain.py.

As it turns out, the yolov5s6 architecture is also incompatible with your algorithm.

So I trained three new networks, based on the yolov5s.pt, yolov5m.pt and yolov5l.pt weight files, respectively. All three resulting networks appear to work just fine with your workflow implemented in explain.py.
So when you state that only the s variant is supported, does that imply that the results for the larger models are less reliable? If so, why, and how would I adjust that?

Are you planning to implement support for the larger variants, as well as the x6 variants?

If no, how would I go about implementing the same? Can you direct me toward the key differences between the architectures that limit/prohibit applicability of your algorithm at this point?

I would likely be keen on implementing at least yolov5m-x and yolov5l-x support (and also then likela the s variant) in a Fork and create a pull request on completion.

[shakib-root]

Hi @Timmimim , could you let me know how you installed the required packages from requirements.txt? For me it's taking a long time to resolve the dependency issues.

[Timmimim]

Hey @shakib-root,
I have installed all my dependencies into a conda environment. I spent some hours figuring everything out, so I cannot really retrace everything. But I can export and share with you my conda environment, which you should be able to then recreate locally. If you run into any problems, I'm also happy to help (if I can).
Important to look out for is the cross-compatibility of your CUDA-version and the desired PyTorch version. This may still break your install, so my solution might not fit your setup perfectly.

The file linked below is the result of running conda env export with my environment active. Unfortunately, GitHub does not like the .yml file type. So I had to convert to .txt (just by re-naming), which you would have to change back.
Simply run conda env create -f <env_file>.yml to re-create my YOLOexplainer environment. Change this name in the .yml-file to make it your own, if you wish.
YOLO-heatmaps_conda_environment.txt (change filetype to .yml after download)

Additionally, this additional file holds the raw list of packages which work for me (probably with some extras you don't need, since I toyed around a bit). It is merely the output of conda list --export, which should also suffice to recreate my environment (see first lines of file: $ conda create --name <env> --file <this file>), and you can pick and choose a new environment name.
yolo-heatmaps_dependencies.txt

[shakib-root]

@Timmimim thank you very much. I was actually able to build a working environment by using current yolov5 repo's requirements.txt. Later I have updated some parts of explain.py with the current yolov5's detect.py but it seems this repository doesn't properly support v6.1 and v6.2 models and I am getting the same error as yours in the lrp module. Were you able to resolve the lrp issue for the yolov5 version6.2 models?