Interpretation of fourth layer preferences

[mateuszmalinowski]

First of all thanks for putting your efforts on making deep visualizations available to a broader audience.
Due to the lack of other options I would like to ask you here about your DeepPref class.

If my memories are correct, visualizing normalized weights from the first layer corresponds to a close-form solution of the optimization problem 'which input maximizes 1st layer activations'. I see that something similar makes sort of sense between the 3rd and 1st layers too - as it is done in the pref_grid function. However, I fail to see any interpretation with further layers. For instance DeepPref between 4th and 1st layers returns K weights between the two first layers, where K is sorted according to the the activations of the 4th layer. What does it mean?

[EderSantana]

The intuition would be just a depth first search on the connection graph. We use the first layer filters as input themselves and check how much that activated a neuron on a deep layer. To do the visualization, we get a random neuron on a deeper layer and ask which samples (i.e. neurons in the first layer) activated the most that deep neuron?