Implement special case connectivity matrices with pre/post IDs for models that effect only pre/post variables

[denisalevi]

We store synapse IDs in our connectivity matrices, which we read during synaptic effect application. If the synaptic effect modifies pre- or postsynaptic neuron variables, the synapse IDs are used to index the array of pre- or postsynaptic neuron IDs.

This general setup allows for spike to have synaptic effects on pre, post and synapse variables.

If the effect only modify pre or post neuron variables though, we can store the pre or post IDs directly in the connectivity matrix and use those for effect application (and even for spike propagation for heterogeneous delays). This would save us one global memory read in the effect application kernel (instead of reading synID, and use that as index to get pre/postID, we would directly have pre/postID). And that memory read is likely not coalesced, so it might be worth it!

See paper figure 2.

EDIT: Should also work if effects have multiple targets. We can just read pre/post/syn IDs directly from YALE format connectivity matrix.

EDIT 2: The discussion below is only for the special case of 1-to-1 connectivity and sorting synapse variables in connectivity structure. The topic of this issue (sorting pre/post IDs into the connectivity matrix) is independent of that. Just one thing to keep in mind: If we only change the sorting of pre/post IDs, that can have negative effects for synapses with clock-driven dynamics, which read pre- or post neuron variables. Those reads are only coalesced if pre- and post IDs are sorted by synapse ID. One way around that would be to assign each thread during synaptic state updates to the synapse ID in the connectivity matrix! Then we have one additional coalesced read, but reading pre- and post IDs would also be coalesced. Worth it?

[denisalevi]

Another special case:

• 1-to-1 connectivity (or more generally: every post neuron is targeted at most by 1 synapse)

Here, we could store the target variable (e.g. post voltage) directly in connectivity matrix structure, saving one more memory read and getting coalesced access during target variable applications. If the same population variable is targeted by multiple synaptic pathways with this setup, then this will only benefit one pathway. Then just pick one, maybe the largest? Or the one that runs most often (based on clock)? Etc.

[mstimberg]

Sorry if I am getting confused, but how would this work in practice given that the target variable, say the membrane potential, is also used elsewhere (e.g. in the state updater)? If it gets changed directly in the connectivity matrix, wouldn't we have to transfer changes back to the "main array" and therefore lose the benefit of saving a memory read?

[denisalevi]

My suggestion is for the special case of 1-to-1 connectivity. Here, we just sort the "main array" in the way the connectivity matrix is sorted (which we can do since we have a 1-to-1 mapping from synapse of post neurons). Then we would just use the thread index (with an offset depending on the presynaptic neuron and the postsynaptic group, a.k.a the start index for a given synapse group in the connectivity matrix) to index the voltage array directly for our update, getting rid of reading synapse ID and form the postsynaptic neuron ID. In the state udpater, thresholder etc. we use that same voltage array (but state updater and thresholder don't care about the order of the neurons).

This sorting of the "main array" (voltages) can only be done for one synaptic pathway of course. If there are multiple, then only one pathway would benefit from that (in all other synaptic pathways, we would have to at least read the correct post neuron ID using the thread id).

And this also fails when having multiple synapses targeting the same post neuron, because then we would suddenly have multiple memory locations for the same post neuron's voltage. But it would work if not all post neurons have a synapse. Then we just sort the voltage array such that all neurons with synapse come first, then all remaining neurons.

Okay. All this was the special case of 1-to-1 connectivity. For all other cases with only pre or only post effects, we can still save one memory read by storing target IDs directly instead of synapse IDs (my initial post).

Does this make more sense now? Or am I thinking something wrong? :)

[mstimberg]

My suggestion is for the special case of 1-to-1 connectivity. Here, we just sort the "main array" in the way the connectivity matrix is sorted (which we can do since we have a 1-to-1 mapping from synapse of post neurons).

Ok, this does indeed make sense. But if the "main array" changes its order in any way, this means we need to 1) sort all the other neuronal arrays in the same way and 2) change the order back at the end of the simulation. And then for things like monitors, we'd also have to take care to translate the indices. I'm a bit worried we are opening a can of worms 😨

Okay. All this was the special case of 1-to-1 connectivity. For all other cases with only pre or only post effects, we can still save one memory read by storing target IDs directly instead of synapse IDs (my initial post).

This would only work for synapses that don't refer to any synaptic variables, right? I.e. things like synapses with constant weights, but if you need synapse-specific weights, then you need to get them from somewhere. Or would they be in a parallel connectivity matrix?

[denisalevi]

My suggestion is for the special case of 1-to-1 connectivity. Here, we just sort the "main array" in the way the connectivity matrix is sorted (which we can do since we have a 1-to-1 mapping from synapse of post neurons).

Ok, this does indeed make sense. But if the "main array" changes its order in any way, this means we need to 1) sort all the other neuronal arrays in the same way and 2) change the order back at the end of the simulation. And then for things like monitors, we'd also have to take care to translate the indices. I'm a bit worried we are opening a can of worms fearful

Ah yes, you are right, we would need some mechanism for sorting and re-sorting IDs. And the question is if that would be worth the effort since it would be only for 1-to-1 connections. For now, I would skip on that special case optimization.

But we might need such a mechanism anyways if we want to explore different sorting strategies for synaptic variables. There we don't need any special connectivity, but could access synaptic variables directly if they are are sorted by the connectivity matrix structure. And that could be quite interesting, see my comment in #32. And if we have a mechanism for going from Python -> C++/CUDA - > Python sorting, then we can also think about the special case here with the 1-to-1 connectivity.

Okay. All this was the special case of 1-to-1 connectivity. For all other cases with only pre or only post effects, we can still save one memory read by storing target IDs directly instead of synapse IDs (my initial post).

This would only work for synapses that don't refer to any synaptic variables, right? I.e. things like synapses with constant weights, but if you need synapse-specific weights, then you need to get them from somewhere. Or would they be in a parallel connectivity matrix?

You can actually do the same for synapses (see the comment I referenced a few lines above). And it would even make much more sense for synapses than for neurons (where it only works in the special 1-to-1 case). Basically, when possible, store state variables sorted in the way of the connectivity matrix, then we don't need the connectivity matrix and have direct access to state variables via synapse index (given by thread index). Again, this probably needs a sorting/re-sorting mechanism.

Thanks for asking those questions. I really had to think about this again and rewrote my answer like 5 times. So many aspects to consider 🙈 Does that make sense to you or am I missing something? Bottom line for me.

1. Find a mechanism to sort / re-sort variables.
2. First look at sorting synapses, which would benefit all plasticity models (synaptic target variables).
3. Look into the special case of 1-to-1 connectivity, where pre- and postsynaptic updates could also benefit from this.

Question is if it would be worth it.