[Draft] MBSK prefetch optimization

[boringmorning]

Currently use "MbskPrefetchOpt" to control the MBSK reduction order:

0 (default): Reduce same element from all WGs, and repeat for all elements. This is the original design.

1: Reduce all elements from one other WG, and repeat for all WGs. This design can potentially improve data prefetch.

[boringmorning]

In these 2 benchmarks, I tuned ~2000 kernels each for MbskPrefetchOpt = 0 and MbskPrefetchOpt = 1 and compare their winners' performance (all other parameters settings are the same). For the benchmark with smaller size, the result are almost the same. In contrast, MbskPrefetchOpt = 1 resulted in noticable improvement with the larger sizes.

[KKyang]

why another new parameter? Can't we make this auto calculated? (Calculated in runtime and use a flag to control the branch)

[boringmorning]

why another new parameter? Can't we make this auto calculated? (Calculated in runtime and use a flag to control the branch)

@KKyang This is an option, but how to decide the branching is an important problem. Also will result in larger code size (hundreds to thousand lines of code).
Note that this is just the draft version, and one reason to use a new parameter is for easier benchmarking. I will have a presentation & discussion about this optimization (maybe next week).