MatMulNBits collapse shape input when > 1d

[TedThemistokleous]

Fix input shape for matmulnbits to fold input via reshape to 1d input

[pfultz2]

Since this is just to check then we should just use .elements instead of inserting a reshape: if(args[2]->get_shape().elements() != (n * n_blocks_per_col}))

[TedThemistokleous]

Easy enough, but don't we need these to be in the correct 1d shape for the input? in the dequantize_b we do another reshape as well on the input scale.

auto scales = info.add_instruction(make_op("reshape", {{"dims", {n, -1}}}), args[2]);

[pfultz2]

No, because the first thing we do is reshape it to a 2d tensor of{n, -1}(where -1 is the remaining elements).

[pfultz2]

Also, you are not even using the reshaped instruction that is inserted, so there is no reason to add something to be always removed by DCE.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 92.23%. Comparing base (4b15b6c) to head (86d9ee3).
Report is 11 commits behind head on develop.

Additional details and impacted files

@@           Coverage Diff            @@
##           develop    #3698   +/-   ##
========================================
  Coverage    92.23%   92.23%           
========================================
  Files          514      514           
  Lines        21746    21746           
========================================
  Hits         20057    20057           
  Misses        1689     1689           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[migraphx-bot]

Test	Batch	Rate new 86d9ee	Rate old 64fe0c	Diff	Compare
torchvision-resnet50	64	3,257.33	3,254.94	0.07%	✅
torchvision-resnet50_fp16	64	6,989.07	6,977.96	0.16%	✅
torchvision-densenet121	32	2,435.20	2,436.55	-0.06%	✅
torchvision-densenet121_fp16	32	4,072.09	4,076.26	-0.10%	✅
torchvision-inceptionv3	32	1,628.24	1,627.46	0.05%	✅
torchvision-inceptionv3_fp16	32	2,743.93	2,741.58	0.09%	✅
cadene-inceptionv4	16	763.69	764.31	-0.08%	✅
cadene-resnext64x4	16	814.08	813.14	0.11%	✅
slim-mobilenet	64	7,463.27	7,466.83	-0.05%	✅
slim-nasnetalarge	64	209.03	209.03	0.00%	✅
slim-resnet50v2	64	3,440.71	3,443.32	-0.08%	✅
bert-mrpc-onnx	8	1,148.20	1,144.17	0.35%	✅
bert-mrpc-tf	1	470.97	474.21	-0.68%	✅
pytorch-examples-wlang-gru	1	438.61	416.53	5.30%	🔆
pytorch-examples-wlang-lstm	1	386.11	384.23	0.49%	✅
torchvision-resnet50_1	1	815.86	783.29	4.16%	🔆
cadene-dpn92_1	1	435.47	398.94	9.16%	🔆
cadene-resnext101_1	1	382.89	383.46	-0.15%	✅
onnx-taau-downsample	1	345.82	345.52	0.09%	✅
dlrm-criteoterabyte	1	33.32	33.33	-0.03%	✅
dlrm-criteoterabyte_fp16	1	52.71	52.73	-0.03%	✅
agentmodel	1	8,192.62	8,127.83	0.80%	✅
unet_fp16	2	58.90	58.89	0.02%	✅
resnet50v1_fp16	1	969.57	938.63	3.30%	🔆
resnet50v1_int8	1	1,011.74	984.73	2.74%	✅
bert_base_cased_fp16	64	1,168.03	1,170.23	-0.19%	✅
bert_large_uncased_fp16	32	362.90	362.94	-0.01%	✅
bert_large_fp16	1	198.96	200.28	-0.66%	✅
distilgpt2_fp16	16	2,196.53	2,198.50	-0.09%	✅
yolov5s	1	530.16	531.33	-0.22%	✅
tinyllama	1	43.38	43.34	0.08%	✅
vicuna-fastchat	1	175.51	172.03	2.02%	✅
whisper-tiny-encoder	1	416.76	418.00	-0.30%	✅
whisper-tiny-decoder	1	428.99	428.83	0.04%	✅

Check results before merge 🔆

[migraphx-bot]

     ✅ bert-mrpc-onnx: PASSED: MIGraphX meets tolerance

     ✅ bert-mrpc-tf: PASSED: MIGraphX meets tolerance

     ✅ pytorch-examples-wlang-gru: PASSED: MIGraphX meets tolerance

     ✅ pytorch-examples-wlang-lstm: PASSED: MIGraphX meets tolerance

     ✅ torchvision-resnet50_1: PASSED: MIGraphX meets tolerance

     ✅ cadene-dpn92_1: PASSED: MIGraphX meets tolerance

     ✅ cadene-resnext101_1: PASSED: MIGraphX meets tolerance

     ✅ dlrm-criteoterabyte: PASSED: MIGraphX meets tolerance

     ✅ agentmodel: PASSED: MIGraphX meets tolerance

     ✅ unet: PASSED: MIGraphX meets tolerance

     ✅ resnet50v1: PASSED: MIGraphX meets tolerance

     ✅ bert_base_cased_fp16: PASSED: MIGraphX meets tolerance

🔴bert_large_uncased_fp16: FAILED: MIGraphX is not within tolerance - check verbose output

     ✅ bert_large: PASSED: MIGraphX meets tolerance

     ✅ yolov5s: PASSED: MIGraphX meets tolerance

     ✅ tinyllama: PASSED: MIGraphX meets tolerance

     ✅ vicuna-fastchat: PASSED: MIGraphX meets tolerance

     ✅ whisper-tiny-encoder: PASSED: MIGraphX meets tolerance

     ✅ whisper-tiny-decoder: PASSED: MIGraphX meets tolerance

     ✅ distilgpt2_fp16: PASSED: MIGraphX meets tolerance