关于实验部分的2D-attention map

[zdz1997]

作者您好，很感谢您的分享，请问在您的论文中实验部分，其中绘制了2D attention map 的可视化的热力图。请问这是如何绘制的，我尝试用grad cam绘制，尝试了很多次，都以失败告终。您方便告诉我这是如何绘制的吗？或者您可以提供一下绘制的程序吗？我将不胜感激。
最后再次感谢作者的分享与工作，谢谢您！

[lexiaoyuan]

Hello, thank you very much for being able to open the source code, it is a very meaningful work.
I am a graduate student and would like to do my own attempt based on your paper. Now I would like to draw an attention map similar to the one shown in Figure 5 and Figure 7 of your paper. Thank you very much!

[luyang-NWPU]

可视化的是Transformer里Q和K计算得到的相似度矩阵。 解码某个字母的时候，有个经过Masked Self Attention的Query特征，它会和6x20个CNN特征（Key）计算相似度。我们可视化的就是这个6x20的数组（由于是Multi-Head Attention，实际会有多个6x20的数组，我可视化的时候是取平均值）。有了这个数组，就可以用opencv进行伪彩色处理，然后叠加到原图上了。例如：         img = cv2.imread(input_img)         h, w, _ = img.shape         scores=  (scores- scores.min()) / (scores.max() - scores.min())         scores= np.uint8(255 * scores)         heatmap = cv2.applyColorMap(cv2.resize(scores, (w, h)), cv2.COLORMAP_JET)         result = heatmap * 0.3 + img * 0.5         cv2.imwrite(filename, result) 很欣慰生命某段时刻 曾一起度过...  

…

------------------ Original ------------------ From: "luyang-NWPU/HGA-STR" ***@***.***>; Date: Fri, Jan 21, 2022 09:46 PM ***@***.***>; ***@***.***>; Subject: Re: [luyang-NWPU/HGA-STR]关于实验部分的2D-attention map (#4) Hello, thank you very much for being able to open the source code, it is a very meaningful work. I am a graduate student and would like to do my own attempt based on your paper. Now I would like to draw an attention map similar to the one shown in Figure 5 and Figure 7 of your paper. Thank you very much! — Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>