03.srt

1
00:00:00,000 --> 00:00:03,600
字幕生成: BLACK 字幕校对: 杨绎

2
00:00:04,350 --> 00:00:06,160
Hello,大家好

3
00:00:06,160 --> 00:00:09,920
来到AI编译器系列之PyTorch

4
00:00:09,920 --> 00:00:14,920
然后在PyTorch 2.0里面推出了Dynamo新的特性

5
00:00:15,920 --> 00:00:17,760
在进入新的特性讲解之前

6
00:00:17,960 --> 00:00:21,880
将会去回顾PyTorch在静态图方面的一个尝试

7
00:00:22,000 --> 00:00:26,640
现在来到了PyTorch FX和PyTorch Lazy Tensor这个环节

8
00:00:27,640 --> 00:00:31,080
现在让来看看PyTorch FX的一个原理

9
00:00:31,080 --> 00:00:37,000
一上来肯定来先看看PyTorch FX官网宣称的一个设计的principle

10
00:00:37,000 --> 00:00:38,360
就是原则

11
00:00:38,360 --> 00:00:41,080
这里面设计原则的第一条就是

12
00:00:41,080 --> 00:00:44,720
尽可能的去捕捉大部分常用的网络模型

13
00:00:44,720 --> 00:00:46,680
能够进行一个控制和转换的

14
00:00:46,680 --> 00:00:51,520
但是有一点很重要的就是去避免支持一些长尾的一些

15
00:00:51,520 --> 00:00:53,240
或者复杂深奥的用例

16
00:00:53,240 --> 00:00:56,000
就是可能有部分场景我是不支持的

17
00:00:56,000 --> 00:00:57,600
我已经明确声明出来了

18
00:00:57,600 --> 00:01:01,080
第二个设计的原则就是使用这个工具了

19
00:01:01,080 --> 00:01:04,000
能让很快很方便的熟悉起来

20
00:01:04,000 --> 00:01:08,320
所以它主要是采用了Python的一个数据结构去操作这个静态图

21
00:01:09,290 --> 00:01:14,690
第三点就是能让开发者对这个静态图能够有一个比较高的可控度

22
00:01:14,720 --> 00:01:17,000
就是你想怎么转换这个静态图

23
00:01:17,000 --> 00:01:18,560
就怎么转换这个静态图

24
00:01:18,560 --> 00:01:20,200
想怎么改就怎么改

25
00:01:20,200 --> 00:01:23,440
那三条设计原则往下继续去看一下

26
00:01:24,320 --> 00:01:28,720
下面两个表格就是Torch FX的一个节点表

27
00:01:28,720 --> 00:01:31,880
这里面可以看到它有很多不同的操作

28
00:01:31,880 --> 00:01:35,840
然后每个操作后面又有不同的含义和behaviors

29
00:01:35,840 --> 00:01:40,680
这里面不会去详细的去展开Torch FX的一个IR

30
00:01:40,680 --> 00:01:41,840
虽然看的是表格

31
00:01:41,840 --> 00:01:46,440
但是实际上Torch FX使用的是一个DAG的一个IR表达形式

32
00:01:46,440 --> 00:01:50,720
通过一系列线性的节点去表示所有的操作

33
00:01:50,760 --> 00:01:53,960
这个节点是由一系列的字符操作码去组成的

34
00:01:53,960 --> 00:01:58,560
去描述算子或者每一层具体表达什么含义

35
00:01:58,560 --> 00:02:01,920
可以看到placeholder就是函数的输入

36
00:02:01,920 --> 00:02:03,920
然后带着三个调用的节点

37
00:02:03,920 --> 00:02:06,280
code method, code module, code function

38
00:02:06,280 --> 00:02:09,080
然后获取算子的attribute

39
00:02:09,080 --> 00:02:10,200
最后输出

40
00:02:10,200 --> 00:02:11,760
这个就是它的IR定义

41
00:02:11,760 --> 00:02:14,560
Torch FX的IR定义的比较简单

42
00:02:14,560 --> 00:02:17,400
但是通过这么简单的几个定义抽象

43
00:02:17,440 --> 00:02:20,920
能够快速的把拍Torch的动态图转为静态图

44
00:02:20,920 --> 00:02:23,680
所以这也是Torch FX的一个IR定义

45
00:02:23,680 --> 00:02:28,680
这个Torch FX的IR定义做的比TorchScript的IR要简单要方便

46
00:02:29,160 --> 00:02:32,520
下面来看看Torch FX的其实有几个IR

47
00:02:32,520 --> 00:02:34,640
刚才简单的去讲了讲

48
00:02:34,640 --> 00:02:37,320
后面这个就是它具体的含义和具体的内容

49
00:02:37,320 --> 00:02:39,000
不在这详细的介绍

50
00:02:39,000 --> 00:02:40,000
往下走

51
00:02:40,000 --> 00:02:41,840
看一下它具体是怎么实现的

52
00:02:41,840 --> 00:02:44,520
首先Torch FX还是比较好实现的

53
00:02:44,520 --> 00:02:45,960
Torch.FX就行了

54
00:02:46,280 --> 00:02:48,440
接下来定义一个函数叫做-

55
00:02:48,440 --> 00:02:50,760
然后输入是两个x和y

56
00:02:50,760 --> 00:02:52,960
然后x和y分别做一个sin和cos

57
00:02:52,960 --> 00:02:54,320
然后把它相加

58
00:02:54,320 --> 00:02:56,360
最后用起来很简单

59
00:02:56,360 --> 00:02:59,000
用Torch FX的SymbioticTrace

60
00:02:59,000 --> 00:03:00,480
SymbioticTrace

61
00:03:00,480 --> 00:03:02,960
看到这个Trace大家有没有熟悉一点

62
00:03:02,960 --> 00:03:07,520
它就是基于TracingBase这个原理去做一个动态图的转换的

63
00:03:07,520 --> 00:03:10,760
下面这个就是print(traced.graph)

64
00:03:10,760 --> 00:03:12,920
就把Torch FX的IR表达出来

65
00:03:12,920 --> 00:03:14,880
可以看到刚才讲的Placeholder

66
00:03:14,920 --> 00:03:15,800
QuadFunction

67
00:03:15,800 --> 00:03:17,840
还有Output都在这里面

68
00:03:17,840 --> 00:03:21,280
这里面就有它的属性Attributes都在这里面

69
00:03:21,600 --> 00:03:23,720
Torch FX的IR其实很简单

70
00:03:23,720 --> 00:03:25,360
所以它组织起来比较简单

71
00:03:25,360 --> 00:03:26,160
也比较易懂

72
00:03:26,160 --> 00:03:28,160
因为它是一个序列化的形态

73
00:03:28,160 --> 00:03:31,440
所以它不能表达所有的场景

74
00:03:31,440 --> 00:03:33,720
遇到一些复杂的if-else-while这些

75
00:03:33,720 --> 00:03:36,400
可能就很难去完全去表达出来

76
00:03:38,040 --> 00:03:39,440
于是做一个总结

77
00:03:39,440 --> 00:03:42,400
看看Torch FX的Pros and Cons

78
00:03:42,560 --> 00:03:45,600
那优点就是能够对正向的计算图进行操作

79
00:03:45,600 --> 00:03:47,600
可以做一些Batch Size的修改

80
00:03:47,720 --> 00:03:48,720
修改某个OP

81
00:03:48,720 --> 00:03:50,200
然后增加某个统计

82
00:03:50,200 --> 00:03:51,480
例如量化

83
00:03:51,480 --> 00:03:54,840
那量化这个功能就是Torch FX主打的功能

84
00:03:54,840 --> 00:03:56,640
因为要做量化训练的时候

85
00:03:56,640 --> 00:03:58,320
需要对图进行修改

86
00:03:58,320 --> 00:04:00,320
插入大量的伪量化的代码

87
00:04:00,320 --> 00:04:04,320
在每个卷迹前面和后面都要插入伪量化的代码

88
00:04:04,320 --> 00:04:06,360
另外还会做一些大算子的替换

89
00:04:06,360 --> 00:04:08,600
就是为了让系统跑得更快

90
00:04:08,600 --> 00:04:11,800
这个时候修改某个操作可能会更加方便

91
00:04:11,800 --> 00:04:15,960
第二点就是它是基于Python对Python的符号trace的方式

92
00:04:15,960 --> 00:04:17,280
那既然是符号trace

93
00:04:17,280 --> 00:04:19,760
是方便学习和源码操作的

94
00:04:19,760 --> 00:04:22,440
因为大家都可以基于Python去操作

95
00:04:22,440 --> 00:04:25,560
而TorchScript它是C++去处理的

96
00:04:25,560 --> 00:04:29,000
第二点就是IR它缩减到了6个操作

97
00:04:29,000 --> 00:04:29,800
简单易学

98
00:04:29,800 --> 00:04:31,440
就刚才讲的那几个

99
00:04:31,440 --> 00:04:33,080
Core就已经占了三个

100
00:04:33,080 --> 00:04:33,960
Placeholder一个

101
00:04:33,960 --> 00:04:34,760
Output一个

102
00:04:34,760 --> 00:04:35,880
ATTR一个

103
00:04:36,240 --> 00:04:39,400
下面来看看Cons它的问题

104
00:04:39,840 --> 00:04:43,160
刚才讲到了它的问题就是

105
00:04:43,160 --> 00:04:47,800
PyTorch的反向是C++去操作的

106
00:04:47,800 --> 00:04:50,240
动态的时候只能表达它一个正向的图

107
00:04:50,240 --> 00:04:51,520
反向由C++

108
00:04:51,520 --> 00:04:53,080
由IR框架去处理

109
00:04:53,080 --> 00:04:53,840
那静态呢

110
00:04:53,840 --> 00:04:57,080
所以基于Python对Python只能表达正向图

111
00:04:57,080 --> 00:04:58,400
很难处理反向图

112
00:04:58,400 --> 00:04:59,720
带有空字流的图

113
00:04:59,720 --> 00:05:01,640
那使用场景还是很有限的

114
00:05:01,640 --> 00:05:02,200
第二个呢

115
00:05:02,200 --> 00:05:03,840
基于PythonTrace的操作

116
00:05:03,840 --> 00:05:07,600
继承TraceBase的一个静态图的一些缺点

117
00:05:07,720 --> 00:05:09,160
还是回到第一点

118
00:05:09,160 --> 00:05:10,960
场景使用受限了

119
00:05:10,960 --> 00:05:14,000
第三点就是需要用户去感知替换

120
00:05:14,000 --> 00:05:15,240
只能处理量化

121
00:05:15,240 --> 00:05:17,120
融合算子等有些情况

122
00:05:17,120 --> 00:05:19,080
就是我转为静态图之后

123
00:05:19,240 --> 00:05:20,560
没有相关的Path

124
00:05:20,560 --> 00:05:22,200
没有整个编译流程

125
00:05:22,200 --> 00:05:24,880
那这些处理量化融合算子这些呢

126
00:05:24,880 --> 00:05:26,120
只能交给用户

127
00:05:26,120 --> 00:05:28,680
让用户有感知性的去做一些替换

128
00:05:28,680 --> 00:05:30,520
没有一个标准的流程

129
00:05:30,520 --> 00:05:31,360
例如量化

130
00:05:31,360 --> 00:05:33,560
我可能有量化的一个Path去处理

131
00:05:33,560 --> 00:05:34,480
这些都没有

132
00:05:34,480 --> 00:05:36,640
但是它有一个标准的参考翻译

133
00:05:37,400 --> 00:05:39,480
现在已经讲完Torch FX

134
00:05:39,480 --> 00:05:42,000
也回顾了Torch FX的一个pros and cons

135
00:05:42,000 --> 00:05:45,280
接下来看看LazyTensor

136
00:05:45,280 --> 00:05:46,440
那LazyTensor这个呢

137
00:05:46,440 --> 00:05:47,240
还是很有意思的

138
00:05:47,240 --> 00:05:48,520
往下瞅瞅

139
00:05:48,520 --> 00:05:49,480
首先第一点

140
00:05:49,480 --> 00:05:50,560
还是来看看

141
00:05:50,560 --> 00:05:52,800
LazyTensor的设计原则

142
00:05:52,800 --> 00:05:53,840
Principle

143
00:05:53,840 --> 00:05:56,000
可以看到PyTorch动态图里面的

144
00:05:56,000 --> 00:05:56,680
主要的操作

145
00:05:56,880 --> 00:05:57,920
就是写了算子之后

146
00:05:58,160 --> 00:05:59,360
就把算子Dispatch

147
00:05:59,360 --> 00:06:01,320
就是下发到Kernel里面

148
00:06:01,320 --> 00:06:03,960
让Kernel去执行到硬件上面

149
00:06:04,640 --> 00:06:06,960
所以说在AI框架层

150
00:06:06,960 --> 00:06:09,080
在上层是有算子这个概念的

151
00:06:09,080 --> 00:06:10,280
但是在硬件层面

152
00:06:10,600 --> 00:06:11,480
硬件看到的

153
00:06:11,480 --> 00:06:12,720
全都是可执行的一些

154
00:06:12,720 --> 00:06:14,080
内核库或者内核代码

155
00:06:14,320 --> 00:06:15,320
所以可以分开

156
00:06:15,920 --> 00:06:17,840
硬件的人可能不太会懂

157
00:06:17,840 --> 00:06:19,880
你这个什么算子

158
00:06:19,880 --> 00:06:21,360
但是算子这个概念

159
00:06:21,360 --> 00:06:23,080
是AI框架引起的

160
00:06:23,520 --> 00:06:24,760
第二点就是

161
00:06:24,760 --> 00:06:26,920
这些下发到硬件上面去执行的

162
00:06:26,920 --> 00:06:28,280
Kernel主要执行的时候

163
00:06:28,280 --> 00:06:29,440
都是一步的

164
00:06:29,840 --> 00:06:30,960
像这种处理方式

165
00:06:31,120 --> 00:06:32,080
在一些大规模的

166
00:06:32,080 --> 00:06:33,320
并行处理硬件上面

167
00:06:33,320 --> 00:06:34,360
是支持的非常好的

168
00:06:34,360 --> 00:06:35,560
例如GPU

169
00:06:36,360 --> 00:06:38,280
了解完PyTorch的一般处理方式

170
00:06:38,400 --> 00:06:40,800
看看Lazy Tensor是怎么做的

171
00:06:41,880 --> 00:06:44,000
左边的这个就是PyTorch的代码

172
00:06:44,800 --> 00:06:45,760
Lazy Tensor的设计

173
00:06:45,960 --> 00:06:48,600
主要是给TPU或者NPU去使用的

174
00:06:48,800 --> 00:06:49,360
可以看到

175
00:06:49,360 --> 00:06:50,760
它把PyTorch的代码

176
00:06:50,960 --> 00:06:52,520
直接交给Torch SLA

177
00:06:52,520 --> 00:06:54,560
做一个异步的缓存和编译

178
00:06:54,840 --> 00:06:57,000
最后把它变成一些子图

179
00:06:57,000 --> 00:06:58,760
进行下发到NPU

180
00:06:59,040 --> 00:07:00,720
或者TPU上面去处理

181
00:07:00,920 --> 00:07:03,320
这个就是Torch FX的一个principle

182
00:07:03,320 --> 00:07:05,280
或者它的一个计算流程

183
00:07:06,120 --> 00:07:07,760
有了对上面一个基础的流程

184
00:07:07,760 --> 00:07:08,320
编译之后

185
00:07:08,440 --> 00:07:10,920
看看了实际上Lazy Tensor

186
00:07:10,920 --> 00:07:13,040
在PyTorch SLA里面

187
00:07:13,200 --> 00:07:15,600
叫做XLA Tensor

188
00:07:16,040 --> 00:07:18,320
这些Tensor都会下发到XLA里面

189
00:07:18,320 --> 00:07:19,360
对它进行一个处理

190
00:07:19,360 --> 00:07:21,080
把它变成一个XLA的Tensor

191
00:07:21,080 --> 00:07:22,640
然后基于XLA的Tensor

192
00:07:23,040 --> 00:07:25,520
又把这些Tensor变成一个子图

193
00:07:25,520 --> 00:07:27,560
例如下面有两个操作

194
00:07:27,560 --> 00:07:28,600
X1 X2

195
00:07:28,600 --> 00:07:29,760
还有一个Y1

196
00:07:29,800 --> 00:07:31,280
会把这三个操作

197
00:07:31,440 --> 00:07:32,800
变成一个子图

198
00:07:32,800 --> 00:07:34,200
上面的demo里面说

199
00:07:34,320 --> 00:07:37,120
把它record into an IR graph

200
00:07:38,240 --> 00:07:39,040
ZOMI老师

201
00:07:39,040 --> 00:07:40,120
我有一个问题

202
00:07:40,120 --> 00:07:42,320
就是我什么时候系统感知到

203
00:07:42,320 --> 00:07:44,280
我需要变成一个子图吗

204
00:07:45,280 --> 00:07:47,480
这位同学的问题问得非常好

205
00:07:48,000 --> 00:07:48,760
这个时候

206
00:07:48,920 --> 00:07:50,600
把它阻塞的情况

207
00:07:50,840 --> 00:07:51,960
叫做barrier

208
00:07:51,960 --> 00:07:53,040
一般的这个barrier

209
00:07:53,040 --> 00:07:55,400
可以手工的去设定

210
00:07:55,400 --> 00:07:58,320
是一个mark Step的API去调用

211
00:07:58,480 --> 00:08:00,120
这个时候就可以告诉

212
00:08:00,120 --> 00:08:01,960
这个时候用户就可以控制

213
00:08:01,960 --> 00:08:04,080
什么时候切分成一个子图

214
00:08:04,440 --> 00:08:05,120
当然了

215
00:08:05,120 --> 00:08:06,600
我没有人工的去试试

216
00:08:06,600 --> 00:08:08,160
mark Step这个API

217
00:08:08,160 --> 00:08:09,880
或调用这个API的时候

218
00:08:09,880 --> 00:08:11,720
系统还会做一些工作

219
00:08:11,720 --> 00:08:14,760
往下看具体的系统做了哪些工作

220
00:08:14,960 --> 00:08:17,720
系统主要会分为三种情况

221
00:08:17,880 --> 00:08:19,840
第一点就是当用户

222
00:08:20,040 --> 00:08:22,800
去调用mark Step这个API的时候

223
00:08:23,120 --> 00:08:24,680
系统就会感知到

224
00:08:24,920 --> 00:08:26,440
这个时候就要阻塞

225
00:08:26,440 --> 00:08:28,000
然后把mark Step之前

226
00:08:28,000 --> 00:08:29,920
缓存下来的一些算子

227
00:08:29,920 --> 00:08:31,040
或者操作

228
00:08:31,440 --> 00:08:32,840
把它变成一个子图

229
00:08:32,840 --> 00:08:34,120
这是第一种方式

230
00:08:34,600 --> 00:08:37,120
第二种方式就是XLA里面

231
00:08:37,320 --> 00:08:40,560
它其实也有自己HALO的一些定义

232
00:08:40,560 --> 00:08:42,240
关于算子的一些定义

233
00:08:42,280 --> 00:08:43,400
但是Pytorch里面

234
00:08:43,520 --> 00:08:45,720
有2000多的一个操作

235
00:08:45,720 --> 00:08:46,800
还有那个算子

236
00:08:46,840 --> 00:08:48,280
当有一些高级的算子

237
00:08:48,280 --> 00:08:48,840
没有办法

238
00:08:48,840 --> 00:08:52,040
印试到XLA编译器里面的算子的时候

239
00:08:52,320 --> 00:08:54,640
它就会做一个子图的拆分

240
00:08:54,760 --> 00:08:55,680
第三种情况

241
00:08:55,880 --> 00:08:57,840
就是在计算图系列里面

242
00:08:57,840 --> 00:08:58,920
经常去谈到的

243
00:08:58,920 --> 00:09:01,000
当我遇到控制流的时候怎么办

244
00:09:01,280 --> 00:09:03,320
当我遇到一些复杂的操作的时候

245
00:09:03,320 --> 00:09:03,760
怎么办

246
00:09:03,960 --> 00:09:05,880
这个时候XLA也会帮

247
00:09:05,880 --> 00:09:07,840
把它切分成一个子图

248
00:09:09,000 --> 00:09:11,000
了解完一些基础的概念之后

249
00:09:11,080 --> 00:09:13,800
现在看看一个整体的自行流程

250
00:09:14,080 --> 00:09:16,040
右边在Devices上面

251
00:09:16,040 --> 00:09:18,400
以谷歌的TPU作为例子

252
00:09:18,400 --> 00:09:21,440
host就是指CPU的处理方式

253
00:09:21,440 --> 00:09:22,560
在我调度的时候

254
00:09:22,680 --> 00:09:25,000
我一般都会在CPU上面去做调度

255
00:09:25,000 --> 00:09:26,280
真正去执行的时候

256
00:09:26,280 --> 00:09:27,320
就把一些kernel

257
00:09:27,320 --> 00:09:29,320
然后下发到Devices里面

258
00:09:29,320 --> 00:09:32,080
让加速器做一个加速的

259
00:09:32,080 --> 00:09:34,440
现在从头捋一捋

260
00:09:34,440 --> 00:09:37,200
首先第一个就是LazyTensor的操作

261
00:09:37,640 --> 00:09:39,000
让系统记录下来

262
00:09:39,240 --> 00:09:41,000
这个有一个输入

263
00:09:41,000 --> 00:09:43,240
输入到三个不同的卷积上面

264
00:09:43,240 --> 00:09:44,440
最后concate在一起

265
00:09:44,440 --> 00:09:46,360
但是后面concate完之后

266
00:09:46,520 --> 00:09:47,640
遇到一个if

267
00:09:47,640 --> 00:09:49,320
就是控制流

268
00:09:49,320 --> 00:09:51,040
这个时候LazyTensor这个特性

269
00:09:51,160 --> 00:09:52,440
就感觉感知到了

270
00:09:52,800 --> 00:09:54,120
我现在需要切图了

271
00:09:54,160 --> 00:09:55,400
因为我遇到if else了

272
00:09:55,400 --> 00:09:57,880
所以我就把这个图贴起来

273
00:09:57,880 --> 00:09:59,360
既然把图一切

274
00:09:59,520 --> 00:10:01,000
这个时候上面这个图

275
00:10:01,000 --> 00:10:02,440
就可以组成一个subgraph

276
00:10:02,440 --> 00:10:04,200
然后下发给SLA

277
00:10:04,200 --> 00:10:05,800
做一个编译和执行

278
00:10:06,000 --> 00:10:07,000
编译执行完之后

279
00:10:07,160 --> 00:10:08,640
我获得到一个结果

280
00:10:08,880 --> 00:10:11,080
这个结果就返回给host测

281
00:10:11,080 --> 00:10:13,840
host测就要继续往下去执行

282
00:10:13,840 --> 00:10:15,040
就继续往下执行

283
00:10:15,040 --> 00:10:16,600
要重新回到这里面

284
00:10:16,600 --> 00:10:18,640
然后去记录一些算子

285
00:10:18,640 --> 00:10:19,640
然后再下发

286
00:10:19,640 --> 00:10:20,800
就是把所有东西

287
00:10:20,800 --> 00:10:22,760
都变成一个子图子图的去下发

288
00:10:22,760 --> 00:10:24,960
虽然没有整图进行一个下层

289
00:10:24,960 --> 00:10:26,640
到硬件上面去执行

290
00:10:26,640 --> 00:10:29,920
但是把整图拆分成好多子图

291
00:10:29,920 --> 00:10:31,400
通过子图去下发

292
00:10:31,400 --> 00:10:34,000
而不是每一次只下发一个算子

293
00:10:34,000 --> 00:10:36,440
下发一个kernel去执行去回调

294
00:10:36,440 --> 00:10:38,200
这个时候就可以充分的利用了

295
00:10:38,200 --> 00:10:40,360
三方硬件加速器的一些功能

296
00:10:42,360 --> 00:10:43,600
最后还是一样

297
00:10:43,600 --> 00:10:45,760
去看看LazyTensor的pros and cons

298
00:10:45,760 --> 00:10:47,480
那pros就是LazyTensor

299
00:10:47,480 --> 00:10:50,840
能够一定程度带来一些优化的收益

300
00:10:51,120 --> 00:10:53,400
第二个点就是语法的限制少

301
00:10:53,400 --> 00:10:56,160
理论上只要找到合适的tensor的操作

302
00:10:56,160 --> 00:10:58,720
都可以把它序列化变成子图

303
00:10:58,720 --> 00:11:02,400
但是cons就是它有一个jit的编译开销

304
00:11:02,400 --> 00:11:05,200
遇到特殊的网络会重复的调度

305
00:11:05,200 --> 00:11:06,240
那遇到特殊网络

306
00:11:06,240 --> 00:11:08,000
就是我有大量的if-else的时候

307
00:11:08,000 --> 00:11:09,920
我不是重复不断的去调度吗

308
00:11:09,920 --> 00:11:11,760
这个时候就可能每一次变化

309
00:11:11,760 --> 00:11:13,360
都会触发到新的编译

310
00:11:13,360 --> 00:11:14,800
导致性能的退化

311
00:11:14,800 --> 00:11:16,800
例如遇到动态shape的时候

312
00:11:16,800 --> 00:11:18,200
可能每一次都要编译

313
00:11:18,240 --> 00:11:21,120
那这时候就会导致大量的时间的浪费

314
00:11:21,120 --> 00:11:22,440
性能就会慢了

315
00:11:22,440 --> 00:11:25,640
第三个因为fusion kernel的执行的原子性

316
00:11:25,640 --> 00:11:27,800
破坏了操作的并行的可能性

317
00:11:27,800 --> 00:11:29,640
这一点你简单听这句话

318
00:11:29,800 --> 00:11:31,800
可能还真不明白

319
00:11:31,800 --> 00:11:35,200
所以后面还有一小部分内容去介绍的

320
00:11:35,200 --> 00:11:37,040
第三个就是影响了PyTorch

321
00:11:37,040 --> 00:11:40,240
在ego模式kernel下发的一个异步计算了

322
00:11:40,240 --> 00:11:41,960
这个时候也会增加开销

323
00:11:41,960 --> 00:11:43,960
后面的第三第四条

324
00:11:43,960 --> 00:11:45,520
现在来看看

325
00:11:45,960 --> 00:11:48,320
首先像现在这种操作

326
00:11:48,440 --> 00:11:49,400
我异步下发的时候

327
00:11:49,400 --> 00:11:51,000
其实不是下发完

328
00:11:51,000 --> 00:11:52,920
一层一的卷机再下发

329
00:11:52,920 --> 00:11:54,040
三层三的卷机

330
00:11:54,040 --> 00:11:55,680
再下发五层五的卷机

331
00:11:55,680 --> 00:11:57,720
再下发三层三的卷机

332
00:11:57,720 --> 00:11:59,840
每一次都要等GPU去执行完

333
00:11:59,840 --> 00:12:01,560
一层一再执行三层三

334
00:12:01,560 --> 00:12:02,960
这样是效率很慢的

335
00:12:02,960 --> 00:12:05,400
所以PyTorch在后台下发的机制的时候

336
00:12:05,760 --> 00:12:07,760
它是并行的去下发

337
00:12:07,760 --> 00:12:10,320
一层一三层三五层五三层三

338
00:12:10,320 --> 00:12:12,320
都同时下发到GPU里面

339
00:12:12,320 --> 00:12:13,720
去做一个执行运算

340
00:12:13,720 --> 00:12:15,200
然后再connect到一起的

341
00:12:15,200 --> 00:12:17,080
这种就是PyTorch下发的时候的

342
00:12:17,080 --> 00:12:18,400
一个并行的操作

343
00:12:18,400 --> 00:12:19,520
变成纸图之后

344
00:12:19,520 --> 00:12:21,880
想下发给TPU或者NPU去执行

345
00:12:21,880 --> 00:12:23,120
我怎么能确保里面

346
00:12:23,120 --> 00:12:24,560
就是高度并行的操作

347
00:12:24,680 --> 00:12:26,360
这个是没有办法去控制的

348
00:12:26,360 --> 00:12:28,520
所以说这种方式一定程度上

349
00:12:28,520 --> 00:12:31,480
破坏了可以并行操作的一个可能性

350
00:12:31,800 --> 00:12:34,200
还有一点就是例如红色的这条线

351
00:12:34,200 --> 00:12:36,880
实际上这条线的传输的时候是异步的

352
00:12:36,880 --> 00:12:38,480
但是我切成纸图的时候

353
00:12:38,640 --> 00:12:40,520
这一步操作就不能异步了

354
00:12:40,520 --> 00:12:41,440
因为异步的时候

355
00:12:41,440 --> 00:12:44,200
可以host跟devices同时计算的

356
00:12:44,200 --> 00:12:46,760
这个时候可以利用CPU和GPU的能力

357
00:12:46,760 --> 00:12:48,480
但是如果把它切成纸图

358
00:12:48,640 --> 00:12:50,720
有可能这条线就断了

359
00:12:50,720 --> 00:12:52,120
然后实行加速的时候

360
00:12:52,320 --> 00:12:54,840
是在TPU里面做一个串行的加速

361
00:12:55,040 --> 00:12:57,880
这种加速到底有没有做异步的快

362
00:12:58,120 --> 00:12:59,200
这个不一定

363
00:12:59,200 --> 00:13:00,680
需要去具体情况

364
00:13:00,680 --> 00:13:01,880
具体打开去测

365
00:13:03,000 --> 00:13:03,520
好了

366
00:13:03,520 --> 00:13:05,480
看完LazyTensor的pros and Coins之后

367
00:13:05,760 --> 00:13:07,720
可能还要回顾一下

368
00:13:07,720 --> 00:13:09,520
之前讲了TorchScript

369
00:13:09,520 --> 00:13:10,360
TorchJIT

370
00:13:10,360 --> 00:13:11,200
Torch FX

371
00:13:11,200 --> 00:13:12,400
LazyTensor

372
00:13:12,440 --> 00:13:15,000
后面会真正来到TorchDynamo里面

373
00:13:15,000 --> 00:13:16,880
正式进入到TorchDynamo之前

374
00:13:17,160 --> 00:13:19,240
会横向的去比较一下

375
00:13:19,240 --> 00:13:21,920
这些特性具体有什么缺点

376
00:13:21,920 --> 00:13:22,720
有什么优点

377
00:13:22,720 --> 00:13:24,600
它的适用场景在哪里

378
00:13:24,600 --> 00:13:27,120
它的技术的演进方式是怎么样的

379
00:13:27,120 --> 00:13:29,040
下期再来分析

380
00:13:29,040 --> 00:13:29,560
好了

381
00:13:29,560 --> 00:13:30,320
谢谢各位

382
00:13:30,320 --> 00:13:31,600
拜拜

383
00:13:32,160 --> 00:13:33,000
卷的不行了

384
00:13:33,000 --> 00:13:33,840
卷的不行了

385
00:13:33,840 --> 00:13:35,280
记得一键三连加关注

386
00:13:35,640 --> 00:13:38,880
所有的内容都会开源在下面这条链接里面

387
00:13:39,240 --> 00:13:40,200
拜拜