06.srt

1
00:00:00,000 --> 00:00:04,070
字幕生成: BLACK 字幕校对: 杨绎

2
00:00:05,220 --> 00:00:07,400
哈喽大家好,我是ZOMI

3
00:00:07,400 --> 00:00:13,000
今天我给大家去讲讲PyTorch里面一个不算太新,但是也不算太旧

4
00:00:13,000 --> 00:00:17,400
可是这个特性非常重要的就是Dispatch的机制

5
00:00:17,400 --> 00:00:19,100
那为什么会讲这个机制呢?

6
00:00:19,100 --> 00:00:22,600
是因为在讲到PyTorch 2.0新特性的时候

7
00:00:22,600 --> 00:00:24,600
会有一个AoT Off-Gate

8
00:00:24,600 --> 00:00:27,500
那AoT就是Ahead-of-Time Auto-Gate

9
00:00:27,500 --> 00:00:29,300
就自动为分这个功能

10
00:00:29,300 --> 00:00:31,300
而这个功能的具体实现呢

11
00:00:31,300 --> 00:00:33,100
在上一节分享里面呢

12
00:00:33,100 --> 00:00:36,200
其实给大家已经安利过

13
00:00:36,200 --> 00:00:36,700
只是呢

14
00:00:36,700 --> 00:00:40,300
它会严重的去依赖于Torch Dispatch这个机制

15
00:00:40,300 --> 00:00:43,200
所以今天来看看Torch Dispatch机制的原理

16
00:00:44,500 --> 00:00:47,800
那在正式进入到Torch Dispatch机制这个原理呢

17
00:00:47,800 --> 00:00:49,700
会分开几个内容去介绍

18
00:00:49,700 --> 00:00:51,900
首先什么是Dispatch

19
00:00:51,900 --> 00:00:53,800
Dispatch到底有什么不同

20
00:00:53,800 --> 00:00:54,400
那接着呢

21
00:00:54,400 --> 00:00:57,000
去看看为什么需要Dispatch

22
00:00:57,000 --> 00:00:58,100
Dispatch的中文呢

23
00:00:58,100 --> 00:00:59,300
等一下会讲

24
00:00:59,300 --> 00:01:01,400
那为什么需要Dispatch之后呢

25
00:01:01,400 --> 00:01:04,800
就会真正的去了解到了Dispatch的注册

26
00:01:04,800 --> 00:01:07,000
和分发的一个具体的内容

27
00:01:07,000 --> 00:01:08,700
或者它的一个Concept

28
00:01:08,700 --> 00:01:09,600
有了这个之后呢

29
00:01:09,600 --> 00:01:14,800
去看看PyTorch Dispatch里面的一个Key的表示和计算

30
00:01:14,800 --> 00:01:18,200
最后的去了解一下PyTorch Dispatch Table

31
00:01:18,200 --> 00:01:21,400
怎么去对一些新的算子进行注册

32
00:01:21,400 --> 00:01:23,800
或者新的功能进行注册的

33
00:01:23,800 --> 00:01:26,000
了解完PyTorch Dispatch机制之后呢

34
00:01:26,000 --> 00:01:29,300
就回头去看看Torch AOT Auto grad

35
00:01:29,300 --> 00:01:31,800
这个特性的具体的实现原理的时候呢

36
00:01:31,800 --> 00:01:34,600
可能就会有一个更加清晰的了解了

37
00:01:36,100 --> 00:01:39,100
现在来了解一下什么是Dispatch

38
00:01:39,100 --> 00:01:40,400
那Dispatch的中文呢

39
00:01:40,400 --> 00:01:42,300
主要是派遣分发的意思

40
00:01:42,300 --> 00:01:45,900
简单的去把它当做分发的机制就好了

41
00:01:47,000 --> 00:01:47,500
假设呢

42
00:01:47,500 --> 00:01:48,900
现在有一个团队

43
00:01:48,900 --> 00:01:49,500
这个团队呢

44
00:01:49,500 --> 00:01:50,700
有四个人去组成

45
00:01:50,700 --> 00:01:52,300
一个是PM

46
00:01:52,300 --> 00:01:55,700
另外的话是三个很惨的程序员

47
00:01:55,700 --> 00:01:56,400
那这个时候呢

48
00:01:56,400 --> 00:01:57,600
甲方的爸爸

49
00:01:57,600 --> 00:01:59,400
不断的去给这个项目

50
00:01:59,400 --> 00:02:02,100
这四个人组成的项目提需求

51
00:02:02,100 --> 00:02:03,500
对这个甲方呢

52
00:02:03,500 --> 00:02:04,600
叫做Crazy甲方

53
00:02:04,600 --> 00:02:07,400
针对Crazy甲方给这个项目团队

54
00:02:07,400 --> 00:02:09,500
提出各种各样的需求

55
00:02:09,500 --> 00:02:10,400
这个时候呢

56
00:02:10,400 --> 00:02:13,400
项目经理就会根据Crazy甲方的需求

57
00:02:13,400 --> 00:02:14,400
进行拆分

58
00:02:14,400 --> 00:02:17,200
匹配到每个程序员适合

59
00:02:17,200 --> 00:02:19,000
擅长他们做的事情

60
00:02:19,000 --> 00:02:20,200
那这个项目经理呢

61
00:02:20,200 --> 00:02:21,600
叫做Dispatch

62
00:02:21,600 --> 00:02:25,600
他所做的工作就是一个简单的工作的分发

63
00:02:26,600 --> 00:02:28,000
那通过这个概念呢

64
00:02:28,000 --> 00:02:29,500
简单的去了解了

65
00:02:29,500 --> 00:02:31,100
什么是Dispatch

66
00:02:31,100 --> 00:02:31,800
那接着呢

67
00:02:31,800 --> 00:02:34,800
去看看为什么需要Dispatch

68
00:02:36,800 --> 00:02:38,600
从刚才的一个例子来看呢

69
00:02:38,600 --> 00:02:39,500
其实Dispatch呢

70
00:02:39,500 --> 00:02:41,400
更多的是一个if-else的工作

71
00:02:41,400 --> 00:02:42,800
如果你适合干这个活

72
00:02:42,800 --> 00:02:44,400
那我就把这个活分给你

73
00:02:44,400 --> 00:02:47,500
就是如果没有一个很好的Dispatch分发器的话

74
00:02:47,500 --> 00:02:49,600
会写很多if-else的代码

75
00:02:49,600 --> 00:02:51,600
如果input的contest等于GPU

76
00:02:51,600 --> 00:02:53,500
如果input的cast等于CPU

77
00:02:53,500 --> 00:02:55,100
或者等于不同的硬件的时候

78
00:02:55,100 --> 00:02:56,400
就会怎么做

79
00:02:56,400 --> 00:02:57,200
那这个时候呢

80
00:02:57,200 --> 00:02:57,900
没有Dispatch

81
00:02:57,900 --> 00:03:00,100
会写大量的if-else

82
00:03:00,100 --> 00:03:00,900
而Dispatch呢

83
00:03:00,900 --> 00:03:04,400
可以很好的帮去管理一些分发的工作

84
00:03:04,400 --> 00:03:05,500
那可以看一下

85
00:03:05,500 --> 00:03:07,000
其实关于上下文

86
00:03:07,000 --> 00:03:09,500
或者关于整个AI编辑器里面呢

87
00:03:09,500 --> 00:03:10,300
关于Tensor

88
00:03:10,300 --> 00:03:13,600
就有非常多不同的内容

89
00:03:13,600 --> 00:03:14,200
那第一个呢

90
00:03:14,200 --> 00:03:15,200
就是Devices

91
00:03:15,200 --> 00:03:16,800
针对一个具体的算子

92
00:03:16,800 --> 00:03:19,500
或者针对一个具体的Tensor的操作

93
00:03:19,500 --> 00:03:21,900
有非常多不同的设备

94
00:03:22,000 --> 00:03:23,500
有CPU、GPU、NPU、TPU

95
00:03:23,500 --> 00:03:26,400
还有FPGA等不同的硬件

96
00:03:26,400 --> 00:03:27,100
那这个时候呢

97
00:03:27,100 --> 00:03:29,300
我需要根据上下文去决定

98
00:03:29,300 --> 00:03:31,600
我这个算子跑在哪个硬件上面

99
00:03:31,600 --> 00:03:32,200
这里面呢

100
00:03:32,200 --> 00:03:34,100
就有了一个分发的机制

101
00:03:34,100 --> 00:03:35,700
分发的工作

102
00:03:35,700 --> 00:03:38,500
如果没有了这个Dispatch的分发的机制之后呢

103
00:03:38,500 --> 00:03:41,100
会写大量的代码去解决这些问题

104
00:03:41,100 --> 00:03:41,700
那代码呢

105
00:03:41,700 --> 00:03:43,300
就会非常的冗余

106
00:03:43,300 --> 00:03:45,600
而且维护起来也非常困难

107
00:03:45,600 --> 00:03:46,700
针对Tensor

108
00:03:46,700 --> 00:03:48,400
还有非常多的内容

109
00:03:48,400 --> 00:03:50,100
就Tensor它有很多的layout

110
00:03:50,100 --> 00:03:52,300
有很多不同的形式和结构

111
00:03:52,300 --> 00:03:53,900
那可以有普通的张量

112
00:03:53,900 --> 00:03:55,100
有稀疏的张量

113
00:03:55,100 --> 00:03:56,600
而且不同的张量

114
00:03:56,600 --> 00:03:57,700
有不同的布局

115
00:03:57,700 --> 00:04:00,700
有NHWC、有NCHW

116
00:04:00,700 --> 00:04:01,300
另外的话

117
00:04:01,300 --> 00:04:02,900
举第三个例子

118
00:04:02,900 --> 00:04:04,700
就是Data Type

119
00:04:04,700 --> 00:04:06,800
张量的数据类型

120
00:04:06,800 --> 00:04:08,500
平时在Tabularly上面

121
00:04:08,500 --> 00:04:10,700
去用的一些数据类型

122
00:04:10,700 --> 00:04:11,600
已经比较固定了

123
00:04:11,600 --> 00:04:13,400
有普通的Float、Long Float

124
00:04:13,400 --> 00:04:14,300
还有Short Float

125
00:04:14,300 --> 00:04:15,300
还有Longed In

126
00:04:15,300 --> 00:04:16,400
还有Shorted In

127
00:04:16,400 --> 00:04:18,000
很多不同的类型

128
00:04:18,100 --> 00:04:20,000
而AI深度学习诞生之后

129
00:04:20,000 --> 00:04:21,700
会出现的更多的

130
00:04:21,700 --> 00:04:23,000
不同的数据类型

131
00:04:23,000 --> 00:04:24,900
有BF16、HF32

132
00:04:24,900 --> 00:04:26,900
跟不同的类型

133
00:04:26,900 --> 00:04:28,500
如果我要去写大量的

134
00:04:28,500 --> 00:04:29,800
If Else的内容

135
00:04:29,800 --> 00:04:31,300
或者Switch Case的内容

136
00:04:31,300 --> 00:04:33,000
就会写非常大量

137
00:04:33,000 --> 00:04:34,500
冗余的代码

138
00:04:34,500 --> 00:04:35,800
那这个时候

139
00:04:35,800 --> 00:04:38,400
就确实需要一个Dispatcher

140
00:04:38,400 --> 00:04:40,900
让它统一帮去调度

141
00:04:40,900 --> 00:04:42,600
管理分配的工作

142
00:04:44,400 --> 00:04:45,800
在Pytorch里面

143
00:04:45,900 --> 00:04:47,300
主要是采用一个

144
00:04:47,300 --> 00:04:49,000
比较常见的架构

145
00:04:49,000 --> 00:04:51,600
那会做一个注册和分发

146
00:04:51,600 --> 00:04:53,300
但是对于这些

147
00:04:53,300 --> 00:04:55,200
苦逼的程序员来说

148
00:04:55,200 --> 00:04:56,700
就需要考虑到

149
00:04:56,700 --> 00:04:58,200
具体的设计模式

150
00:04:58,200 --> 00:04:59,700
用哪一种方式

151
00:04:59,700 --> 00:05:01,800
那实际上注册分发这种机制

152
00:05:01,800 --> 00:05:03,300
在设计模式里面

153
00:05:03,300 --> 00:05:05,500
会使用到注册器的模式

154
00:05:05,500 --> 00:05:07,000
还有工厂的模式

155
00:05:07,000 --> 00:05:09,600
去具体的实现这个功能

156
00:05:10,600 --> 00:05:12,700
下面对着具体的代码

157
00:05:12,700 --> 00:05:14,800
来看看什么是具体的

158
00:05:14,800 --> 00:05:16,600
Dispatcher的注册和分发

159
00:05:16,600 --> 00:05:18,800
还有责任的下放

160
00:05:18,800 --> 00:05:19,900
那下面也add

161
00:05:19,900 --> 00:05:21,100
这个算子作为例子

162
00:05:21,100 --> 00:05:22,500
那在上层

163
00:05:22,500 --> 00:05:25,000
或者在Pytorch的API层

164
00:05:25,000 --> 00:05:26,600
大部分都是Python的代码

165
00:05:26,600 --> 00:05:29,100
那传进去的是两个Tensor

166
00:05:29,100 --> 00:05:31,900
然后进行一个累加的操作

167
00:05:31,900 --> 00:05:33,800
关于加的复数的表示

168
00:05:33,800 --> 00:05:35,700
其实有两种操作

169
00:05:35,700 --> 00:05:38,000
一种是实部和虚部的表示

170
00:05:38,000 --> 00:05:39,200
Rectangular

171
00:05:39,200 --> 00:05:41,900
一种就是模和浮角的表示

172
00:05:41,900 --> 00:05:43,300
叫做Polar

173
00:05:43,300 --> 00:05:45,200
如果要实现一个系统

174
00:05:45,200 --> 00:05:46,300
或者实现一个算子

175
00:05:46,300 --> 00:05:49,400
同时支持这两种形式的加法的运算

176
00:05:49,400 --> 00:05:50,900
那这个时候应该怎么做呢

177
00:05:50,900 --> 00:05:52,700
那首先由这一段代码

178
00:05:52,700 --> 00:05:56,300
会把一个接口实现定义好

179
00:05:56,300 --> 00:05:57,800
一个叫做Rectangular

180
00:05:57,800 --> 00:05:59,200
一个叫做Polar

181
00:05:59,200 --> 00:06:01,400
然后去通过if else

182
00:06:01,400 --> 00:06:04,500
去判断输进去的Tensor的一个类型

183
00:06:04,500 --> 00:06:05,800
它到底是哪种

184
00:06:05,800 --> 00:06:09,500
选择具体的它对应的实现的方式

185
00:06:09,500 --> 00:06:11,400
或者它实现的方法

186
00:06:11,400 --> 00:06:13,500
而采用了注册分发的机制

187
00:06:13,500 --> 00:06:16,200
实际上会保存一张表

188
00:06:16,200 --> 00:06:18,600
这张表叫做Vtable

189
00:06:18,600 --> 00:06:21,600
这个跟C++里面的一个虚拟表

190
00:06:21,600 --> 00:06:23,500
是类似相同的概念

191
00:06:23,500 --> 00:06:25,200
那这个时候会有一个key

192
00:06:25,200 --> 00:06:26,100
有个value

193
00:06:26,100 --> 00:06:27,700
那么可以看到key里面

194
00:06:27,700 --> 00:06:29,100
刚才的add操作

195
00:06:29,100 --> 00:06:30,900
有Rectangular有Polar

196
00:06:30,900 --> 00:06:32,700
然后具体的value

197
00:06:32,700 --> 00:06:34,100
就是对应的函数

198
00:06:34,100 --> 00:06:35,100
拿到具体的key

199
00:06:35,100 --> 00:06:38,400
就可以调用具体的一个函数的value

200
00:06:38,400 --> 00:06:40,600
有两个对应的具体的函数的实现

201
00:06:40,600 --> 00:06:42,300
实际上存的不是一个函数

202
00:06:42,300 --> 00:06:44,000
而实际上存的是地址

203
00:06:44,000 --> 00:06:45,600
每次有新方法的时候

204
00:06:45,600 --> 00:06:47,200
就通过register

205
00:06:47,200 --> 00:06:49,200
去注册到刚才的那个表

206
00:06:49,200 --> 00:06:51,000
然后具体的接口的时候

207
00:06:51,000 --> 00:06:52,100
就会通过get

208
00:06:52,100 --> 00:06:55,100
来去获取对应的实现函数的地址

209
00:06:55,100 --> 00:06:57,600
有一个add的算子的操作

210
00:06:57,600 --> 00:07:00,600
数同样是z1跟z2

211
00:07:00,600 --> 00:07:03,200
最重要的是第18行代码

212
00:07:03,200 --> 00:07:04,800
通过get z1的tag

213
00:07:04,800 --> 00:07:07,300
而z1的tag就是对应的key

214
00:07:07,300 --> 00:07:08,600
通过get的方式

215
00:07:08,600 --> 00:07:11,700
去获取对应的要运行的函数

216
00:07:12,900 --> 00:07:13,900
那这种方式

217
00:07:13,900 --> 00:07:16,000
就是实际上注册和分发

218
00:07:16,000 --> 00:07:18,900
具体的实现的原理和过程了

219
00:07:18,900 --> 00:07:20,700
更多的会模仿

220
00:07:20,700 --> 00:07:23,200
C++里面的Virtual Table

221
00:07:23,200 --> 00:07:24,600
就是它的虚拟表

222
00:07:24,600 --> 00:07:26,500
PyTorch里面的Vtable

223
00:07:26,500 --> 00:07:28,100
首先具体的看看

224
00:07:28,100 --> 00:07:29,800
就是C++的Vtable

225
00:07:29,800 --> 00:07:32,300
它是每一个类都有一个Vtable

226
00:07:32,300 --> 00:07:33,800
而且只有dist

227
00:07:33,800 --> 00:07:35,800
指针指向Vtable

228
00:07:35,800 --> 00:07:37,600
而PyTorch里面的Vtable

229
00:07:37,600 --> 00:07:39,800
就是每一个算子

230
00:07:39,800 --> 00:07:42,200
都会维护一个自己的Vtable

231
00:07:42,200 --> 00:07:44,300
而且不仅仅要考虑到

232
00:07:44,300 --> 00:07:45,800
Tensor相关的信息

233
00:07:45,800 --> 00:07:48,000
还要考虑到很多的上下文

234
00:07:48,000 --> 00:07:50,200
就刚才提到的data type的信息

235
00:07:50,200 --> 00:07:52,100
还有不同硬件的信息

236
00:07:52,100 --> 00:07:54,500
而且在PyTorch的一个Vtable

237
00:07:54,500 --> 00:07:56,700
基本上只会扩展

238
00:07:56,700 --> 00:07:59,300
对应的算子的操作

239
00:07:59,300 --> 00:08:00,800
而不会像C++这样

240
00:08:00,800 --> 00:08:04,300
根据每个类去提供对应的Vtable

241
00:08:04,300 --> 00:08:05,300
所以这里面很重要

242
00:08:05,300 --> 00:08:07,400
因为在AI框架里面

243
00:08:07,400 --> 00:08:09,300
更关注的是一个计算

244
00:08:09,300 --> 00:08:11,700
而计算的逻辑是用户提供的

245
00:08:11,700 --> 00:08:13,200
就是用户的脚本告诉我

246
00:08:13,200 --> 00:08:15,900
我应该怎么去算这个深度学习的模型

247
00:08:15,900 --> 00:08:17,300
而PyTorch的Vtable

248
00:08:17,300 --> 00:08:19,300
或者PyTorch的Dispatch机制

249
00:08:19,300 --> 00:08:22,100
更多的是做一个具体的分发的工作

250
00:08:23,600 --> 00:08:25,100
那现在来看看

251
00:08:25,100 --> 00:08:28,100
实际的Dispatch的key是怎么操作的

252
00:08:28,100 --> 00:08:30,200
假设现在去执行

253
00:08:30,200 --> 00:08:33,600
一个Torch Add的一个算子的操作的时候

254
00:08:33,600 --> 00:08:36,100
就执行简单一个加法的时候

255
00:08:36,200 --> 00:08:38,600
Dispatch首先会找到这个算子

256
00:08:38,600 --> 00:08:41,300
就是这个加对应的Dispatch Key

257
00:08:41,300 --> 00:08:43,600
就刚才的那个Vtable

258
00:08:43,600 --> 00:08:46,100
然后根据这个Dispatch的key

259
00:08:46,100 --> 00:08:48,900
去找到对应的kernel的函数

260
00:08:48,900 --> 00:08:51,400
就对应实现真正实现的函数

261
00:08:51,400 --> 00:08:53,100
而在PyTorch里面

262
00:08:53,100 --> 00:08:54,600
这个对应的kernel的函数

263
00:08:54,600 --> 00:08:56,200
是Primp IR

264
00:08:56,200 --> 00:08:57,600
或者aten IR

265
00:08:57,600 --> 00:08:59,200
对应的算子的实现了

266
00:09:02,400 --> 00:09:03,800
那在这里面

267
00:09:04,200 --> 00:09:07,100
非常推荐大家去看一下

268
00:09:07,100 --> 00:09:09,100
这一个网站就是

269
00:09:09,100 --> 00:09:12,200
easyyang的一个blog

270
00:09:12,200 --> 00:09:14,400
它里面就分享了非常多

271
00:09:14,400 --> 00:09:16,100
关于PyTorch Dispatch

272
00:09:16,100 --> 00:09:19,700
还有PyTorch原生的一些具体的原理

273
00:09:19,700 --> 00:09:20,600
那在这里面

274
00:09:20,600 --> 00:09:22,600
我对它截了一张图

275
00:09:22,600 --> 00:09:24,200
右边的这个就是

276
00:09:24,200 --> 00:09:26,800
C++的一个Vtable的具体的实现

277
00:09:26,800 --> 00:09:28,400
而Vtable里面的key

278
00:09:28,400 --> 00:09:30,900
就对应它的一个Disk的指针

279
00:09:30,900 --> 00:09:32,500
PyTorch这里面的实现

280
00:09:32,500 --> 00:09:35,200
更多的是对Tensor的执行

281
00:09:35,200 --> 00:09:36,600
去做一个Dispatch的

282
00:09:36,600 --> 00:09:39,100
所以它大部分针对的是一个Tensor

283
00:09:39,100 --> 00:09:39,900
那可以看到

284
00:09:39,900 --> 00:09:41,100
Dispatch的key

285
00:09:41,100 --> 00:09:43,500
主要是由一系列的byte set来做

286
00:09:43,500 --> 00:09:44,300
而这个byte

287
00:09:44,300 --> 00:09:45,100
一般来说

288
00:09:45,100 --> 00:09:47,700
它里面设置了成64位

289
00:09:47,700 --> 00:09:49,800
然后每一个标志位

290
00:09:49,800 --> 00:09:52,600
代表它具体的一个不同的一个内容

291
00:09:52,600 --> 00:09:54,900
这里面有几个内容来去组成的

292
00:09:54,900 --> 00:09:57,500
第一大内容就是Tensor的input

293
00:09:57,500 --> 00:10:00,100
那包括一个CPU CUDA

294
00:10:00,100 --> 00:10:01,800
还有不同的后端

295
00:10:01,800 --> 00:10:04,500
第二个内容就是local include

296
00:10:04,500 --> 00:10:06,600
GPyTorch里面的一些local的功能

297
00:10:06,600 --> 00:10:08,600
当然了它有一些global的功能

298
00:10:08,600 --> 00:10:10,600
包括backend的selection

299
00:10:10,600 --> 00:10:12,000
还有一些autograde

300
00:10:12,000 --> 00:10:13,600
当然了后来autograde

301
00:10:13,600 --> 00:10:16,700
已经移到Tensor的内容里面了

302
00:10:17,600 --> 00:10:19,800
然后把不同的输入输进来之后

303
00:10:19,800 --> 00:10:21,900
通过byte set进行一个concat

304
00:10:21,900 --> 00:10:25,300
然后把它变成具体的一个byte set里面

305
00:10:25,300 --> 00:10:27,500
当然了会做一个local exclude

306
00:10:27,500 --> 00:10:29,700
就把相关的内容把它去掉

307
00:10:29,800 --> 00:10:32,800
那最后真正的到后端执行的时候

308
00:10:32,800 --> 00:10:35,100
就会去执行这个dispatch key

309
00:10:35,100 --> 00:10:37,400
能够留下的一些内容

310
00:10:38,900 --> 00:10:43,200
下面简单的来去看一些相关的概念

311
00:10:43,200 --> 00:10:47,900
首先dispatch并不只是针对后端来去实现调度的

312
00:10:47,900 --> 00:10:51,400
也就是说dispatch它不是一个具体的实现

313
00:10:51,400 --> 00:10:53,900
也不是针对仅仅的硬件

314
00:10:53,900 --> 00:10:56,200
它更多的是对应一个抽象

315
00:10:56,200 --> 00:10:59,500
包括一些PyTorch后端的代码

316
00:10:59,500 --> 00:11:02,000
包括它的tracing、JIT还有autogrid

317
00:11:02,000 --> 00:11:03,600
这些函数的功能

318
00:11:04,600 --> 00:11:07,000
第二个就是dispatch的key的计算

319
00:11:07,000 --> 00:11:10,900
刚才讲了它是由一个具体的数据结构来去实现的

320
00:11:10,900 --> 00:11:14,600
简单的理解它为一个64bit的一个数字

321
00:11:14,600 --> 00:11:17,500
每一个bit都代表一个dispatch的key

322
00:11:17,500 --> 00:11:20,600
所以从左到右是有个具体的优先级的

323
00:11:20,600 --> 00:11:24,900
同一个算子可能会有针对不同的dispatch的key的实现

324
00:11:24,900 --> 00:11:27,500
最后在执行或者调度的时候

325
00:11:27,500 --> 00:11:31,600
就会根据最高优先级的dispatch的key对应的kernel

326
00:11:31,600 --> 00:11:33,500
然后去调度执行

327
00:11:33,500 --> 00:11:36,500
这个就是dispatchkey的具体的原理

328
00:11:36,500 --> 00:11:38,100
当然可以理解一下

329
00:11:38,100 --> 00:11:41,600
其实PyTorch它不仅仅只有上层的API

330
00:11:41,600 --> 00:11:44,300
还有底层的kernel对应的算子的实现

331
00:11:44,300 --> 00:11:46,900
实际上它有非常多不同的功能

332
00:11:46,900 --> 00:11:49,400
例如它有autocast

333
00:11:49,400 --> 00:11:51,700
超入一些转块的算子

334
00:11:51,700 --> 00:11:54,700
当然针对不同的后端还有XLA CUDA CPU

335
00:11:54,700 --> 00:11:56,000
有不同的backend

336
00:11:56,000 --> 00:11:59,800
而backend只是占了其中的一小部分

337
00:11:59,800 --> 00:12:01,800
所以说dispatch的机制

338
00:12:01,800 --> 00:12:05,700
不仅仅是对后端的硬件进行一个分发调度

339
00:12:05,700 --> 00:12:09,800
它还会对它内部的一些功能函数进行调度

340
00:12:12,600 --> 00:12:16,100
跟Registeration API的交互的方式有三种

341
00:12:16,100 --> 00:12:18,700
第一种就是定义具体的screenmark

342
00:12:18,700 --> 00:12:20,700
第二个就是实现Registeration

343
00:12:20,700 --> 00:12:22,700
就是实现具体的初始的功能

344
00:12:22,700 --> 00:12:24,700
第三个就是fallback

345
00:12:24,700 --> 00:12:26,700
现在分开三个的方式

346
00:12:26,700 --> 00:12:29,700
看一下具体是怎么做注册的

347
00:12:29,700 --> 00:12:33,200
这里面有算子作为例子

348
00:12:33,200 --> 00:12:35,700
就怎么去注册一个具体的算子

349
00:12:35,700 --> 00:12:37,700
到Register table里面

350
00:12:39,400 --> 00:12:42,500
那么可以看到具体的某个算子的注册

351
00:12:42,500 --> 00:12:44,300
就可以用Torch library

352
00:12:44,300 --> 00:12:45,700
impl

353
00:12:45,700 --> 00:12:48,700
然后去声明用的是哪个算子的IR

354
00:12:48,700 --> 00:12:50,700
然后用的是哪个后端

355
00:12:50,700 --> 00:12:53,900
接着去用m.implement

356
00:12:53,900 --> 00:12:56,900
然后声明具体需要用到的算子

357
00:12:56,900 --> 00:12:58,900
或者注册的算子叫什么名字

358
00:12:58,900 --> 00:13:00,900
跑在哪个后端的函数里面

359
00:13:00,900 --> 00:13:03,900
刚才注册了一个aten的mod

360
00:13:03,900 --> 00:13:05,900
跑在CPU里面

361
00:13:05,900 --> 00:13:08,900
这个就是对单个算子注册在

362
00:13:08,900 --> 00:13:11,900
Dispatcher里面的一个分发器里面

363
00:13:11,900 --> 00:13:13,900
实际上只是一个注册

364
00:13:13,900 --> 00:13:14,900
它还没有去运行的

365
00:13:14,900 --> 00:13:17,900
真正运行的时候是通过Python

366
00:13:17,900 --> 00:13:18,900
执行的时候跑的时候

367
00:13:18,900 --> 00:13:21,900
才去真正的去调到具体的后端

368
00:13:23,900 --> 00:13:24,900
现在来看看

369
00:13:24,900 --> 00:13:25,900
m.define

370
00:13:25,900 --> 00:13:26,900
然后mod-add

371
00:13:26,900 --> 00:13:28,900
这种方式就是定义

372
00:13:28,900 --> 00:13:30,900
具体的算子的一个screen

373
00:13:30,900 --> 00:13:34,900
通过刚才的Vtable来看一下

374
00:13:34,900 --> 00:13:36,900
通过Cache2这个操作

375
00:13:36,900 --> 00:13:38,900
就是把kernel注册到

376
00:13:38,900 --> 00:13:39,900
所有的DispatchKey里

377
00:13:39,900 --> 00:13:42,900
或者通过m.format这个接口

378
00:13:42,900 --> 00:13:44,900
为所有的算子和kernel

379
00:13:44,900 --> 00:13:46,900
提供一个Dispatch的key

380
00:13:48,900 --> 00:13:49,900
当然了刚才讲到的

381
00:13:49,900 --> 00:13:51,900
有三种的方式

382
00:13:51,900 --> 00:13:53,900
去注册到对应的Dispatch的key

383
00:13:53,900 --> 00:13:55,900
对应注册到Vtable里面

384
00:13:55,900 --> 00:13:57,900
但是这三种方式

385
00:13:57,900 --> 00:13:59,900
还是有一个优先级的

386
00:13:59,900 --> 00:14:00,900
Top1的一个优先级

387
00:14:00,900 --> 00:14:02,900
就是针对一个特定的kernel

388
00:14:02,900 --> 00:14:04,900
或者特定的screener的一个实现

389
00:14:04,900 --> 00:14:06,900
那第二种就是Cache2

390
00:14:06,900 --> 00:14:09,900
第三种才是Fallback

391
00:14:12,900 --> 00:14:13,900
作弊老师你好

392
00:14:13,900 --> 00:14:15,900
你讲了关于很多

393
00:14:15,900 --> 00:14:17,900
Dispatch的一些原理

394
00:14:17,900 --> 00:14:19,900
和Dispatch怎么去注册

395
00:14:19,900 --> 00:14:20,900
Dispatch这个机制

396
00:14:20,900 --> 00:14:22,900
除了让去选择

397
00:14:22,900 --> 00:14:25,900
一些不同的后端和函数之外

398
00:14:25,900 --> 00:14:26,900
它有哪些功能吗

399
00:14:26,900 --> 00:14:28,900
或者具体的实现的功能

400
00:14:28,900 --> 00:14:29,900
有哪些吗

401
00:14:30,900 --> 00:14:32,900
这位小新同学

402
00:14:32,900 --> 00:14:33,900
你问的问题非常好

403
00:14:33,900 --> 00:14:35,900
那下面的这个表格

404
00:14:35,900 --> 00:14:36,900
或者下面的内容

405
00:14:36,900 --> 00:14:37,900
就是PyTorch

406
00:14:37,900 --> 00:14:39,900
Develop社区的网站

407
00:14:39,900 --> 00:14:42,900
给提供的一个具体的例子

408
00:14:42,900 --> 00:14:43,900
可以看到

409
00:14:43,900 --> 00:14:45,900
其实可以有很多的内容

410
00:14:45,900 --> 00:14:46,900
例如会做一些

411
00:14:46,900 --> 00:14:48,900
8bit的量化

412
00:14:48,900 --> 00:14:51,900
可以做很多的AOTO的柜

413
00:14:51,900 --> 00:14:53,900
而通过统一的一个调度接口

414
00:14:53,900 --> 00:14:55,900
就是Dispatch的调度器

415
00:14:55,900 --> 00:14:57,900
去做具体的实现

416
00:14:57,900 --> 00:14:58,900
我举一个最大的例子

417
00:14:58,900 --> 00:14:59,900
就是昇腾

418
00:14:59,900 --> 00:15:01,900
作为一个第三方的后端

419
00:15:01,900 --> 00:15:03,900
想对接到PyTorch里面

420
00:15:03,900 --> 00:15:04,900
其实更多的是

421
00:15:04,900 --> 00:15:06,900
用刚才小小讲到的

422
00:15:06,900 --> 00:15:07,900
一个Dispatch的机制

423
00:15:07,900 --> 00:15:09,900
去提供一个新的后端

424
00:15:09,900 --> 00:15:11,900
让PyTorch去执行

425
00:15:11,900 --> 00:15:13,900
这个新的后端的

426
00:15:13,900 --> 00:15:14,900
而PyTorch也是通过

427
00:15:14,900 --> 00:15:15,900
Dispatch的机制

428
00:15:15,900 --> 00:15:16,900
去蓝向的对接到

429
00:15:16,900 --> 00:15:19,900
很多不同的硬件厂商里面

430
00:15:19,900 --> 00:15:21,900
而之前上一节讲到的

431
00:15:21,900 --> 00:15:22,900
AOTO的柜

432
00:15:22,900 --> 00:15:24,900
它作为一个具体的

433
00:15:24,900 --> 00:15:26,900
一个后端的C++的功能

434
00:15:26,900 --> 00:15:28,900
也是通过Dispatch的机制

435
00:15:28,900 --> 00:15:30,900
去做一个调度和分发

436
00:15:33,900 --> 00:15:34,900
好了

437
00:15:34,900 --> 00:15:36,900
今天的内容就到这里为止

438
00:15:36,900 --> 00:15:37,900
谢谢各位

439
00:15:37,900 --> 00:15:38,900
拜了个拜

440
00:15:39,900 --> 00:15:40,900
卷的不行了

441
00:15:40,900 --> 00:15:41,900
卷的不行了

442
00:15:41,900 --> 00:15:42,900
记得一键三连加关注哦

443
00:15:42,900 --> 00:15:44,900
所有的内容都会开源

444
00:15:44,900 --> 00:15:46,900
在下面这条链接里面

445
00:15:46,900 --> 00:15:47,900
拜了个拜