-
Notifications
You must be signed in to change notification settings - Fork 29
/
profiler.html
153 lines (134 loc) · 8.6 KB
/
profiler.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Profiling for Varuna — Varuna documentation</title>
<link rel="stylesheet" href="_static/alabaster.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: './',
VERSION: '',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true,
SOURCELINK_SUFFIX: '.txt'
};
</script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="Morphing" href="morphing.html" />
<link rel="prev" title="The Varuna class" href="varuna.html" />
<link rel="stylesheet" href="_static/custom.css" type="text/css" />
<meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
</head>
<body>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<div class="section" id="profiling-for-varuna">
<h1>Profiling for Varuna<a class="headerlink" href="#profiling-for-varuna" title="Permalink to this headline">¶</a></h1>
<p>The Varuna <code class="xref py py-class docutils literal"><span class="pre">Profiler</span></code> provides an easy interface for users to profile
the compute and communication operations of a model. This processes the model cutpoints in parallel
and captures the time and memory consumption for each cutpoint. This profile can then be used to calculate
various parameters for Varuna - ideal pipeline and data-parallel dimensions for a given number of GPUs and
suitable microbatch sizes for different configs.</p>
<dl class="class">
<dt id="varuna.Profiler">
<em class="property">class </em><code class="descclassname">varuna.</code><code class="descname">Profiler</code><span class="sig-paren">(</span><em>model</em>, <em>get_batch</em>, <em>device=-1</em>, <em>gpus_per_node=None</em>, <em>fp16=False</em>, <em>out_folder='profiles'</em>, <em>pstages_to_profile=None</em>, <em>from_cache=True</em>, <em>add_to_existing=False</em><span class="sig-paren">)</span><a class="headerlink" href="#varuna.Profiler" title="Permalink to this definition">¶</a></dt>
<dd><p>Module for varuna profiling. Similar to <code class="docutils literal"><span class="pre">Varuna</span></code> class, the model must be
wrapped in an instance of <code class="docutils literal"><span class="pre">Profiler</span></code> before optimizer creation and the
<code class="xref py py-attr docutils literal"><span class="pre">model</span></code> passed should be on CPU.</p>
<p>Varuna profiling runs in a distributed process and the <code class="docutils literal"><span class="pre">Profiler</span></code> should
be used by each worker. Each worker profiles compute for the different <a href="#id1"><span class="problematic" id="id2">``</span></a>CutPoint``s
in the model while simultaneously measuring communication links between workers.
The profiler should be used in three steps:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">get_batch</span><span class="p">(</span><span class="n">size</span><span class="p">):</span>
<span class="c1"># function to get sample batches of given size for profiling</span>
<span class="k">return</span> <span class="n">batch</span>
<span class="n">profiler</span> <span class="o">=</span> <span class="n">Profiler</span><span class="p">(</span><span class="n">model</span><span class="p">,</span> <span class="n">get_batch_fn</span><span class="p">,</span> <span class="n">fp16</span><span class="o">=</span><span class="n">args</span><span class="o">.</span><span class="n">fp16</span><span class="p">,</span> <span class="n">device</span> <span class="o">=</span> <span class="n">args</span><span class="o">.</span><span class="n">local_rank</span><span class="p">,</span>
<span class="n">from_cache</span><span class="o">=</span><span class="bp">True</span><span class="p">,</span> <span class="n">out_folder</span><span class="o">=</span><span class="n">args</span><span class="o">.</span><span class="n">save</span><span class="p">)</span>
<span class="n">profile</span> <span class="o">=</span> <span class="n">profiler</span><span class="o">.</span><span class="n">profile_all</span><span class="p">(</span><span class="n">microbatch_sizes_to_profile</span><span class="p">)</span>
</pre></div>
</div>
<table class="docutils field-list" frame="void" rules="none">
<col class="field-name" />
<col class="field-body" />
<tbody valign="top">
<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first last simple">
<li><strong>model</strong> (<em>torch.nn.Module</em>) – The model to profile.</li>
<li><strong>get_batch_fn</strong> (<em>function</em><em>(</em><em>size</em><em>, </em><em>device='cpu'</em><em>)</em>) – Function to get batch of a given size, used for different sizes by the profiler</li>
<li><strong>device</strong> (<em>int</em>) – index of the cuda device to use. Recommended to be the same as local_rank,
which is the default if not specified.</li>
<li><strong>fp16</strong> (<em>bool</em>) – whether to enable mixed precision training.</li>
<li><strong>from_cache</strong> (<em>bool</em>) – Whether to use cached information anout model structure, if available.</li>
<li><strong>out_folder</strong> (<em>string</em><em> or </em><em>PathLike object</em>) – Path to folder for saving compute and communication profiles</li>
<li><strong>pstages_to_profile</strong> – List of indices of cutpoints to profile, by default this contains all cutpoints</li>
</ul>
</td>
</tr>
</tbody>
</table>
<p>:type list or None
:param add_to_existing: Whether to continue profiling by adding to cutpoint profiles already saved in out_folder
:type add_To_existing: bool</p>
<dl class="method">
<dt id="varuna.Profiler.set_optimizer">
<code class="descname">set_optimizer</code><span class="sig-paren">(</span><em>optimizer</em>, <em>amp_opt_level='O2'</em>, <em>loss_scale='dynamic'</em>, <em>init_loss_scale=1048576</em>, <em>min_loss_scale=None</em><span class="sig-paren">)</span><a class="headerlink" href="#varuna.Profiler.set_optimizer" title="Permalink to this definition">¶</a></dt>
<dd></dd></dl>
<dl class="method">
<dt id="varuna.Profiler.profile_all">
<code class="descname">profile_all</code><span class="sig-paren">(</span><em>microbatch_sizes</em><span class="sig-paren">)</span><a class="headerlink" href="#varuna.Profiler.profile_all" title="Permalink to this definition">¶</a></dt>
<dd></dd></dl>
</dd></dl>
</div>
</div>
</div>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper"><div class="relations">
<h3>Related Topics</h3>
<ul>
<li><a href="index.html">Documentation overview</a><ul>
<li>Previous: <a href="varuna.html" title="previous chapter">The Varuna class</a></li>
<li>Next: <a href="morphing.html" title="next chapter">Morphing</a></li>
</ul></li>
</ul>
</div>
<div role="note" aria-label="source link">
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="_sources/profiler.rst.txt"
rel="nofollow">Show Source</a></li>
</ul>
</div>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
<form class="search" action="search.html" method="get">
<div><input type="text" name="q" /></div>
<div><input type="submit" value="Go" /></div>
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="footer">
©2021, Nitika Saran.
|
Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.7</a>
& <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.8</a>
|
<a href="_sources/profiler.rst.txt"
rel="nofollow">Page source</a>
</div>
</body>
</html>