Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exp matcher #65

Merged
merged 80 commits into from
Jul 26, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
80 commits
Select commit Hold shift + click to select a range
4a6f3ba
Simple BFS Graph Matcher
dboyliao Dec 11, 2018
6daeb73
Add tests (failed)
dboyliao Apr 13, 2019
2d9d193
Fix bug
dboyliao Apr 13, 2019
291f9c7
Syncing develop branch
dboyliao Apr 24, 2019
f15048f
syncing with develop, fix conflicts
dboyliao Apr 30, 2019
725e9d9
Add replace util methods
dboyliao May 1, 2019
ee54a0b
update tests
dboyliao May 1, 2019
53065fd
linear_reorder: wip
neil-tan May 4, 2019
0f802fb
ugraph seems correct, but tf quantization complains
neil-tan May 4, 2019
574f4d7
quantization passed, good until: unsupported op type in uTensor: Quan…
neil-tan May 4, 2019
b046b82
CONV_POOL_Transformer WIP: matcher graph completed
neil-tan May 5, 2019
7870206
fused, but with graph integrity problem
neil-tan May 5, 2019
55fbf77
output graph topologically correct
neil-tan May 5, 2019
eb1a510
attaching op_attribs to the fused-node
neil-tan May 5, 2019
324287c
Merge branch 'f/conv_pool_fusion' into PR69
neil-tan May 7, 2019
d232e02
fix up
neil-tan May 7, 2019
c7687e2
fixing the types
neil-tan May 7, 2019
e66bf54
resolving conflict
neil-tan May 8, 2019
82f5e96
generates some source files, weights file missing
neil-tan May 8, 2019
51641b8
inline transformer enabler hotfix
neil-tan May 8, 2019
428f59c
fixing GatherOpSnippet
neil-tan May 9, 2019
4a2cb98
minor bug fix
dboyliao May 1, 2019
4fc9b14
subgraph view
dboyliao May 6, 2019
1bf33d5
fix bug
dboyliao May 9, 2019
4a07522
merge develope: sync bug fix and resolve conflict
dboyliao May 9, 2019
ad00ee1
null tensor match support
dboyliao May 10, 2019
74a2140
update tests
dboyliao May 11, 2019
654a42f
showing unsupported ops with cli: show subcmd
dboyliao May 11, 2019
4be2c6a
Fix bug and update tests
dboyliao May 19, 2019
403ac64
Merge branch 'develop' into ubit
mbartling May 20, 2019
4aea27e
Add replace method and bugs fixed
dboyliao May 19, 2019
351c061
Add move synmatic to ugraph, op and tensor
dboyliao May 23, 2019
8abc65f
Add first simple replace test
dboyliao May 23, 2019
c250561
fix conflict
dboyliao May 23, 2019
fd525d9
update tests
dboyliao May 23, 2019
c73e9fa
minor refmt
dboyliao May 26, 2019
f0b938e
Fix merge graph bug
dboyliao May 26, 2019
235a529
syncing with develop
dboyliao May 27, 2019
ea059f2
merge ubit and fix conflicts
dboyliao May 27, 2019
c5f8766
linear reorder transformer impl. with matcher
dboyliao May 27, 2019
e5c665e
Add tests for linear reorder transformer
dboyliao May 27, 2019
654b20d
minor fix
dboyliao May 27, 2019
22185fa
remove hardcode graph viz
dboyliao May 31, 2019
5b773e1
Add ConvPool transformer, impl. with matcher api
dboyliao Jun 4, 2019
3736a42
make cli default pipeline using trans impl. with matcher
dboyliao Jun 4, 2019
390bccf
remove incorrect match logic and update tests
dboyliao Jun 5, 2019
5f9c22c
minor fix
dboyliao Jun 5, 2019
6997508
fix bug
dboyliao Jun 7, 2019
89a9c8d
Add tests for convpool transformer (testing against vgg-like graphs)
dboyliao Jun 7, 2019
a0c6ae6
refactor test script a bit
dboyliao Jun 7, 2019
587eadb
minor fix and update Pipfile
dboyliao Jun 8, 2019
0bd9bd8
minor refactor
dboyliao Jun 8, 2019
7aed322
matcher corner case fixed: null tensors
dboyliao Jun 9, 2019
3c03ab0
update README.md
dboyliao Jun 10, 2019
bfb3a12
minor fix and update tests for dropout transformer
dboyliao Jun 10, 2019
7dbd455
clean code
dboyliao Jun 10, 2019
115bfa5
enhance graph viz
dboyliao Jun 10, 2019
7b6502e
Add node fuse image to README.md
dboyliao Jun 10, 2019
ddca306
Add notes to README.md
dboyliao Jun 10, 2019
b797413
update cnn pb file (new dropout api)
dboyliao Jun 11, 2019
fae1129
remove out-dated notebooks
dboyliao Jun 11, 2019
e90771a
Update README and images
dboyliao Jun 11, 2019
2671be1
Add old dropout trans back, update tests and add subcmd
dboyliao Jun 12, 2019
1121d72
minor fix
dboyliao Jun 12, 2019
0e46059
Add lockfile
dboyliao Jun 12, 2019
18a7697
merge develop and resolve conflicts
dboyliao Jun 15, 2019
9d52464
fix bug and make op name pattern as kwarg to dropout tranformer
dboyliao Jun 16, 2019
85c1758
update doc str
dboyliao Jun 16, 2019
3648a27
add flags to show subcmd
dboyliao Jun 20, 2019
5ae22f7
Few bug fixs
dboyliao Jun 23, 2019
8744bb1
Fix bug and update REAME.md
dboyliao Jul 3, 2019
665ecfd
Add explanation to matcher show cases in README.md
dboyliao Jul 7, 2019
f56b40b
Fix typos
dboyliao Jul 7, 2019
40f846f
Fix quant fused conv op snippet bug
dboyliao Jul 11, 2019
9d4153c
remove old experimental matcher and cmsis nn transformer
dboyliao Jul 14, 2019
ca0d37b
update README.md
dboyliao Jul 17, 2019
ea99036
- setup basic doc files
dboyliao Jun 23, 2019
5677dd4
update README
dboyliao Jul 25, 2019
c3af9d3
Fix imgs path
dboyliao Jul 25, 2019
5024724
Add ir chapter
dboyliao Jul 25, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,6 @@ tests/*/cpp
*/*/models
tests/deep_mlp/data
.vscode
# docs
doc/build
doc/source/_*
4 changes: 2 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

tests:
rm -f tests_log.txt
make test_utils test_ir test_transformer test_frontend
make test_utils test_ir test_transformer test_frontend test_matcher

test_%:
@if [ -d .venv ]; then \
Expand All @@ -12,4 +12,4 @@ test_%:
fi;

clean:
rm -f tests_log.txt
rm -f tests_log.txt *.pdf
3 changes: 2 additions & 1 deletion Pipfile
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ verify_ssl = true
name = "pypi"

[packages]
"e1839a8" = {path = ".", editable = true}
e1839a8 = {path = ".",editable = true}

[dev-packages]
pylint = "*"
Expand All @@ -14,3 +14,4 @@ rope = "*"
pillow = "*"
scipy = "*"
graphviz = "*"
sphinx = "*"
901 changes: 901 additions & 0 deletions Pipfile.lock

Large diffs are not rendered by default.

124 changes: 0 additions & 124 deletions README.md

This file was deleted.

235 changes: 235 additions & 0 deletions README.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,235 @@
Installation (Python 2 & 3)
===========================

For Users
---------

- with ``setup.py``

.. code:: console

$ python setup.py install

- with `pip <https://pip.pypa.io/en/stable/>`_

.. code:: console

$ pip install utensor_cgen

For Developers:
---------------

- with `pip <https://pip.pypa.io/en/stable/>`_

.. code:: console

$ pip install -e .[dev]

- with `pipenv <https://github.com/pypa/pipenv>`_

.. code:: console

# install `utensor_cgen` (develop mode)
$ PIPENV_VENV_IN_PROJECT=1 pipenv install -d

# spawn a subshell and activate virtualenv
$ pipenv shell

# get help message of `utensor-cli`
(utenosr_cgen)$ utensor-cli -h

Troubleshooting with ``pipenv``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

- If you have troubles with installation using ``pipenv``, try
``PIPENV_VENV_IN_PROJECT=1 pipenv install -d --skip-lock``
- there is known issue of ``pip`` and ``pipenv``, plz refer to this
`issue <https://github.com/pypa/pipenv/issues/2924>`__ for detail

- short answer: downgrade to ``pip==18.0`` may help :)

- Tensorflow requires ``setuptools<=39.1.0`` (the latest is ``40.4.3``
by the time this README is writen)

- plz downgrade to ``setuptools==39.1.0``
- my recommendation is to use ``virtualenv``

Overall Architecture
====================

::

============ +-----------------+ ===================
|| model file || --> | frontend Parser | --> || uTensorGraph (IR) ||
============ +-----------------+ ===================
|
+-------------------------------+ |
| graph transformer | |
| (legalization & optimization) | <------‘
+-------------------------------+
|
v
===========================
|| uTensorGraph ||
|| (legalized and optimized) ||
===========================
|
+--------------------------+ |
| backend (code generator) | <----‘
+--------------------------+
|
`---> (target files, ex: model.cpp, model.hpp, weights.idx)

Basic Usage
===========

- ``utensor-cli show <model.pb>``

Show all nodes and detailed information of given pb file.

Run ``utensor-cli show --help`` for detailed information.

- ``utensor-cli convert \
--output-nodes=<node_name>[,<node_name>,...] \
<model.pb>``

Convert given pb file into cpp/hpp files.

Note that ``--output-nodes`` is required options. It's the names of
nodes you want to output, seperated by comma if there are many.

In graph theory terminology, they are ``leaf`` nodes of your graph.

Run ``utensor-cli convert --help`` for detailed information.

``utensor_cgen`` as Library
===========================

Subgraph Isomorphic Matcher and Nodes Fusion
--------------------------------------------

With ``uTensorGraphMatcher``, performing common subgraph tasks such as
isomorphic matching along with replacing or manipulating the matched
subgraph(s) takes just a few line of code!

Node Fusion
~~~~~~~~~~~

Note: we'll use operation/node/layer interchangeably in the
documentation

- It's commonly seen pattern in convolution neural network (``CNN``),
``conv -> relu -> pooling``. That is, a 2D convolution followed by a
relu layer and then a pooling down sampling layer.
- With our ``uTensorGraphMatcher``, you can locate such pattern in your
``CNN`` model and fuse/replace matched nodes into one optimized
``QuantizedFusedConv2DMaxpool`` node.

- Left: original graph
- Middle: matched convolution layer
- Right: replace the matched layer with specialized
``QuantizedFusedConv2DMaxpool`` node

\ |conv-pool-fuse|


Dropout Layer Removal
~~~~~~~~~~~~~~~~~~~~~

- Though ``dropout`` is an effective technique to improve training
performance of your model, it's not necessary during inference
phrase.
- In the mainstream frameworks such as ``Tensorflow`` or ``PyTorch``,
an ``dropout`` layer is typically implemented with other elementary
operations/nodes. As a result, finding and removing those nodes for
interence optimization (both in model size and prediciton time) is
not trivial and error prone.
- With our ``uTensorGraphMatcher``, you can find and remove the dropout
nodes as illustrated in the following picture.

- Left: original graph with dropout Layers
- Middle: matched dropout layers
- Right: graph with dropout layers removed

\ |cnn-dropout|


We use mainly ``Tensorflow`` for declaring the pattern graph for matcher now.

High-level graph builder is on its way, see `Future Works <#future-works>`_ for detail.

Examples
========

- `Deep Multilayer
Perceptron <https://github.com/uTensor/utensor_cgen/tree/develop/tests/deep_mlp>`__
- `End-to-End Convolution
NN <https://github.com/uTensor/simple_cnn_tutorial>`__

User Guide
==========

Following steps are a general guild for user how to porting a
``TensorFlow`` protobuf file into a ``uTensor`` implementation:

1. install ``utensor_cgent``

- run ``python3 setupt.py install``

2. run ``utensor-cli convert --output-nodes='NODE,NODE,...' graph.pb``

- run ``utensor-cli -h`` for help
- the ``graph.pb`` is the pb file of *original* graph (not
quantized)

3. If you want to see what ops/nodes are in the pb file, you can run
``utensor-cli show <pbfile>``

Testing
=======

1. follow the steps in `setup <#setup-with-pipenv>`_ section
2. run ``make tests``

- Or you can use ``pipenv run pytest tests`` instead

.. design philosophy
.. `12 Factor CLI App <https://medium.com/@jdxcode/12-factor-cli-apps-dd3c227a0e46?fbclid=IwAR1Gfq0D7oh3b-mXaIMV3RwYu39TAPrPXfz5sBKC4Rz1t-cckvC8WjBVl_w>`_


Future Works
============

1. High-level graph builder api for building `uTensorGraph`.

- Currently ``utensor_cgen`` uses ``TensorFlow`` api for building IR graph, ``uTensorGraph``.
- With high-level graph builder, users can build their ``uTensorGraph`` easily and do not need
to take care of the integrity of the graph.
The builder will take care of it automatically.


.. TODOs
.. =====

.. 1. (done?) core code generator implementation

.. - We need some refactoring, PRs are welcomed!

.. 2. type alias in C/C++

.. - ex: use ``uint8_t`` or ``unsigned char``?
.. - a lot more about this....

.. 3. Relation among snippets/containers

.. - shared template variables? (headers, shared placeholders...etc)

.. 4. Better configuration schema

.. - json
.. - yaml
.. - or ?

.. |cnn-dropout| image:: ./images/cnn_dropout.png
.. |conv-pool-fuse| image:: images/conv_pool_fuse.png
:alt: conv-pool-fuse
Loading