Authors: Manny Ko, Ujjawal K. Panchal, Héctor Andrade Loarca, Andres Mendez-Vazquez

CoShNet Architecture

Architecture

CoShNet is a fully complex hybrid neural network. We use the CoShREM (now call SymFD) http://www.math.uni-bremen.de/cda/software.html signal transform to produce a stable embedding. The network operates entirely in $\mathbb{C}$ domain to take advantage of unique properties of CVNNs.

Architecture Brief

1. Input is any $32\times32 \in \mathbb{R}$ image.
2. Input is CoShREM transformed to produce a $32\times32\times20 \in \mathbb{C}$ output.
3. CoShREM output is convolved with the $2$ cplx-conv layers.

   Each cplx-conv layer is composted of := $\mathbb{C}$-Conv + $\mathbb{C}$-ReLU + $\mathbb{C}$-AvgPool2d.

4. The response is flattened and passed through $2$ cplx-linear layers.

   Each cplx-linear layer is composted of := $\mathbb{C}$-linear layer + $\mathbb{C}$-ReLU.

5. The $\mathbb{R}$, $\mathbb{I}$ components are stacked together (see shape) and passed through $1$ final $\mathbb{R}$-linear layer.

Setup

Python 3.8.x and newer are supported:

Automated Setup

1. Create a virtualenv at the root of the repo: python -m venv venv4coshnet
2. Activate venv4coshnet:
   • Windows: venv4coshnet\Scripts\activate
   • Linux/MacOS: source venv4coshnet/bin/activate
3. Run setup.py:
   • with CUDA: python setup.py
   • without CUDA: python setup.py --no-cuda
   • use --no-venv to disable venv check (e.g. inside a docker)

Manual Setup

• Please follow: setup.txt

Docker Setup

• Build image: docker build -t coshnet-docker:latest . (Some systems might require running this in `sudo` mode.)

Contents

Contents List

1. code/: Contains all code essential to run experiments in this repo.
2. libs/: Contains all custom-made and collected libs and modules we use for our experiments. (Installed automatically in setup.txt)
3. data/: Folder where datasets are present. Created automatically when running for first time.
4. setup.txt: Steps for setting up repo.
5. requirements.txt: requirements file.
6. changelog.md: all changes relevant to releases, branch prs, or any other general notes needed for maintenance.

How to Run?

Running in Local

cd code/. Following are the possible cases:

1. Running our models: run: python test_fashion.py --help to see several arguments you are allowed to tune. (Default run (10k20E) gets 89.2% on RTX 2080 Super). The default will use the 10k test set of Fashion to train for 20 epochs, and the 60k training set to test.
2. Running resnet(18|50): run: python test_resnet.py --help to see several arguments you are allowed to set. (Default run (RN18, 10k20E) gets 88.3% on RTX 2080 Super).

Note: This code (shown in test_fashion.py,test_resnet.py) will not run in (jupyter|google colab) notebook(s). This is because our code defaults to using `asyncio` for batch generation for speed. Hence, if you absolutely have to run in a notebook, please create your own batch generation code.

Running in Docker

• Run Image: docker run coshnet-docker:latest (Some systems might require running this in `sudo` mode.)

Note: The above is a brief demo for running our codebase in a docker. If you want to do something specific, e.g. deliver an API endpoint through a docker, you will have to edit the Dockerfile

Some Results

Model	Epochs	Parameters	Size Ratio	Top-1 Accuracy (60k)	Top-1 Accuracy (10k)
ResNet-18	100	11.18M	229	91.8%	88.3%
ResNet-50	100	23.53M	481	90.7%	87.8%
CoShNet(base)	20	1.37M	28	92.2%	89.2%
CoShNet (tiny)	20	49.99K	1	91.6%	88.0%

Note: 60k = train on train-set (60k observations), test on test-set (10k observations). 10k = vice-versa. K or k = 1000, M = Million.

Cite

@misc{coshnet2022,
  doi = {10.48550/ARXIV.2208.06882},
  url = {https://arxiv.org/abs/2208.06882},
  author = {Ko, Manny and Panchal, Ujjawal K. and Andrade-Loarca, Héctor and Mendez-Vazquez, Andres},
  keywords = {Computer Vision and Pattern Recognition (cs.CV), FOS: Computer and information sciences, FOS: Computer and information sciences},
  title = {CoShNet: A Hybird Complex Valued Neural Network using Shearlets},
  publisher = {arXiv},
  year = {2022},
  copyright = {Creative Commons Attribution Non Commercial Share Alike 4.0 International}
}

License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.