Train task classifyapp on same data as for training the embedding

[dinhi]

Hello together,
we want to train a Keras model with the train_task_classifyapp.py script to make a simple binary classification:

1. class: Applications which perform a stencil operation
2. class: Applications which do not perform a stencil operation

For this purpose we created a dataset based on your synthetic datasets

• Eigen-synthetic https://polybox.ethz.ch/index.php/s/52wWiK5fjRGHLJR
• GEMM-synthetic https://polybox.ethz.ch/index.php/s/Bm6cwAY3eVkR6v3
• Stencil-synthetic https://polybox.ethz.ch/index.php/s/OOmylxGcBxQM1D3

The dataset has the following directory structure so the python script can handle it:

.
├── ncc
│   ├── train
│   │   ├── classifyapp
│   │   │   ├── ir_train
│   │   │   │   ├── 1
│   │   │   │   ├── 2
│   │   │   ├── ir_val
│   │   │   │   ├── 1
│   │   │   │   ├── 2
│   │   │   ├── ir_test
│   │   │   │   ├── 1
│   │   │   │   ├── 2

Folder 2 is a mixture of applications from the Eigen- and GEMM-synthetic dataset, folder 1 has only applications from the Stencil-synthetic dataset.

My questions are the following:

• Since the Eigen-, GEMM- and Stencil-synthetic dataset have been used for training the inst2vec embedding, will this affect the training for classifyapp Keras model (positive or negative way)?
• What was your setup for training and how long did it take? In our current setup, each folder for class 1 and 2 has 80 randomly picked applications, batch size is 4, epochs is 20 and number of training samples per classis 20. We are running the training on a Nvidia 1080 Ti. Only with this setup we could train the network in an affordable time (45 minutes per epoch). We are aware that this can yield in bad accuracy.
  In another setup, we had 2000 sample applications for each class in each set (train, val and test). The batch size was 4, number of training samples per class were 30 and 20 epochs. With this setup, the training time for each epoch went up to 55 hours (Keras ETA)! Using larger batch sizes leads to an error within Cuda since it can not allocate enough memory.
  Do you have any experience with these parameters for training? What could be the reason for such a high training time? In your script, you are using 64 batch size and 1500 training samples for class. Did it also take so much time for training 104 classes?

[tbennun]

1. For the sake of making the training setting more challenging, we tried to completely separate the tasks of training inst2vec from training the downstream tasks. However, for real problems I don't see a point in ensuring such a split. It's ok if you use Stencil-synthetic in your classifyapp sample
2. The slow training time may be related to a bug that was fixed in Classify app: Leak fix and bucketized sampling #16, which isn't merged yet since we need to verify some further results. Please try it and see the performance for your case.

[tbennun]

Closed due to inactivity. Hope I could help!