Skip to content

bioailab/Aux-Net-Archive

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
Code
Code
 
 
data
data
 
 
results
results
 
 
.gitignore
.gitignore
 
 
LICENSE
LICENSE
 
 
README.md
README.md
 
 

Repository files navigation

See the author's git repository here: https://github.com/Rohit102497/Aux-Net

Auxiliary Network: Scalable and Agile Online Learning for Dynamic System with Inconsistently Available Inputs Link

Citation

Please consider citing the below paper, if you are using the code provided in this repository.

@inproceedings{agarwal2022auxiliary,
  title={Auxiliary Network: Scalable and agile online learning for dynamic system with inconsistently available inputs},
  author={Agarwal, Rohit and Agarwal, Krishna and Horsch, Alexander and Prasad, Dilip K},
  booktitle={International Conference on Neural Information Processing},
  pages={549--561},
  year={2022},
  organization={Springer}
}

Overview

This repository contains datasets and implementation codes of different models for the paper, titled "Auxiliary Network: Scalable and Agile Online Learning for Dynamic System with Inconsistently Available Inputs".

Datasets

The link of all the datasets can be found below. The italy power demand (ipd) datasets in its corresponding respective folder inside data/ directory. To run them, please download the datsets files form the given link below and place them inside their respective directories (see instructions for each dataset below...).

Small Datsets

Medium Datsets

Large Datsets

Raw Data Transformation

Some of the data sets need to be cleaned and processesed before they can be used in the models for inference. Details of how to process those datasets are given below.

  • Spamassasin

    • Download the files form the link provided and unzip them.
    • Use the sctipt Code\DataCode\DataPreparation\data_spamassasin_conversion.py to clean the data.
    • Modify the 'path' variable at line 13 to the path of the directory where the unzipped files are located.
    • The data will automatically be saved in the appropriate directory.

  • IMDB

    • Download the files form the link provided and unzip them.
    • Use the sctipt Code\DataCode\DataPreparation\data_imdb_conversion.py to clean the data.
    • Modify the 'data_path' variable at line 10 to the path of the directory where the unzipped files are located.
    • The data will automatically be saved in the appropriate directory.

  • Diabetes_us

Dataset Preparation

Variable P

For synthetic datasets, we varied the availability of each auxiliary input feature independently by a uniform distribution of probability $p$, i.e., each auxilairy feature is available for $100p%$. For more information about this, follow paper - Aux-Net (https://link.springer.com/chapter/10.1007/978-3-031-30105-6_46)

Files

To run the models, see Code/main.py. After running a model on a certain dataset, run Code/read_results.py to display and save the evaluation in csv format.

Control Parameters

For main.py file,

  1. seed : Seed value
    default = 2023

  2. type: The type of the experiment. If type is basefeatures then please provide the number of base feature using --nbasefeat. If type is noassumption then please provide either the imputation or dummy feature details. default="noassumption", type=str,
    choices = ["noassumption", "basefeatures"]

Data Variables

  1. dataname: The name of the dataset
    default = "wpbc"
    choices = ["all", "synthetic", "crowdsense_c5", "crowdsense_c3" "spamassasin", "imdb", "diabetes_us", "higgs", "susy", "a8a" "magic04", "spambase", "krvskp", "svmguide3", "ipd", "german" "diabetes_f", "wbc", "australian", "wdbc", "ionosphere", "wpbc"]

  2. syndatatype: The type to create suitable synthetic dataset
    default = "variable_p"

  3. probavailable: The probability of each feature being available to create synthetic data
    default = 0.5, type = float

Base feature argument

  1. nbasefeat: The number of base feature. This code considers the first nbasefeat as the base features. default = 0, type = int

Method Variables

  1. ifimputation: If some features needs to be imputed
    default = False

  2. imputationtype: The type of imputation technique to create base features
    default = 'forwardfill'
    choices = ['forwardfill', 'forwardmean', 'zerofill']

  3. nimputefeat: The number of imputation features
    default = 2

  4. ifdummyfeat: If some dummy features needs to be created
    default = False

  5. dummytype: The type of technique to create dummy base features
    default = 'standardnormal'

  6. ndummyfeat: The number of dummy features to create'
    default = 1

  7. nruns: The number of times a method should runs (For navie Bayes, it would be 1 because it is a deterministic method)
    default = 5

For read_results.py file,

  1. type: The type of the experiment
    default ="noassumption"
    choices = ["noassumption", "basefeatures"]

  2. dataname: The name of the dataset
    default = "wpbc"
    choices = ["synthetic", "real", "crowdsense_c5", "crowdsense_c3", "spamassasin", "imdb", "diabetes_us", "higgs", "susy", "a8a", "magic04", "spambase", "krvskp", "svmguide3", "ipd", "german", "diabetes_f", "wbc", "australian", "wdbc", "ionosphere", "wpbc"]

  3. probavailable: The probability of each feature being available to create synthetic data
    default = 0.5

Dependencies

  1. numpy
  2. torch
  3. pandas
  4. random
  5. tqdm
  6. os
  7. pickle

Running the code

To run the models, change the control parameters accordingly in the main.py file and run

python Code/main.py
  1. To run the code by the assigning the first 2 features as base features, do
python Code/main.py --type basefeatures --dataname ipd --nbasefeat 2
  1. To run the code without any assumption (as these models were modified from their original implementation to support the absence of (previously required) base-feature), we have two options:
    • Imputing some features and consider them as base feature 
      python Code/main.py --type noassumption --dataname ipd --ifimputation True --imputationtype forwardfill --nimputefeat 2
    • Making dummy features and consider them as base feature 
      python Code/main.py --type noassumption --dataname ipd --ifdummyfeat True --dummytype standardnormal --ndummyfeat 2

To read the results and save them in .csv format, run read_results.py with appropriate control parameters.

python Code/read_results.py --type basefeatures --dataname ipd

About

Auxiliary Network to handle dimension varying inputs in an online setting

github.com/Rohit102497/Aux-Net

Resources

Readme

License

GPL-3.0 license
Activity
Custom properties

Stars

0 stars

Watchers

2 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages