examples/notebooks/imbalanced_learn.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "# Imbalanced Learn with DeepMol\n",
    "\n",
    "Imbalanced data is a common problem in machine learning and deep learning, including in the field of chemoinformatics. Imbalanced data refers to datasets where the number of instances of one class is significantly larger or smaller than the number of instances of other classes. For example, in chemoinformatics, there may be datasets where the number of active compounds is much smaller than the number of inactive compounds.\n",
    "\n",
    "Imbalanced data can lead to biased and suboptimal models, as traditional machine learning algorithms may not be able to learn the minority class effectively. This is where imbalanced learning techniques, such as those implemented in the imbalanced-learn library, can be important.\n",
    "\n",
    "DeepMol provides various methods for handling imbalanced data, including oversampling, undersampling, and combination methods. These techniques can help to balance the data and improve the performance of the machine learning models, especially in chemoinformatics where imbalanced data is common.\n",
    "\n",
    "For example, in a chemoinformatics dataset where there are many inactive compounds and few active compounds, oversampling techniques such as SMOTE (Synthetic Minority Over-sampling Technique) can be used to generate synthetic examples of the minority class, improving the model's ability to learn from the minority class. On the other hand, undersampling techniques can be used to reduce the number of majority class samples, making the dataset more balanced."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Let's create a dataset with imbalanced labels"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2023-05-31 17:16:00,140 — INFO — Assuming classification since there are less than 10 unique y values. If otherwise, explicitly set the mode to 'regression'!\n",
      "2023-05-31 17:16:00,140 — INFO — Assuming classification since there are less than 10 unique y values. If otherwise, explicitly set the mode to 'regression'!\n",
      "2023-05-31 17:16:00,140 — INFO — Assuming classification since there are less than 10 unique y values. If otherwise, explicitly set the mode to 'regression'!\n"
     ]
    },
    {
     "data": {
      "text/plain": "Counter({1: 100, 0: 1000})"
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from deepmol.compound_featurization import TwoDimensionDescriptors\n",
    "from collections import Counter\n",
    "from deepmol.datasets import SmilesDataset\n",
    "\n",
    "import pandas as pd\n",
    "\n",
    "df = pd.read_csv('../data/CHEMBL217_reduced.csv', header=0)\n",
    "# pick 100 cases where 'Activity_Flag' (label) is 1 and 1000 cases where 'Activity_Flag' is 0\n",
    "# select 100 cases where the label is 1\n",
    "cases_1 = df[df['Activity_Flag'] == 1].head(100)\n",
    "# select 1000 cases where the label is 0\n",
    "cases_0 = df[df['Activity_Flag'] == 0].head(1000)\n",
    "\n",
    "unbalanced_data = pd.concat([cases_1, cases_0])\n",
    "\n",
    "data = SmilesDataset(smiles=unbalanced_data.SMILES,\n",
    "                     ids=unbalanced_data.Original_Entry_ID,\n",
    "                     y=unbalanced_data.Activity_Flag,\n",
    "                     label_names=['Activity_Flag'])\n",
    "TwoDimensionDescriptors().featurize(data, inplace=True)\n",
    "\n",
    "\n",
    "# count y values in dataset.y\n",
    "Counter(data.y)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-05-31T16:16:01.808825141Z",
     "start_time": "2023-05-31T16:15:59.972325670Z"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Over Sampling Methods"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### RandomOverSampler\n",
    "\n",
    "The RandomOverSampler is a technique used to address the problem of imbalanced data in machine learning. It is a data augmentation technique that creates synthetic samples of the minority class by randomly duplicating existing samples until the number of samples in the minority class matches the number of samples in the majority class."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### Sampling strategies\n",
    "Sampling information to resample the data set.\n",
    "When float, it corresponds to the desired ratio of the number of samples in the minority class over\n",
    "the number of samples in the majority class after resampling.\n",
    "When str, specify the class targeted by the resampling. The number of samples in the different classes\n",
    "will be equalized. Possible choices are:\n",
    "   - 'minority': resample only the minority class;\n",
    "   - 'not minority': resample all classes but the minority class;\n",
    "   - 'not majority': resample all classes but the majority class;\n",
    "   - 'all': resample all classes;\n",
    "   - 'auto': equivalent to 'not majority'.\n",
    "\n",
    "When dict, the keys correspond to the targeted classes. The values correspond to the desired number of\n",
    "samples for each targeted class.\n",
    "When callable, function taking y and returns a dict. The keys correspond to the targeted classes.\n",
    "The values correspond to the desired number of samples for each class."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "outputs": [
    {
     "data": {
      "text/plain": "Counter({1: 750, 0: 1000})"
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from copy import deepcopy\n",
    "from deepmol.imbalanced_learn import RandomOverSampler\n",
    "\n",
    "d1 = deepcopy(data)\n",
    "sampler = RandomOverSampler(sampling_strategy=0.75, random_state=123)\n",
    "d1 = sampler.sample(d1)\n",
    "\n",
    "Counter(d1.y)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-05-31T16:16:02.731522378Z",
     "start_time": "2023-05-31T16:16:02.689990804Z"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### SMOTE\n",
    "\n",
    "SMOTE (Synthetic Minority Over-sampling Technique) is another data augmentation technique used to address the problem of imbalanced data in machine learning. SMOTE is similar to the RandomOverSampler, but instead of randomly duplicating minority class samples, it creates synthetic samples by interpolating between pairs of minority class samples."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### Sampling strategies\n",
    "Sampling information to resample the data set.\n",
    "When float, it corresponds to the desired ratio of the number of samples in the minority class over\n",
    "the number of samples in the majority class after resampling.\n",
    "When str, specify the class targeted by the resampling. The number of samples in the different classes\n",
    "will be equalized. Possible choices are:\n",
    "   - 'minority': resample only the minority class;\n",
    "   - 'not minority': resample all classes but the minority class;\n",
    "   - 'not majority': resample all classes but the majority class;\n",
    "   - 'all': resample all classes;\n",
    "   - 'auto': equivalent to 'not majority'.\n",
    "\n",
    "When dict, the keys correspond to the targeted classes. The values correspond to the desired number of\n",
    "samples for each targeted class.\n",
    "When callable, function taking y and returns a dict. The keys correspond to the targeted classes.\n",
    "The values correspond to the desired number of samples for each class.\n"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/joao/miniconda3/envs/deepmol/lib/python3.8/site-packages/imblearn/over_sampling/_smote/base.py:336: FutureWarning: The parameter `n_jobs` has been deprecated in 0.10 and will be removed in 0.12. You can pass an nearest neighbors estimator where `n_jobs` is already set instead.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "data": {
      "text/plain": "Counter({1: 800, 0: 1000})"
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from deepmol.imbalanced_learn import SMOTE\n",
    "\n",
    "d2 = deepcopy(data)\n",
    "sampler = SMOTE(sampling_strategy=0.8, random_state=123, k_neighbors=5, n_jobs=-1)\n",
    "d2 = sampler.sample(d2)\n",
    "\n",
    "Counter(d2.y)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-05-31T16:31:31.574894106Z",
     "start_time": "2023-05-31T16:31:31.529347351Z"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Under Sampling Methods"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### RandomUnderSampler\n",
    "\n",
    "The RandomUnderSampler is a technique used to address the problem of imbalanced data in machine learning. It is a data reduction technique that reduces the number of samples in the majority class by randomly removing samples until the number of samples in the majority class matches the number of samples in the minority class."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### Sampling strategies\n",
    "\n",
    "Sampling information to resample the data set.\n",
    "When float, it corresponds to the desired ratio of the number of samples in the minority class over\n",
    "the number of samples in the majority class after resampling.\n",
    "When str, specify the class targeted by the resampling. The number of samples in the different classes\n",
    "will be equalized. Possible choices are:\n",
    "\n",
    "   - 'minority': resample only the minority class;\n",
    "   - 'not minority': resample all classes but the minority class;\n",
    "   - 'not majority': resample all classes but the majority class;\n",
    "   - 'all': resample all classes;\n",
    "   - 'auto': equivalent to 'not majority'.\n",
    "\n",
    "When dict, the keys correspond to the targeted classes. The values correspond to the desired number of\n",
    "samples for each targeted class.\n",
    "When callable, function taking y and returns a dict. The keys correspond to the targeted classes.\n",
    "The values correspond to the desired number of samples for each class.\n"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "outputs": [
    {
     "data": {
      "text/plain": "Counter({0: 200, 1: 100})"
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from deepmol.imbalanced_learn import RandomUnderSampler\n",
    "\n",
    "d3 = deepcopy(data)\n",
    "sampler = RandomUnderSampler(sampling_strategy=0.5, random_state=123, replacement=True)\n",
    "d3 = sampler.sample(d3)\n",
    "\n",
    "Counter(d3.y)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-05-31T16:31:35.855529172Z",
     "start_time": "2023-05-31T16:31:35.825971343Z"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### ClusterCentroids\n",
    "\n",
    "ClusterCentroids is a technique used to address the problem of imbalanced data in machine learning. It is a data undersampling technique that creates synthetic samples of the majority class by clustering the majority class data and then generating centroids for each cluster. These centroids are then used as representative samples for the majority class."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### Sampling strategies\n",
    "Sampling information to resample the data set.\n",
    "When float, it corresponds to the desired ratio of the number of samples in the minority class over\n",
    "the number of samples in the majority class after resampling.\n",
    "When str, specify the class targeted by the resampling. The number of samples in the different classes\n",
    "will be equalized. Possible choices are:\n",
    "   - 'minority': resample only the minority class;\n",
    "   - 'not minority': resample all classes but the minority class;\n",
    "   - 'not majority': resample all classes but the majority class;\n",
    "   - 'all': resample all classes;\n",
    "   - 'auto': equivalent to 'not majority'.\n",
    "\n",
    "When dict, the keys correspond to the targeted classes. The values correspond to the desired number of\n",
    "samples for each targeted class.\n",
    "When callable, function taking y and returns a dict. The keys correspond to the targeted classes.\n",
    "The values correspond to the desired number of samples for each class."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "outputs": [
    {
     "data": {
      "text/plain": "Counter({0: 100, 1: 100})"
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.cluster import KMeans\n",
    "from deepmol.imbalanced_learn import ClusterCentroids\n",
    "\n",
    "d4 = deepcopy(data)\n",
    "sampler = ClusterCentroids(sampling_strategy=1, random_state=123, estimator=KMeans(), voting='hard')\n",
    "d4 = sampler.sample(d4)\n",
    "\n",
    "Counter(d4.y)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-05-31T16:31:41.777388622Z",
     "start_time": "2023-05-31T16:31:39.526270051Z"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Combination of Under and Over Sampling"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### SMOTEENN\n",
    "\n",
    "SMOTEENN is a hybrid technique that combines two other techniques, SMOTE (Synthetic Minority Over-sampling Technique) and Edited Nearest Neighbors (ENN), to address the problem of imbalanced data in machine learning. SMOTE is used to oversample the minority class by creating synthetic samples, while ENN is used to undersample the majority class by removing samples that are misclassified by a k-NN classifier."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### Sampling strategies\n",
    "\n",
    "Sampling information to resample the data set.\n",
    "When float, it corresponds to the desired ratio of the number of samples in the minority class over\n",
    "the number of samples in the majority class after resampling.\n",
    "When str, specify the class targeted by the resampling. The number of samples in the different classes\n",
    "will be equalized. Possible choices are:\n",
    "\n",
    "   - 'minority': resample only the minority class;\n",
    "   - 'not minority': resample all classes but the minority class;\n",
    "   - 'not majority': resample all classes but the majority class;\n",
    "   - 'all': resample all classes;\n",
    "   - 'auto': equivalent to 'not majority'.\n",
    "\n",
    "When dict, the keys correspond to the targeted classes. The values correspond to the desired number of\n",
    "samples for each targeted class.\n",
    "When callable, function taking y and returns a dict. The keys correspond to the targeted classes.\n",
    "The values correspond to the desired number of samples for each class."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/joao/miniconda3/envs/deepmol/lib/python3.8/site-packages/imblearn/over_sampling/_smote/base.py:336: FutureWarning: The parameter `n_jobs` has been deprecated in 0.10 and will be removed in 0.12. You can pass an nearest neighbors estimator where `n_jobs` is already set instead.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "data": {
      "text/plain": "Counter({0: 922, 1: 772})"
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from deepmol.imbalanced_learn import SMOTEENN\n",
    "\n",
    "d5 = deepcopy(data)\n",
    "sampler = SMOTEENN(sampling_strategy=0.8, random_state=123, n_jobs=-1)\n",
    "d5 = sampler.sample(d5)\n",
    "\n",
    "Counter(d5.y)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-05-31T16:34:05.207568897Z",
     "start_time": "2023-05-31T16:34:04.646389609Z"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### SMOTETomek\n",
    "\n",
    "SMOTETomek is a hybrid technique that combines two other techniques, SMOTE (Synthetic Minority Over-sampling Technique) and Tomek Links, to address the problem of imbalanced data in machine learning. SMOTE is used to oversample the minority class by creating synthetic samples, while Tomek Links is used to undersample the majority class by identifying and removing samples that are close to the boundary between the minority and majority classes."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### Sampling strategies\n",
    "\n",
    "Sampling information to resample the data set.\n",
    "When float, it corresponds to the desired ratio of the number of samples in the minority class over\n",
    "the number of samples in the majority class after resampling.\n",
    "When str, specify the class targeted by the resampling. The number of samples in the different classes\n",
    "will be equalized. Possible choices are:\n",
    "\n",
    "   - 'minority': resample only the minority class;\n",
    "   - 'not minority': resample all classes but the minority class;\n",
    "   - 'not majority': resample all classes but the majority class;\n",
    "   - 'all': resample all classes;\n",
    "   - 'auto': equivalent to 'not majority'.\n",
    "\n",
    "When dict, the keys correspond to the targeted classes. The values correspond to the desired number of\n",
    "samples for each targeted class.\n",
    "When callable, function taking y and returns a dict. The keys correspond to the targeted classes.\n",
    "The values correspond to the desired number of samples for each class."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/joao/miniconda3/envs/deepmol/lib/python3.8/site-packages/imblearn/over_sampling/_smote/base.py:336: FutureWarning: The parameter `n_jobs` has been deprecated in 0.10 and will be removed in 0.12. You can pass an nearest neighbors estimator where `n_jobs` is already set instead.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "data": {
      "text/plain": "Counter({1: 693, 0: 993})"
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from deepmol.imbalanced_learn import SMOTETomek\n",
    "\n",
    "d6 = deepcopy(data)\n",
    "sampler = SMOTETomek(sampling_strategy=0.7, random_state=123, n_jobs=-1)\n",
    "d6 = sampler.sample(d6)\n",
    "\n",
    "Counter(d6.y)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-05-31T16:44:01.385143115Z",
     "start_time": "2023-05-31T16:44:00.994718418Z"
    }
   }
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}