codes/19_efficient-computing.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "📝 **Author:** Amirhossein Heydari - 📧 **Email:** AmirhosseinHeydari78@gmail.com - 📍 **Linktree:** [linktr.ee/mr_pylin](https://linktr.ee/mr_pylin)\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dependencies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# NumPy - Efficient Computing\n",
    "📝 Tutorial:\n",
    "   - Efficient array computing: [enccs.github.io/hpda-python/stack](https://enccs.github.io/hpda-python/stack/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Do Not Be A Purist\n",
    "   - NumPy is designed to perform array operations efficiently\n",
    "   - Optimized algorithms implemented in C\n",
    "   - There's no need to reinvent the wheel."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr_2d_1 = np.array([[1, 4, 2], [3, 1, 0], [2, 4, 2]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sum_1: 19\n"
     ]
    }
   ],
   "source": [
    "# not preferred\n",
    "sum_1 = 0\n",
    "for row in range(arr_2d_1.shape[0]):\n",
    "    for col in range(arr_2d_1.shape[1]):\n",
    "        sum_1 += arr_2d_1[row, col]\n",
    "\n",
    "# log\n",
    "print(f\"sum_1: {sum_1}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sum_2: 19\n"
     ]
    }
   ],
   "source": [
    "# preferred\n",
    "sum_2 = arr_2d_1.sum()\n",
    "\n",
    "# log\n",
    "print(f\"sum_2: {sum_2}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Avoid Loops\n",
    "   - Create a vectorized version of a python function using <np.vectorize>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr_1d_1 = np.array([0, 1, 2, 2, 4, 1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def even_odd(i):\n",
    "    if i % 2:\n",
    "        return 'O'\n",
    "    else:\n",
    "        return 'E'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "arr_1d_2: ['E' 'O' 'E' 'E' 'E' 'O']\n"
     ]
    }
   ],
   "source": [
    "arr_1d_2 = np.zeros_like(arr_1d_1, dtype=str)\n",
    "\n",
    "# not preferred\n",
    "for i in range(len(arr_1d_1)):\n",
    "    arr_1d_2[i] = even_odd(arr_1d_1[i])\n",
    "\n",
    "# log\n",
    "print(f\"arr_1d_2: {arr_1d_2}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "arr_1d_3: ['E' 'O' 'E' 'E' 'E' 'O']\n"
     ]
    }
   ],
   "source": [
    "# preferred\n",
    "even_odd_2 = np.vectorize(even_odd)\n",
    "arr_1d_3 = even_odd_2(arr_1d_1)\n",
    "\n",
    "# log\n",
    "print(f\"arr_1d_3: {arr_1d_3}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Specify Data Types\n",
    "   - Explicitly specify the data type (dtype) of your arrays whenever possible\n",
    "   - It significantly reduce memory usage and improve performance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "arr_1d_4 size: 117 bytes\n",
      "arr_1d_5 size: 122 bytes\n",
      "arr_1d_6 size: 152 bytes\n"
     ]
    }
   ],
   "source": [
    "arr_1d_4 = np.array([1, 2, 3, 4, 5], dtype=np.int8)\n",
    "arr_1d_5 = np.array([1, 2, 3, 4, 5], dtype=np.int16)\n",
    "arr_1d_6 = np.array([1, 2, 3, 4, 5], dtype=np.int64)\n",
    "\n",
    "# log\n",
    "print(f\"arr_1d_4 size: {sys.getsizeof(arr_1d_4)} bytes\")\n",
    "print(f\"arr_1d_5 size: {sys.getsizeof(arr_1d_5)} bytes\")\n",
    "print(f\"arr_1d_6 size: {sys.getsizeof(arr_1d_6)} bytes\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Avoid Unnecessary Copies\n",
    "   - It can consume memory and impact performance\n",
    "   - Instead of creating copies, try to utilize views or slices of existing arrays"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "flatten_1   : [0 2 3 4 5 6 7 8 9]\n",
      "arr_2d_2[0] : 1\n"
     ]
    }
   ],
   "source": [
    "arr_2d_2 = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])\n",
    "\n",
    "# copy\n",
    "flatten_1 = arr_2d_2.flatten()\n",
    "flatten_1[0] = 0\n",
    "\n",
    "# log\n",
    "print(f\"flatten_1   : {flatten_1}\")\n",
    "print(f\"arr_2d_2[0] : {arr_2d_2[0, 0]}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "flatten_2   : [0 2 3 4 5 6 7 8 9]\n",
      "arr_2d_3[0] : 0\n"
     ]
    }
   ],
   "source": [
    "arr_2d_3 = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])\n",
    "\n",
    "# in-place\n",
    "flatten_2 = arr_2d_3.ravel()\n",
    "flatten_2[0] = 0\n",
    "\n",
    "# log\n",
    "print(f\"flatten_2   : {flatten_2}\")\n",
    "print(f\"arr_2d_3[0] : {arr_2d_3[0, 0]}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reuse\n",
    "   - Use previously unnecessary variables instead of creating new ones\n",
    "   - Skip the initialization step when possible"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "arr_1d_7: [0. 1. 2. 3. 4. 5.]\n"
     ]
    }
   ],
   "source": [
    "arr_1d_7 = np.zeros(shape=6)  # not preferred\n",
    "\n",
    "# do some stuff\n",
    "for i in range(6):\n",
    "    arr_1d_7[i] = i\n",
    "\n",
    "# log\n",
    "print(f\"arr_1d_7: {arr_1d_7}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "arr_1d_8: [0. 1. 2. 3. 4. 5.]\n"
     ]
    }
   ],
   "source": [
    "arr_1d_8 = np.empty(shape=6)  # preferred\n",
    "\n",
    "# do some stuff\n",
    "for i in range(6):\n",
    "    arr_1d_8[i] = i\n",
    "\n",
    "# log\n",
    "print(f\"arr_1d_8: {arr_1d_8}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Structured Arrays\n",
    "   - Structured arrays provide efficient storage and manipulation for structured data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "data_1:\n",
      "[['John' '30' '5000']\n",
      " ['Alice' '35' '6000']]\n"
     ]
    }
   ],
   "source": [
    "# not preferred\n",
    "data_1 = np.array([\n",
    "    ['John', 30, 5000],\n",
    "    ['Alice', 35, 6000]\n",
    "])\n",
    "\n",
    "# log\n",
    "print(f\"data_1:\\n{data_1}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "data_2:\n",
      "[{'name': 'John', 'age': 30, 'salary': 5000}\n",
      " {'name': 'Alice', 'age': 35, 'salary': 6000}]\n"
     ]
    }
   ],
   "source": [
    "# not preferred\n",
    "data_2 = np.array([\n",
    "    {\"name\": 'John', \"age\": 30, \"salary\": 5000},\n",
    "    {\"name\": 'Alice', \"age\": 35, \"salary\": 6000},\n",
    "])\n",
    "\n",
    "# log\n",
    "print(f\"data_2:\\n{data_2}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "data_3:\n",
      "[<__main__.Person object at 0x00000226FF723F80>\n",
      " <__main__.Person object at 0x00000226FF722F00>]\n"
     ]
    }
   ],
   "source": [
    "# not preferred\n",
    "class Person:\n",
    "    def __init__(self, name: str, age: int, salary: int) -> None:\n",
    "        self.name = name\n",
    "        self.age = age\n",
    "        self.salary = salary\n",
    "\n",
    "\n",
    "data_3 = np.array([\n",
    "    Person('John', 30, 5000),\n",
    "    Person('Alice', 35, 6000)\n",
    "])\n",
    "\n",
    "# log\n",
    "print(f\"data_3:\\n{data_3}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "data_4         : [('John', 30, 5000) ('Alice', 35, 6000)]\n",
      "data_4['name'] : ['John' 'Alice']\n",
      "data_4['age']  : [30 35]\n",
      "data_4[0]      : ('John', 30, 5000)\n"
     ]
    }
   ],
   "source": [
    "# preferred\n",
    "data_4 = np.array(\n",
    "    [('John', 30, 5000), ('Alice', 35, 6000)],\n",
    "    dtype=[\n",
    "        ('name', '<U10'),\n",
    "        ('age', int),\n",
    "        ('salary', int)\n",
    "    ]\n",
    ")\n",
    "\n",
    "# log\n",
    "print(f\"data_4         : {data_4}\")\n",
    "print(f\"data_4['name'] : {data_4['name']}\")\n",
    "print(f\"data_4['age']  : {data_4['age']}\")\n",
    "print(f\"data_4[0]      : {data_4[0]}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Broadcasting\n",
    "   - It refers to how NumPy treats arrays with different dimension\n",
    "   - Avoid making explicit copies or reshaping arrays"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr_1d_9 = np.array([1, 2, 3, 4, 5])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "arr_1d_10: [2 3 4 5 6]\n"
     ]
    }
   ],
   "source": [
    "# not preferred\n",
    "arr_1d_10 = np.empty_like(arr_1d_9)\n",
    "\n",
    "# addition by 1\n",
    "for i in range(arr_1d_9.shape[0]):\n",
    "    arr_1d_10[i] = arr_1d_9[i] + 1\n",
    "\n",
    "# log\n",
    "print(f\"arr_1d_10: {arr_1d_10}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "arr_1d_11: [2 3 4 5 6]\n"
     ]
    }
   ],
   "source": [
    "# preferred\n",
    "arr_1d_11 = arr_1d_9 + 1\n",
    "\n",
    "# log\n",
    "print(f\"arr_1d_11: {arr_1d_11}\")"
   ]
  }
 ],
 "metadata": {
  "author_email": "AmirhosseinHeydari78@gmail.com",
  "author_github": "https://github.com/mr-pylin",
  "author_name": "Amirhossein Heydari",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.3"
  },
  "origin_repo": "https://github.com/mr-pylin/numpy-workshop"
 },
 "nbformat": 4,
 "nbformat_minor": 2
}