pytorch-segnet-deeplabv3-training.ipynb

{"cells":[{"metadata":{},"cell_type":"markdown","source":"## Pytorch SegNet and DeepLabV3 Implementation\n**Please upvote if this notebook is useful **"},{"metadata":{},"cell_type":"markdown","source":"![image.png](attachment:image.png)\nAfter Resnet34 Classifier Training ( https://www.kaggle.com/robinreni/heng-s-pytorch-resnet34-classifier-training ) we have to train our segmentation module","attachments":{"image.png":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAssAAAGBCAIAAAAfQWsbAAAgAElEQVR4Ae29f2wbx533P7Tl/HJsaUk1bp08bo5Uvpc8yUF+nJV9TeQCDpCQlQCjwbkl1QSFARuWyAYBGqSiLbl/FImVkPU5QNFKVJEARmGD1CVBg6JSJPm+MRAxRmOxecQmufRqLZv6EseOSa1lx3YsS94H8aedbHYpainxx3L5Jop0dvYzn/nMa0jPW7MzszZFURg+IAACIAACIAACIFBQAssK6g3OQAAEQAAEQAAEQOALAlAY+B6AAAiAAAiAAAgUngAURuGZwiMIgAAIgAAIgAAUBr4DIAACIAACIAAChScAhVF4pvAIAiAAAiAAAiAAhYHvAAiAAAiAAAiAQOEJQGEUnik8ggAIgAAIgAAIQGHgOwACIAACIAACIFB4AlAYhWcKjyAAAiAAAiAAAlAY+A6AAAiAAAiAAAgUngAURuGZwiMIgAAIgAAIgAAUBr4DIAACIAACIAAChScAhVF4pvAIAiAAAiAAAiAAhYHvAAiAAAiAAAiAQOEJQGEUnik8ggAIgAAIgAAIQGHgOwACIAACliUQDodtNpvH41lKC8PhcENDg+36JxAILMVVwct2d3fbbLZkMrmgZ4/HY7PZwuHwgpZFMqAAeF8QTx5PLBaz2WxDQ0NFqr0sbqEwyoIdlYIACIBAZRDo7u4OBoOSJBU73EwmEw6H+/v7jVcUj8d7enr8fn9jY6PxUua09Pl8brf78ccfz2Qy5oxwEVFBYSwCGoqAAAiAQLUQ6OvrY4x5vd50Oq0oSm9vb5Fa/thjjwWDwenpaeP+t2/fLgjCM888Y7yImS337t0ry/KPfvQjMweZV2xQGHnhgjEIgAAIVBcBWZYZY0888YTD4TBVy8PhsCRJu3fvNltgi6bU3NzsdrsHBgbi8fiinZiqIBSGqboDwYAACIAACCxMIJPJPP/884yxbdu2LWxtDou6ujrG2EMPPUThCILAGHvggQfU0T355JOMsWeffVadWblpKIzK7TtEDgIgYF4CtKwvHo+nUqlAIGC32202m91u9/l88y1LjMViVIpb5vW3LK1j4EsyfT5fKpXKAai/v7+pqYnWGzY0NAQCAY093SIPmzdvVl9S5oIeeO2pVKq7u5vH1tTUpF5vQctRR0ZGGGPBYNDI0tQjR47Isuz1ep1OJ6+FMcaxh8NhYt7U1JQb49DQkM/no9ZRB+W2V1eXV3rDhg2MsXXr1lGpjRs3MsbWrl2rdtLS0iIIwsjIiKYv1DaVlFbwAQEQAAEQKDQBt9vNGItEIvSnqmZUGBsbU1eYTqfJXmPGGPP7/WrL+dLpdFoURU1xQRC8Xi9jzO12qwtmNWaMCYKgDkzjjS7Jj0EPZByNRrNCEEWR1naEQiFNXZqA1cFTmhobiUQ0twgjtZr7lCRJURS6FQqF1EX8fj83Uyf0ntWleFpdZL40Rzo4OMgY45ddXV2MMe6KJyikrq4unlO5iSzNq9zGIHIQAAEQMAkBrhgEQYhGoxQVH2tFUVTHyUfEUChEw6EkSTzTyGBD1VFdNGzzuvQKg4ZnQRAikQgZT0xMcA8UAA+PBk4+LlK+cQ+SJJG8EEVxcHCQikciEXKrblpWBcDDUCfS6TQVn5iYUOdzGUHKjJrGK9X7pzGeMdbV1cWx80xeUFOF+nI+VaHO16BTF8+ajkajjDGXy5X1bmVlQmFUVn8hWhAAgcogQOMZY0wzCtL4of7jdWxsjAYk/d/NfLTTjPoaBNyDZjCbmJggz+opAR6AfgQl3aCZNSEPas95eaC/yF0uF433PHKatxAEgefoFQC/pUnQZICaITcgJ4IgaKrj4oPPYfD9t2qVQ34Iu5ExfszARx8JjzZrgvda7k7PWtZsmVAYZusRxAMCIGAFAjTUqYd2apVeDfAxWN/sdDpNEwB68aE2phFRMy9CBuRcHUbW5yZknHXk1iuMvDzMFz/NQ7jdbq7AjCsMUidZ20tOvF6vmg+lNf5pHiWrFuFzJGpdpXdYvBw98+LVVVTPWOlJXYn/ggAIgEDhCfCNA9x1c3MzT1OC1vTxZyLquw6Hg9YDHj16VJ2vSf/xj39kjD3yyCOafMbYt7/9bU3m6OioekeD+u6mTZvoMvdSR+MeUqkUbXa999571RUxxhwOh6Iow8PDizgsi87MyLFJldZUamrUXBLSjRs36v04HA6azjl27JimVCkv33///VJWV4y6oDCKQRU+QQAEQMAogePHjzPGamtrsxa4//77GWPnzp3LeledmdXDHXfcobZhjNGQT1s2aAMF/299fb3GOOulcQ+nTp0iD5odE1ndGs8kRWXcPqslIR0ZGeHNVycSiUTWUqXMzOvwsVIGZrwuKAzjrGAJAiAAAqUmQMN5iWtd+l/PS/dQ4iZnre6dd97Jms8z1aJkvnTuCSHuypIJKAxLdisaBQIgUDEE6DnIfH+w0jMUOqwpd5Oyejh//nzWUnx7S9bH8O3t7VlLqTONeFi1ahUV4ZMZag+LTtO8zqKLqwtqlrVqaMRiMbVxidNZJ6VKHMMSq4PCWCJAFAcBEACBJRGgM6MGBgb0XjKZDJ1DtWXLFv1dnkMjblYPb731FjejBC14fPPNNzX5xi+Ne2hsbKSVnlmnNJqufxYxitPQu8Q3hNESGVpTYrztGksDW0nG7rnnHk0pg5f6xSsGC5rHDArDPH2BSEAABKqRwA9+8APGmCRJ6mMuCcSBAwco8fDDD+dA8/3vfz+rh1QqRe8tU5fdvn07Y6yvr09/tGh/f7/NZmtqasp9oGReHnw+H2Ps5z//uUYQxOPxxPXPIgbg++67jzG2xKUSdNx4VuzJZNJmszU0NCz4LvVmAx/9SlJ1d+jT/KlKYRev6CsqQQ4URgkgowoQAAEQmJdAc3MzbSTp6OgIh8M0utMx2z09PXQelOZsbI2vxsZG2rBKHmgsHxoaeuSRR/TLOHw+H22U2LJlSywWI+NUKhUOhzs6OmiXR+7q8vLw9NNPC4IgSZLH4+FjZywW27p1Kx0FptlLsuDSB8YY3/OiF0kaMjkunU6nGhphz2QysViMZoympqZ4RTn8FPwWzfe4XK7cvVDweoviUPPYCZcgAAIgAAJLJ6A5fUHtkP4pV5+1UJBTw6lGzThBYkJ9HoaiKPOd+c0Y4yd584D10ebrYXBwkJ6V6GNTn0ZFR3eQjSZgHgxPkCbTHxOSA3vWW+pK1eEJgsAP6uCVliZBTdOfA1aa2gtbC+Yw1F8qpEEABECgDAQcDsfw8HA0GuUqgV4pMjY21tvbayQg7oEkBWmFaDT6wgsv6IuTcSQS4dXRdEI0Gh0fHzcyq5+Xh5aWlkQi4ff7XS4XBSOKYiQS0dT1zDPP8PGeVp/oI+c53/3udxljL774Is9ZXKK3tzcajXq9Xq6BRFEMhUInTpzQTK4szv8iStHSkB07diyirNmK2BRFMVtMiAcEQAAEQAAEchNoaGiQrn+s8DThH02NxWJtbW1ut3t4ePgfeRX8/5jDqODOQ+ggAAIgULUEdu3axRjbv3+/lQgcPHiQMbZ3715rNApzGNboR7QCBEAABKqOQENDw9TU1IkTJ4w82TE/nXg8vnnzZq/Xu4gdvOZsHeYwzNkviAoEQAAEQGABAgcPHpRlme/pXcDa9LefffZZQRBoA5HpgzUUIBSGIUwwAgEQAAEQMBuB5ubmrq6unp6epWxbNUmjYrHYyMhIb2+vlZaV4CmJSb5dCAMEQAAEQAAELEUAcxiW6k40BgRAAARAAARMQgAKwyQdgTBAAARAAARAwFIEoDAs1Z1oDAiAAAiAAAiYhECNSeJAGJVCIHy4nK8zrhRKiLNIBDof++I1WviAAAhUBAHMYVRENyFIEAABEAABEKgwAlAYFdZhCBcEQAAEQAAEKoIAFEZFdBOCBAEQAAEQAIEKI4B1GBXWYaYKFw/FTdUdVg0GS3+s2rNol+UJYA7D8l2MBoIACIAACIBAGQhAYZQBOqoEARAAARAAAcsTgMKwfBejgSAAAiAAAiBQBgJQGGWAjipBAARAAARAwPIEoDAs38VoIAiAAAiAAAiUgQAURhmgo0oQAAEQAAEQsDwBKAzLdzEaCAIgAAIgAAJlIACFUQboqBIEQAAEQAAELE8ACsPyXYwGggAIgAAIgEAZCEBhlAE6qgQBEAABEAAByxOAwrB8F6OBIAACIAACIFAGAlAYZYCOKkEABEAABEDA8gSgMCzfxWggCIAACIAACJSBABRGGaCXrMrLs7Mzc3Mlqw4VgQAI5EsAP9J8icG+ggjg7e0V1Fl5hDp9ZebPU/KFmauMsdU33NAg1NpvujGP8jAFARAoMgH8SIsMGO7LTwBzGOXvg4JHcHl29p0zZ0leMMbOz8y8c+Zs8mzm0uxsweuCQxAAgUUQwI90EdBQpOIIYA6j4rps4YDPXLw8pygau7OXLmcuf/7N1av+qXbVMptNcxeXIAACpSSAH2kpaaOuchHAHEa5yBex3ms6eUGVXVOUv06ff+vj02cuXipi9XANAiCwEAH8SBcihPtWIACFYYVe1LTh5hW5pqauzM29m55KnD772dUvVmngAwIgUHoC+JGWnjlqLD0BKIzSMy96jV9feYuw0LrOc1euvH3qzJ+n5Nlr14oeECoAARD4KgH8SL/KA1fWJACFYcF+tTG2Yc3X7lxovYXC2EcXLsY/Pv3RhYvaVRsWpIImgYCJCOBHaqLOQChFIwCFUTS0ZXVsY6yhrvZf166pv/mm3IHMXrv25yn5+Cdnzl25ktsSd0EABApIAD/SAsKEK3MSgMIwZ78UJqpbamrW31b/f26rX5lzZQZj7MLM1cTps++mp67ghK7CsIcXEDBEAD9SQ5hgVJkEoDAqs9/yidpx803/uvbrdwm1NcsW6O4zFy8d+/j0X6fPz7fQPZ9qYQsCIGCUAH6kRknBrqIILDDkVFRbEOy8BGyMfXP1qgfWrrn91pXzGl2/Maco0rnzfzh15uyly7ktcRcEQKCABPAjLSBMuDIJASgMk3REKcK4YfnyexzCxm/cVnvjDbnruzQ7mzyb+b+fpi9exTGguVEV5W4ymQwEAna73Xb94/P5hoaGjNSUyWSoVDweN2IPG7MRwI/UbD2CeJZCINfBCUvxW1llU+fOp6bPV1bMJYg2c/nzP1w+ve76MaALPmEpQTzWqyIejz/77LPDw8PqpsVisba2NnXOwPWP3+/v7e1V5+vTjz32mCzL+nwL5Hxj3bpvrFv3n3/7yAJtKWAT8CMtIEy4KjgBzGEUHKmlHCqM/e38hWOnzpz67KKlGmaCxsTj8c2bN4+MjKhjSaVSJC/cbvfExISiKOl0OhQKMcb6+vr6+/vVxpp0f3+/xpvGAJeWJIAfqSW71RqNgsKwRj8WtxUzc3P/lZHHT386fWWmuDVVvff9+/czxlwu1+HDhxsbGxljDoejs7PT7/czxn7+85/PRyiZTHZ0dAiCMJ8B8q1NAD9Sa/dvhbYOCqNCO64MYU9fmRk//an9ttvKUHc1VSmK4q5duxwOh7rRP/jBDxhjkiSpM3k6k8ns3LmTMXbo0CGeiUQVEqAfKWYcq7DrzdlkKAxz9osZo7q5puZf6u1Tn35a9uBo/WMmkwkEAnwtZCqV4oH19/c3NTXRrYaGhkAgoL7LzVKpVCAQaGhoyGEZj8dtNpvH42GMxWIx7rapqSkcDnNX6kQmk+nu7uZuyTKTyahtbDbb5s2bKYdqp7WZvb294+PjnZ2dauMF0wcOHEgkEn6/v6WlZUFjGFiYAP1I1y60ZczCBNA0UxHASk9TdYdJg1lms925etWCx5CXOPof/ehHAwMDVGkikXA6nYyxTCbj8XgSiQQPRpKkvr6+WCz2u9/9rrm5mecnk8ktW7ao10WSZV9f3+DgoH6oDgQCfX19vHji+ueNN97QrNPUuyXLl19+eWBggILkTownfvnLX9LTE32ReDze09PjcrmeeeYZ/V3kVAkBc/5IqwQ+mjkfAcxhzEcG+X8nsGblLd9au8ZZt3qZzWYqKAMDA5FIhNZCHjx4kGIjeSEIQiQSSafTiqJMTEy43W5Zlrdu3aqeyQgGg7Isi6JICyrJUhRFxtiTTz6paenIyEhfX5/X65UkSVEUSZK8Xi9jbGRkRL2PNJVKkWpxuVyDg4PK9c/g4KDL5UokElSEPCuKMjY2xtOKoqjVj7r2eDzu8/lIS/FmcoNMJrN161bG2MGDBzUPVrgNEpYnYNofqeXJo4G5CWAO4ws+zrrVzrrVuUlZ5u7Fq7N/kc9lLn++YItuXbHin+11C76mdUE/RTJwu93t7e20FpKG51gsRrMXhw4d4pMQjY2Nw8PDTU1NiURi//79fMMnbbt44YUXaEElY6yxsXFgYMDlcjU0NKRSKc18g9frjcVi1Ban00l1SZL03nvv8br2798vy7IgCKOjo7x4S0vL3XffLYpiIpGIxWI+n88gkHA4HAwGyVgQBM0cDOX/6Ec/kmU5FArNJ1AM1mV+s09Onvzk5MnOx4zSM3+LckRomR9pjjbiVjUQwBxGNfTy39s4e+3aCXn6D6dOLygvapYtu9subFq7xrTygjH26KOPajrvtddeY4y53W4+5HODn/3sZ7Thk+dQ4tixY+ocp9OpKMrw8DDXB/zuE088wdOUaGhoYIy98cYbPJ8kyO7duzXFnU4nbQbRT0LwsrkTsixv375dc45Wf3//wMCAKIr5rtvIXRfulpGAxX6kZSSJqs1AAArDDL1Qihg+/uzisVNn/nb+Qu4XtdsYu2PVyubbv37HqpXmeiiig3Tvvfdq8kZHRxljDz30kCafMbZp0ybK5IM0DfnBYJCWgg4NDWkWY2qc6CcJNBUlk0la1fHAAw9oyjLGHnzwQXqqor81X47b7ebPWURRlCRp8+bNyWSS7JPJ5J49ewRBePHFF+fzgPzKImC9H2ll8Ue0BScAhVFwpKZzOH1l5vgnn36QkWcWem+qcNONm9auudsuVOgJnjTAB4NB2p2h/m99fb2mY5555hm3201bQPv6+lpbW+vr65uammKxWG6pofHDLy9cuEDpzZs3q6umdGtrK7c0mOCPb1paWoaHh10uF2Psueeeo+I7d+6UZfm5557jZgbdwsyEBKrnR2pC+AipeASgMIrHtvyeZ+bm3k9PjZ/+9PzMAidl0Sa3+9d87dYVK8ofdzEjeP/998m9w+EYHh6emJjo6uqiBZ6MsUQi0dbW5vF4FicyjATOJyGMGHMbh8Oxa9cuxhgt+YzH47TipKOjQ6NmqAipHNpky50gYUIC+JGasFMQUqEIQGEUiqS5/CiMfTh94a2PT39y8VLuyJbZbM7a1d9au2bNyltyW1bK3Wg0Sg8Xsv6XFofytjQ2Nu7bt298fDydTkejUZrVSCQSBw4c4Db5Jmi/SdbaFUXBlEO+PK1qX80/Uqv2KdqlIQCFoQFikcs/nc1MnpueU3IvumAW2+RG+uDNN99csBeHhoY8Hg8t1SRjh8Ph8/mGh4fJyR//+McFnWgM+EKN48ePa24ZvKTDPOx2u3oHLC87PT3NGKNzwZubm+dTMGQ/NjZGS1Z5cSTMRqA6f6Rm6wXEU1QCUBhFxVse52cvXT576XLuum9dseL+NV/7l3r7zTXW2bG8fft22jCifxLR399vs9mamproSIzbb799ZGREkiS+8JPjqqurY4zRf3mmwQStHt27d6/+IYvP57PZbLm3qjocjsnJSVmWf/GLX2hqTCaTdN5Xbg+aUrg0LYGq/ZGatkcQWDEIFEBheDwem8023wnKxQgaPnMTuDBzNYdBRexEzRF/jls+n49WVGzZsoUv2EylUuFwuKOjg07OoH2kjY2NNFexdetWjSUtdPjhD3+Yo6L5bj399NOCIEiS5PF4+DxEMpnkR2Zt2LBBX1ath5599lnacuLz+Sg/k8nEYjE6yEsQBBzcqQdYiTlV+yOtxM5CzIsmYJ2/XxeNwHoFly/Lvs/Uxtjtq1Y21NVW6FYRIz01PDxMx3rSO9DVRURRPHz4MM/p7e0VRVGWZb3lol/w4XQ6jx49umXLlkQiod884vf71QdXrF27loJZv349Y2xsbKy5udnn87377rs9PT0D1z88Wjoy/NVXX8XBnWomlZuu5h9p5fYaIs+XQAHmMPKtEvbFJnDbLTcv153wXek7UQ1Cox0ikUiEpiiolNvtjkaj4+Pj6uHZ6XSeOHEiFArxjSSCIHi93rGxMX7up8FK1WaNjY2JREK9P2U+t06nkw4Up+L87K99+/aNjY35/X7+KnZRFEOh0Ntvv41VomrUFZ2u5h9pRXccgs+LgE1ZaDFgXu5gbBIC8udXPsjIl2ZnGWMrV9Q4a1cXaqtI+PDfT85mjFXJEc4m6dOqDcOqX7ni/Uir9quChpuNAJ6SmK1HChOPcNON37r96xevfrEgw/JHXBQGGbyAQGkJ4EdaWt6orQwECvCURL/Sk44AYozRayHpkk5LpCbOl68BQGZ2u5080OnO+sX/9M7ucDjc1NRElj6fL379wyNRe85kMt3d3Q0NDTywcDisX/yvLlKJadt1bQF5UYl9h5irhAB+pFXS0VXbzCLOYcRiMfUaOjotcXp6ura2Nmu+5iikQCBA2/N430iS1Hf9Mzg4qH61FZ0iQEcckjGtkqOtg7w4JZLJJC3L5/mJ65+XX355YGBA88IqboMECIAACIAACIBAXgQKMIcxX31tbW1ut3tiYkJRFEmSaD3dnj172travF4vnXuozlf7icViJC9CoVA6nabDhQYHB2nt25NPPqk2fuyxxxKJhCAIkUiEjMlSI1AYY6lUiuSFy+UaHBzkbl0uVyKR8Hq9ardIgwAIgAAIgAAILJpAERWGy+UaHh6m1e9Op5Neny3LsiiKsViMZgucTucLL7zAGJNlWX0qwN69exljtLuPr/9vaWk5dOgQvaqKNziZTI6MjDDGDh061N7eTsYtLS1Hjx7lS/G58f79+2VZFgRhdHSUz4K0tLSMjo4KgpBIJOjt29weCRAAARAAARAAgcURKKLCoBc18bD4iL5z506eyRjjZy3zV1NmMpmGhgZBEDTPTRhjq1evVpdljP3Hf/wHY0wURe6fDBobG/VPSUhA7N69W/M0xOl0kvHBgwc1/nEJAiAAAiAAAiCwCAJFXIfxwAMPZA3o3nvvzZrPM+lIA36ZyWQ++OCDjz766M033xwdHaX8ZDJJsyP0/ojvfe973J4nvvOd7/T09PDLZDJJb/fOGtiDDz5IZylyeyRAAARAAARAAAQWTaCICmPRMVHBeDz+y1/+cnR0lGSBxhuf8DD+lileZPPmzRpvuAQBEAABEAABECgsAZMqjO7ubj79IIqiw+F46KGH1q1bxxhT70OhBRyFJcJnRwrrFt5AAARAAARAoKoImFFhJJNJkhd+v/+ZZ57hKz3pgA1N97jdblrpqcnPcSlJkmYdRg5j3AIBEAABEAABEFgEgSKu9FxENFSEK4be3l61vGCMffTRRxq3999/P2Ps5Zdf1uQzxl5//XV1Jl9SavzBiro40iAAAiAAAiAAAsYJmFFhzBd9JpOhXaxqg+985zuMsUQioTnrM5PJ6M/DoA0je/fu1Z/g6fP5bDabz+dTO0caBEAABEAABEBgcQTMqDDuu+8+akwgEEilUnQoeCwW27RpkyRJmnY2NzfTSVlbt27t7+8n6TA0NOTxePRLRJ9++mlBECRJ8ng8Q0ND5CqZTPp8voGBAcbYhg0bNP5xWbkE9OfZl7EtWYPJZDKBQICfix+LxcLhsM1m83g8ZQwVVYMACIBAQQiYUWG0tLTQq7f7+vpcLpfNZquvr29ra5MkKRKJ0Dla58+f5+3/1a9+JYqiLMsdHR319fU2m621tTWRSOjPw3A6nXQSVyKRaG1tpfeSrF+/nuQFHfDF3SIBAsUm4PF4+vr69FK42PXCPwiAAAiUgIAZFQZjbHh4OBQK0UHjjDGXy9XV1SVJUnt7+yOPPMIY+/3vf8/p0PkZoVDI5XJRptfrHRsb+/a3v80Y05zs2djYmEgkurq6uHNBEMi+t7eX+0QCBIpNIJlM0st0QqEQHWCPh3TFZg7/IAACpSRgUxSllPWVsq5wOBwMBt1u9/DwcCnrtXZd4cMx3sDOx0y9bMXj8YyMjIRCoc7OTh6zeRLxeJyOZrHwb7AgtCvoK1eQ9sIJCFiGgEnnMIzzDYfD9FZ3fZFf//rXjDHabKK/ixwQAAEQAAEQAIHiEah4heF2u+mt7nxZKGOMFm/SstAdO3YUDx88l5FALBajHUA2m81utwcCAfXL83IEFovFaN0lLcRpamrq7u6mNcXqUplMJhwONzU1kZndbvf5fJotS2SvcejxePSv0FOv9IzH4zabjZ8tS/5pded8Kz0zmUx3d3dDQwOPORwO67dE0V1aQEppn8+nb5q6mUiDAAiAQLEI0APgiv6vfkUnwRIEIRqNVnTTTBh86FCU/69c4aXTaVoLrP9VqHucbPgqB0VR0uk0X3+jKUubjHiLclh2dXVxM0VR5vv6iaKoNlMHMzY2pqmdMeZ2uxVFCYVCPM2LT0xMaJYTUXFRFCVJ4mb8aQvtriIbl8ulNqjENP++hQ7h51yJHYiYq5dAxc9hMMZ6e3vHxsa8Xi//V9jlcvn9/kQigaVz+pHMAjkHDhygY9lCoRANsZIkkXRQT2XpW/rTn/40kUiopWc6nY5EInT8/P79+3mRl156iSwHBwfpnwdJkkhM9PT08FmBeDxOx65EIpF0Ok0ihhwmEgn9TAb5b25uVhSF6wzyP99qoVQqtWXLFlmWXS4XD2ZwcNDlciUSCbWY4MEPDAxEIhEKBq8L5liQAAEQKDEBM54avggEzdc/iyiIIhVHIJVK0aHykUikvb2d4nc6ncPDw3fddZcsy0eOHOH56talUilSA729vVx6OhyO9vb2kydPqnUDY+yNN95gjO3evT0TPzoAACAASURBVLulpYVX0dvbOzo6KgjCn//8Zzp4/tixYzTlwGskh8lkcnR09OTJk+oAFpfev3+/LMuCIIyOjvLT7ltaWu6++25RFEnH8OZQFW63m+JxOBz8KNvF1Y5SIAACILBoAlaYw1h041GwEgkcOXKENiHzQZ1a4XA4/H6/y+Wab1w/deqU2+0WBEEzHjPGamtrs6J45513NGsdJicnx8fHueygUpOTk3xWg3J6e3snJycLsoeFJkJ2797N5QVV4XQ6aU5FP0vx6KOPZm0OMkEABECglASgMEpJG3UVgAAt59y4caPe1759+yYnJ/ft26e/xRhrbm4eHh6empqiu6lUKh6P9/f3+3y+559/njGmfmHN9u3bGWMDAwN33XWXz+eLxWIaDUFOtm3bxhiTJMnlcnk8nv7+foOrTbNGqM9MJpN0HtcDDzygv/vggw8yxvh7fLjBvffey9NIgAAIgEC5CFjkKUm58KHe0hOgkV7zB73xMGKx2MGDB/WjMi3F4H58Pt+7777b09Mjy/LA9Q+d/LZr165t27bx2p1OZzQaDQQCsiyPXP/Q/IrP52tvb29sbOQOF5e4cOECFeQbTxbnB6VAAARAoPQEMIdReuaosWwEmpqa2traSF643W6v1xsKhQYHB7u6uvQx7du3jw6q5/tWJEkKBoOiKKonKnw+34kTJ6LRKF9rLMtyX1/f+vXr51vpqa9riTnqeJboCsVBAARAoFAEoDAKRRJ+SkSgrq6OMZb1mUXuCPr7++mUbtpnMTw8HIvFOjs7W1pa5luH4XQ629vbh4eHFUUZHBykdQ+yLO/cuVNdl8PhoCcpU1NTExMTXV1dtK2pra1Ns4xDXSqvtGZXqmb329InS/IKBsYgAAIgYIQAFIYRSrAxEYEtW7Zo1kzw4IaGhux2+3wvJv3tb3+r2ffBC3744Yc8TfLF4/HY7Xa1jmlpaent7aXzKkipMMZ8Pl9DQ4N6rqKxsXHfvn2/+93vyOEHH3yg9pxvmu8EUa8RydcJ7EEABECgLASgMMqCHZUunsDDDz9Mayb6+/s1Xn7xi1/IskyTHJpbOS6TySTtYuU2Tqfz+PHjsiy/8sorPJMSNNvBT16hZZ763RyrVq0ie57Q+DF+SRMne/fu1U+H0JGm+q0xxp3DEgRAAASKRwAKo3hs4bkoBJxOJy2b6Ojo4Cdnp1Ipn89HCyyeeOKJrBXTG2pGRkb6+/tptE6lUuFwmCZFNEV2797NGAsGg+FwmGYyMplMLBbbs2cPY4xGfcYY1TUyMuLz+fhiiKGhIXqMIori0p9fPP3003TeqMfjGRoaojjpXPyBgQHG2IYNGzTB4xIEQAAEzEAACsMMvYAY8iPw1FNP0VmWwWCwvr7eZrO5XC4abqPRKH+yoHH61FNP0dxDR0cHLxUMBhljdAonY4zPE3R2dtIhocFg0OVy2Wy2+vr6trY2WZZFUXzqqafIeXNzM8mdgYGB9evX06tAWltb6TzQF198URPDIi6dTufRo0cFQUgkEq2trVTF+vXrqb1+v78gp24sIjAUAQEQAIHcBKAwcvPBXTMScDgcsVgsGo3yXR6CIPj9/omJiRyPDBwOx4kTJ+hULmqVKIqRSOTEiRPt7e0kPug4L7o7Pj6uroLWcESj0fHxcYfDwbns27dPc2i9KIqhUOjEiRNLn8CgWhobGxOJRFdXF3+piiAIXq93bGyst7eXR4IECIAACJiKgI2/LclUYSEY0xIIH47x2Dof8/E0EiBQJAL4yhUJLNyCQLEJYA6j2IThHwRAAARAAASqkQAURjX2OtoMAiAAAiAAAsUmAIVRbMLwDwIgAAIgAALVSAAKoxp7HW0GARAAARAAgWITgMIoNmH4BwEQAAEQAIFqJACFUY29jjaDAAiAAAiAQLEJQGEUmzD8gwAIgAAIgEA1EoDCqMZeR5tBAARAAARAoNgEoDCKTRj+QQAEQAAEQKAaCUBhVGOvo80gAAIgAAIgUGwCUBjFJgz/IAACIAACIFCNBKAwqrHX0WYQAAEQAAEQKDYBKIxiE4Z/EAABEAABEKhGAlAY1djraDMIgAAIgAAIFJsAFEaxCcM/CIAACIAACFQjASiMaux1tBkEQAAEQAAEik0ACqPYhOEfBEAABEAABKqRABRGNfY62gwCIAACIAACxSZQU+wK4N/CBMKHYxZuHZoGAiAAAiCwFAKYw1gKPZQFARAAARAAARDITgBzGNm5IBcEFk2gzuG4eeVKXnxmZka5du3ihfNXLn/OM5EAARAAAcsTgMKwfBejgaUmUOdw2G+7TVPrh3/5CxSGhgkuQQAErE3ApiiKtVuI1oFAiQn8V0Y+9dlFTaX/2yGsvfXLiQ3NXVyCAAiAgPUIYB2G9foULQIBEAABEACB8hOAwih/HyACEAABEAABELAeASgM6/UpWlRmAjNzc/oIapbht6anghwQAAErE8C/elbuXbStLATmsq1tWgGFUZbOQKUgAALlIwCFUT72qBkEQAAEQAAErEsACsO6fYuWgQAIgAAIgED5CEBhlI89agYBEAABEAAB6xKAwrBu36JlZSJwde6avubly2z6TOSAAAiAgIUJQGFYuHPRtPIQwErP8nBHrSAAAiYjAIVhsg5BOCAAAiAAAiBgCQJQGJboRjQCBEAABEAABExGAArDZB2CcEAABEAABEDAEgSgMCzRjWiEmQjgTE8z9QZiAQEQKBsBKIyyoUfFViWAlZ5W7Vm0CwRAIC8CUBh54YIxCIAACIAACICAIQJQGIYwwQgEQAAEQAAEQCAvAlAYeeGCMQiAAAiAAAiAgCECUBiGMMEIBAwSuHot24GeNhzoaZAfzEAABKxDAArDOn2JlpiBwGw2hXHD8uVmiA0xgAAIgEApCUBhlJI26gIBEAABEACBaiEAhVEtPY12ggAIgAAIgEApCUBhlJI26gIBEAABEACBaiEAhVEtPY12loYAVnqWhjNqAQEQMD8BKAzz9xEirCQCc9cUfbgrluOHpqeCHBAAAYsTwD98Fu9gNA8EQAAEQAAEykIACqMs2FEpCIAACIAACFicABSGxTsYzQMBEAABEACBshCAwigLdlRqWQJY6WnZrkXDQAAE8iQAhZEnMJiDQE4CONMzJx7cBAEQqCICUBhV1NloKgiAAAiAAAiUjAAURslQoyIQAAEQAAEQqCICUBhV1NloKgiAAAiAAAiUjAAURslQo6KqIICVnlXRzWgkCICAAQJQGAYgwQQEDBPIfqbnMvzQDBOEIQiAgFUI4B8+q/Qk2gECIAACIAACZiIAhWGm3kAsIAACIAACIGAVAlAYVulJtAMEQAAEQAAEzEQACsNMvYFYKp9A9pWey2yV3zK0AARAAATyIwCFkR8vWINAbgJzSra3t2OlZ25quAsCIGBFAlAYVuxVtAkEQAAEQAAEyk0ACqPcPYD6QQAEQAAEQMCKBKAwrNiraBMIgAAIgAAIlJsAFEa5ewD1W4vAzNycvkErsA5DDwU5IAACVicAhWH1Hkb7Sksg60rPGiiM0vYCagMBEDADASgMM/QCYgABEAABEAABqxGAwrBaj6I9IAACIAACIGAGAlAYZugFxAACIAACIAACViMAhWG1HkV7ykvg6tw1fQBYh6FnghwQAAHLE4DCsHwXo4ElJTDPSk+cGl7SXkBlIAACZiAAhWGGXkAMIAACIAACIGA1AlAYVutRtAcEQAAEQAAEzEAACsMMvYAYQAAEQAAEQMBqBKAwrNajaE95CeBMz/LyR+0gAALmIQCFYZ6+QCRWIDDPSk/80KzQuWgDCIBAXgTwD19euGAMAiAAAiAAAiBgiAAUhiFMMAIBEAABEAABEMiLABRGXrhgDAIgAAIgAAIgYIgAFIYhTDACASMErl7DgZ5GOMEGBECgKghAYVRFN6ORpSEwm01hrMCr20tDH7WAAAiYjAAUhsk6BOGAAAiAAAiAgCUIQGFYohvRCBAAARAAARAwGQEoDJN1CMIBARAAARAAAUsQqLFEK9AIECgRgTlFyVz+fLnNVnvjDfp3ss+z0hMvVi1R76AaEAABUxGAwjBVdyAYUxPIXP78T2czdGpnzbJl99Xb62++SR3x3DVFfUlpvRBhjF2anf3b9IVLs7Orbljxv1bdenMNfol6csgBARCobAJ4SlLZ/YfoS0ZgZm4u+Q95wRibvXbt3bOZy7Oziwhg+srM26fOfPzZRfnzKyfPfzZ++uylRflZRNUoAgIgAAIlIwCFUTLUqKiyCZy9/Pk15StTFHOKkjp3Pt9WXVOU99NT6teXzMzNnZCn8/UDexAAARAwOQEoDJN3EMIzC4Grc1lO0/rk4qULM1fzCvHD6w9HNEU+y9OJpjguQQAEQMCEBKAwTNgpCMmMBBxfXXLBQ/yLfI6n51np+eWv7NLs7IfnL3B7nqhZhtWgHAYSIAACFiHw5b99FmkQmgECxSGw6oYVX7vlZr1v+fMr6cufU/6CZ3r+OSNrHrVQwW+sXKn3jBwQAAEQqGgCUBgV3X0IvqQE7hJqs041TMrTX1mgMU9QZy5emvr8iv7myhU1/2v1rfp85IAACIBARROAwqjo7kPwJSVwS03N7auyTDZ8dvXqqc8u5g5l9tq1/55nOec9DiGrcMntEHdBAARAwOQEoDBM3kEIz1wEnLWrl9uy6AHp3Pmsjz949JPnpmfm5vglT6y9dWXdjTfySyRAAARAwDIEoDAs05VoSCkI3LB8+TdXr9LXNDM39+H0hXlWetqmr8x8fCHLJEfNsmV3CbV6b8gBARAAAQsQgMKwQCeiCSUlcGftqhuWL9dX+bfzF7IewLXctuy/p85lXahxl1CLd7vrSSIHBEDAGgSgMKzRj2hF6Qgss9lcdav19c0pyqnPLunzT3128fzMjD6/7sYbb781y6oOvSVyQAAEQKASCUBhVGKvIeYyE1h768qVK7K8SSTrUowr2ZZf2Bi721FX5magehAAARAoJgEojGLShW+LErAxdpewJH3wzdpVt65YYVE8aBYIgAAIfEEACgPfAxBYDIH6m28SblrkHpCba2r+qTbLc5bFxIEyIAACIGBWAlAYZu0ZxGV6AoveBvLP9rqsW15N32IECAIgAAJ5EIDCyAMWTEFATWD1DTesWXmLOsdI+mu33Fw/zytOjBSHDQiAAAhUCgEojErpKcRpRgKuutXLsh3ANV+sy222u+1LWsAxn2fkgwAIgIDZCEBhmK1HEE8lEbilpuaObOeIz9cGZ93qG7OdpTGfPfJBAARAoHIJQGFUbt8hclMQcNaurllm6He06oYV67KdB2qKZiAIEAABECg0AUP/Mha6UvgDAesQqFm27E5juuFuO95wZp1+R0tAAAQWJACFsSAiGIDAAgTWrb715posB3Cpi92xamXtjTeoc5AGAQ2BcDhss9k8Ho8mP6/LcDjc0NBgu/4JBAJ5lS22cXd3t81mSyaTC1bk8XhsNls4HF7QskgGFMAS+6JIsRXWbTwep29LPB4nz4FAwG63p1KppVcEhbF0hvBQ7QSW2Wz/VJvldWicy4plyxrq8IYzzgOJYhHo7u4OBoOSJBWrgn/4zWQy4XC4v7//HxkL/388Hu/p6fH7/Y2NjQtbw6J8BJ555hnGWEHkKRRG+boRNVuIwNpbV666Yd4zOv8/e53BtRoWQoKmlIFAX18fY8zr9abTaUVRent7ixTEY489FgwGp6enjfvfvn27IAg0ehkvBcvSE3A4HLt37x4ZGclLQWaNEwojKxZkgkDeBOY7R1y46cZv5H9sRt7VowAIMCbLMmPsiSeecDgcpuIRDoclSdq9e7fZAjMVJfMEs2PHDkEQ9uzZk8lklhIVFMZS6KEsCHxJwH7TjQ7dUVrLvjgAQ/jSCCkQqD4CmUzm+eefZ4xt27atUlpfV/fFuTUPPfRQpQS86DhXrfr7E97m5mbuxOFw+P1+WZZfeuklnrmIBBTGIqChCAhkJ9BQV2v76p1vrl6V9S2sX7XCVdURoHUMfEmmz+fLvbCuv7+/qamJVuQ1NDQEAgGNPd0ijps3b1ZfUuaCHngfpFKp7u5uHltTU5N6tpyWo46MjDDGgsGgkaWpR44ckWXZ6/U6nU5eC2OMXIXD4aGhIaquoaFBXZfamNJDQ0M+n49aZ7fbfT4fX5+oN15KzoYNGxhj69atY4ylUim73Z61pYFAwGaz8XWR1Ed2u13zp38sFqOYh4aG5otqaGiIbDTrWzOZDLltamoqRjC0LMblcmkC+/73v88Y+/Wvf63Jz+9SwQcEQKBwBN5PTx358H/of2999MnctWuF8w1PFiGQTqdFUdT8Sy0IgtfrZYy53W51O7MaM8YEQRgbG+OWGm90SXcNeiDjaDQqCFlm3URRpLUdoVBIU5cmYB4ST1BjI5EIz6EEuaJWc5/RaFRRFLfbzRgLhULqIn6/n5upE3rP6lI8rS4yX5ojHRwcZIzxy0gkQkUoPPJJNowxHsDExASZ+f1+Xq8kSYTU6/XyzKwJjkKSJG7Q1dVFPicmJiiz4MGIopi1E0l2DA4O8mDyTbB8C8AeBEAgB4HPZ2f//799RAojfelyDkvcqloCNHwKghCNRmnYVo/rmn/raXgWBCESiZDxxMQE96AeihRFoaGIj4tE2LgHPhaKosjHFT6edXV18S7LqgD4XXUinU5rBkh+l4sVURSpIWNjY9RGvX8+0HZ1dZGxJEk8k0fLnesTFEbu/2rQqZ1w5hRhOp3Oqht4o7grKuhyuaig2qcmzX3y78DY2BgFzEUMFSlBMIqikKRTqyVNwAteQmEsiAgGIJAfgUl5+siH//Ons5n8isG6OgjwMYOPQNRu/ucvH10URYlGozTA6EdQ0g2af/3JWO05Lw80oujHQho1BUHgXaRXAPyWJsH/0NfkK4qiH4y5jcY/33+rVjlkTCLD5XLxsvMlxgx8cogALr+IOc03CIKgL0JdQyFxfabulPkiVBSF4yL1SbMI6q8ElS1NMBS8EbbztWiBY4Jyyz3crSoC4cOxqmrvohu7vKbm7sb1/3n87ddnri7aCQpqCHQ+5tPkVOjl66+/zhgTRVG9sI4x1tjY6Pf7abspb9prr71Gz01aWlp4JiV+9rOftba29vX15d6SmpeHWOyL3/hPfvITzY6PHTt2BIPBjRs3JpPJfE+zeO+996i9mvj5pSAIGhT8Fk8cOXKEHgw99dRTPJMSTz31VE9PjyRJ8Xg8t5/cdzVu9ZdOp/O5557r6Ojo6+sTBGFgYIAx9rvf/U7DijH24osvrl+/XpKkQCBASEOhkMHaW1pavF7vwMBAIBDw+XykJA4fPqyJpzTB3HvvvYwxLu80MRi5xEpPI5RgAwJ5EJibnf3vPyWvQl7kwayKTP/4xz8yxh555BF9m7/97W9rMkdHR+fb0bBp0yYyzr3U0biHVCpFm11pXFFH4nA4FEUZHh7OV14wxujMDP0wzP1v3LiRp+dLHD16lDG2ceNGvR+Hw0FzBseOHZuveKHy29vbaXKlp6eHMdbV1ZVVNzQ2NtL0TF9fnyzLoih2dnYaj+FXv/qVIAiyLJPcPHTokL7VjLHSBENhGzmGNWsDoTCyYkEmCCyJwOxVzF4sCaDlC9fWZjnj9Y477tA0nIZ82rJBGw34f+vr6zXGWS+Nezh16hR5WLt2bVZXi8skRbW4srzUuXPnGGMjIyO8+epEIpHglsVO8OkEQRD0Eyq89s7OTtI9jLEXXniB5xtJOByOQ4cOkaXb7dZPX3EnJQiG6rpw4QKvNK8EFEZeuGAMAiAAAuYi8P777y8xoKV7WGIABSn+zjvv5PajFiXzpXNPCDHGXnnlFapFlmWe1tebTCa57vnlL3+pN8id85vf/IYMRkZGcswf8ACKGkzuUHPfxTqM3HxwNzsByzwUz9485JqDgIWX/mQ9b/v8+fNZwUejUZ9vSctQjHjgJy+dOnVKc3BF1qgMZt5///10eIZB+xxmfr8/96KTHGULciuVSu3Zs4cx5nK5JEnas2fPww8/nJXVzp07udnAwMAPf/jDHFMRmtiGhoZokQfVsnPnzvHxcY0NHYxRgmCoXv7d0IeROwdzGLn54C4IgAAIFJLA/fffzxijIUTj96233tLk0FP/N998U5Nv/NK4h8bGRtp+mXVKo+n6h9YtGq+dMUbPgzQnUOXlga9EoTUl+Zbl9ga2kozdc8893F6fCAQCtK5icnJSFEVZlrO+HiwcDicSCUEQRkdHaavL448/bpBAJpN5/PHHaZHH6OioIAiJREJzBhcFVoJgOIFFrL/5e9n5NpkgHwQ0BEKHovx/mlu4BIFiEODft9ChL05hssaH70rVnHDA9x+qtybyvab8tCUOgXYS8pMkKJ/+WVdvjMzLw3y7VfkOWx6GZjcpj0qf4Nsv9bdoOaS6vdxG459vZ9BAUxSFeLpcLv2GXu6tIAm+75QgcCaakDT9y4+4WPC4LQqSNsHyDcN8Qy8nT2alCYZqX8puVZyHUZDvXlU4seQ/91XRcxXbSKt+5fg5UaFQiE5TGBwc5Mc2a0ZcWjCoPp5LkiQ+8GiM9QpDURTjHrjKEUWRyxR+Gpi6LlIARkbNBU/cUrvlX1WNwlAURQ2NTtxKp9M8tqznUnBvS09MTEzQBI/6QA4KSRAEfu4ZPz5VFEVeKddYC2ogLgfVltR9aoelCUZRFJI7mjNXeLuMJKAwjFCCzRcErPrPPXrXtASs+pVLp9M0gvJZaErQWKIZcfmgpTGmQyY0xz1lVRh5eRgcHKShVFMdPzWcvi3qA7w1Aeu/TjRQaf7W5yduZS2uVxj8iElNYHROhuZPfH0MS8yhruFTC+QtnU6TLuTDP5dBmnj48Vxci+jj4fJOo9v4pAgXNyUIhsKjb4Ja7ujDzp0DhZGbD+5+ScCq/9x/2UKkTEbA2l+5aDRKQwVphWg0ShPv+hE3nU5HIhG1KHG73eoXZPB+y6owFEUx7kFRFEmS/H4/n1MRRVEvDtLptFpk8ACyJuhPcz4McxvjT0l4kWg06vV6uQYSRZHPA3Gbgif4jBGf1+FV8PmJUCjEn5toXqdCSClmfedyV9S/WSdjuHAZGxsrTTDqx088wkUkbPwoe70wRA4IqAmoF/ZjL4maDNJFIoCvXJHAlt5tQ0ODdP2TdedF6eNBjQsSCAQCfX19oVAor+PCNG6xl0QDBJcgAAIgAAIFJrBr1y7G2P79+wvsF+6KQyCTycRiMUEQduzYsZQaoDCWQg9lQQAEQAAEFibQ2dnpcrlisZjBTZsLe4RFMQm89NJLsiw/99xzWQ8sN14zFIZxVrAEARAAARBYJIGDBw/KsnzgwIFFlkexUhHIZDLPP/+8KIrt7e1LrBMKY4kAURwEQAAEQGBhAs3NzV1dXT09PTmOwV7YCyyKT+CnP/0pvSF26VVBYSydITyAAAiAAAgsTGDfvn2Koiz+gMiFa4BFAQj09vZOTU0VpJugMArQH3ABAiAAAiAAAiCgIQCFoQGCSxAAARAAARAAgQIQgMIoAES4AAEQAAEQAAEQ0BCAwtAAwSUIgAAIgAAIgEABCEBhFAAiXIAACIAACIAACGgIQGFogOASBEAABEAABECgAASgMAoAES5AAARAAARAAAQ0BKAwNEBwCQIgAAIgAAIgUAACUBgFgAgXIAACIAACIAACGgJQGBoguAQBEAABEAABECgAASiMAkCECxAAARAAARAAAQ0BKAwNEFyCwCIJxONxm80WDofnK59MJgOBgN1ut13/+Hy+oaGh+YyRDwIgAAKVTgAKo9J7EPGXgkAsFgsEAjlqymQyW7duzWEQi8XWr1/f19cnyzKZDQwMtLa25nabwyFugQAIgIDJCUBhmLyDEF75CYTD4ba2tlQqNV8omUzG4/Fw6aA3S6VSpCREURwbG1MURZIkv9/PGOvr64vFYvoiyAEBEACBSicAhVHpPYj4y0wgmUxu2rQpkUjkiOPIkSOyLAuCMDw83NzczBhzOp29vb1er5cx9u///u85yuIWCIAACFQoASiMCu04hF1+AplMJhwOr1+/XpIkt9stiuJ8Mf32t79ljG3cuNHhcKhtfvjDHzLGcqsTtT3SIAACIFBBBKAwKqizKixUWvno8XiSyaTH47HZbHa7XbPsoL+/v6mpiVY+NjQ0BAKBrA8j4vG4z+fjaySbmprC4XAmk1ETCYfDtNCSBv6GhgZy6/F45ltQqVl66fF4NA8sqAnBYJAxNjIyQg55pS+99FIwGBQEIRQKDQ8Pa9QDN0MCBEAABKqTABRGdfZ76VqdyWT+7d/+bWRkhDGmXqmQyWSampo6Ojr4X/CSJPX19YmiGI/H1fHFYrHNmzcPDAzw4olEIhgM3nXXXXo5Mj097fF4gsGgJEnkZGRkpLW1Vb/FQ7/0cmRkpK2tzefzqWvPnfb7/YlEorOzM7fZo48+yhg7fvy4RhX95je/YYy5XK7cxXEXBEAABCqRABRGJfZaJcWcSCSmpqb48sb29naK3uPxJBIJQRAikUg6nVYUZWJiwu12y7K8detWLh0ymQxNe/j9fkmSlOufwcFBQRBkWd6/f7+GRU9PTyKRCIVC5HNsbIweXgSDQfXoPjQ01NbWxhhzu90TExOKoqTT6UgkIgjCwMBAd3c3uW1ublYUJRQKkSXVzmvs7Ozs7e11Op08Z77Etm3bRFGUZdnj8ZB+ymQy3d3dAwMDjLFf/OIX8xVEPgiAAAhULgEojMrtu4qJfPfu3Xx5Y2NjI2MsFovR1MWhQ4fa29vp+UJjY+Pw8DCNxFw6fPDBBzR1oR7LW1paent7BUGYmprSU4hEIp2dneSzubmZRnHG2AcffMCNn3zyScaYKIrDw8MUksPhaG9vP3ToEGOsp6eHSxxeZCkJh8MxPDxMEx6bN2+22Wz19fU9PT0ul2twcLClpWUpzlEWBEAABMxJAArDnP1i3CY53AAAFbJJREFUqajcbremPa+99hrNCugH15/97Ge0h1NTRPPoxOfzTU1NaZZNUBE+TUKXfI7h2LFjlJNMJukZygsvvKCppaWlheY8XnnlFc2tJV6+/fbbo6OjGidTU1PvvfeeenJFY4BLyxO4PDs7Mzdn+WaigdVJAAqjOvu9pK2mSQJ1lTTWPvTQQ+pMSm/atIkSJCmam5tpmcLmzZubmpq6u7s1UkPjQa9mSMqozf7whz/QJc2sqG8xxh555BHG2BtvvKHJX8plOBxubW2VJKmrq4ue9aTT6Wg0yhgLBoMej2cpzlG2QglMX5l5+5Mzb318+s2PPjn+yadTn1+p0IYgbBCYjwAUxnxkkF9EAvTgIxgM0u4M9X/r6+s1Fb/66qskMhKJRE9PDz1lWMqR29PT01SFul6e7unp0QSwxMtUKkW7UUKh0L59+2hOxeFw+Hy+o0eP0m7VrJMxS6wXxc1M4PLs7Dtnzl6YuUpBnp+ZeefM2eTZzKXZWTOHjdhAIC8CUBh54YJx6Qi8//77VFljY+Pk5OTg4KDf7+fbLop95Pbx48cL1VTuaseOHRqfjY2NdLLnwYMHNbdwaW0CZy5enlMUTRvPXrr8h1NnpHPnr+luaSxxCQIVQQAKoyK6yZpBRqNR2p2R9b+a5RS0unNyclKSpEgkQlKjr69vvrMuFkTmcrmy1kuZWdeQLugzq8HJkycpP+uBGXfeeWfWUsi0NoH5NMQ1Rfnr9Pm3Pj595uIlaxNA66qBABRGNfSy6dpIqyXefPPNBSMLh8NNTU3qc7qcTmd7e/vbb78tCAJj7L333lvQicbggQceYIxJklTYDSOaWvjlunXrKJ11ReeHH37ILZGoHgI3r6jJ0dgrc3PvpqcSp89+dvXvj1FyGOMWCJiWABSGabvGyoFt376dNowkk0lNO/v7+202W1NTEw3/tbW1iUQiFotphmc+H1BbW6vxsOAlXz3a1dWlMc5kMg0NDXa7XX9Cl8bS+OXGjRvJ+MCBA5pSyWSyr6+PMUZHcmnu4tLCBL6+8hbhphtzN/DclStvnzrz5yl59tq13Ja4CwLmJACFYc5+sXhUPp+PNoVu2bKFq4dUKhUOhzs6OhhjDoeDVkRu27aNDtdSH/6dTCZ9Ph+9S+zhhx9eBCw65GpgYMDn83GVMzQ05PF4JEmSZZnmOdSeJycnNSpHfTdH2ul0kpTp6ekJBAJUXSaTicViW7ZsoTM9t23blsMDblmPgI2xDWu+dmftqmU2W47WKYx9dOFi/OPTH124qF21kaMYboGAOQjkmqkzR4SIwpoEhoeH6VhPOltT3UhRFA8fPkw5Dofj0KFDra2tiUSitbVVbcYYUx/DpbmV+7KlpSUajba1tQ1c/2iMo9GoeiMrPeaQJIn2uSj5r8Lbt2+fLMt9//ioq3O5XK+++iqfklHfQtraBGyMNdTVrr115V+mzqUvf56jsbPXrv15Sv74s8/+2V5Xd+MCMx85/OAWCJSYAOYwSgwc1f2dAB1zGYlE1CdYuN3uaDQ6Pj6uHnFbWlokSVJvJHG5XHSIeF7vENGg9/l8Y2NjRtz6fL5QKETLPhhjuQ/k0NTCL3t7e8fGxrxeL/cjimIkEnn77bf154XwUkhYnsAtNTXrb6v/P7fVr8y5MoMxdmHmauL02XfTU1dwQpflvxZWaaBtEX+QWaXtaEd+BMKHY7xA52N5vB6Ml0ICBPIiUFVfOYWxk+cv/HX6woKrLpbbbHfWrvrm6gWesOSFGsYgUAwCmMMoBlX4BAEQAIH8CNgY++bqVQ+sXXP7rStzl5xTFOnc+T+cOnP20uXclrgLAuUlAIVRXv6oHQRAAAS+JHDD8uX3OISN37it9sYbvszNlro0O5s8m/m/n6YvXsUxoNkAIc8EBLDS0wSdgBBAAAQWIpA6dz41fX4hq6q7n7n8+R8un163etU/1a6qWYa/GKvuC2DyBuMbafIOQnggAAIgkIuAwtjfzl84durMqc8u5rLDPRAoOQEojJIjR4UgAAIgUGgCM3Nz/5WRx09/On1lptC+4Q8EFkkACmOR4FAMBEAABMxGYPrKzPjpTzGZYbZ+qdp4sA6jarseDQcBELAagZtrahrqVq9ZeYvVGob2VCYBKIzK7DdEDQIgAAIqAststjtXr1rwGHJVCSRBoOgEoDCKjhgVgAAIgEBRCaxZeUtD3eqba/DveVExw3neBPCNzBsZCoAACJSegLNutbNudenrLUuNF6/O/kU+l8n5shIK7NYVK/7ZXrfga1rL0gpUCgJQGPgOgAAIgIBZCMxeu/bX6Qsnz19Y8E2qNcuWNdTV3r5qZa53s5qlWYijSglAYVRpx6PZIAACZiPw8WcXpXPnZxZ6sZmNsdtXrWyoq8URW2brQcSjIYDdqhoguASBCiYQDodtNpvH46ngNlRl6NNXZo5/8ukHGXlBeSHcdOOmtWvutguQF1X5TamwRkNhVFiHIVzLEIjFYoFAYCnNWbqHpdSOsgUhMDM39356avz0p+dnFjgp6+aamn+pt9+/5mu3rlhRkKrhBASKTQAKo9iE4R8EshAIh8NtbW2pVCrLPWNZS/dgrB5YFYuAwtiH0xfe+vj0Jxcv5a5jmc3mrF39rbVrcNBFblC4azYCUBhm6xHEAwKLJ9DZ2akoyvDw8OJdoGSpCPzpbGby3PScssCazjUrb/nW2jXOutXLbFjTWaq+QT0FIoCVngUCCTcgAAIgYJjA2UuXz166nNscO1Fz88Fd8xPAHIb5+6iCI8xkMuFwuKmpyXb9Y7fbfT5fPB7P2qShoSGfz2fEMpVKBQKBhoYGm81mt9sDgUAqldIvcqSccDhMYWjsGWPz5WvCy2Qy3d3dVNxmszU1NZFPjZnH47HZbPF4nMKz2+0Uns/nSyaT3Dgej9tstmAwyBgbGRmh9vK7jDF167iHWCzGbXJ40EOgUvF43OfzUUi0FFTtkHvmwZA9XTY0NAQCgUwmw82QWDqBCzNXczipWbbsbruwae0aHHSRgxJuVQABBR8QMEYgdCjK/2ekRDqdFkUx62+gq6tL48Hv92e1jEQiGsuJiQlBEDTGgiB4vV7GmNvt5vahUIgx1tXVpQ/D5XJJkpQ1P51Ocw+KomStjjEmiqIkSWpLt9vNGItEIvrwGGNjY2NkPDY2pgmeMcb9RKNR/V3K4dByeKAmqyEoitLV1ZXVpyiKmsaSWdYYBEHQGPOYi5fg37fQoWjxaimL5w+nzx/58H/0//vPD//ng8zU1bm5skSFSkGgsAQwh5H1315kFoDASy+9lEgkBEEYHBykb60kSaQkenp61Iscu7u7+/r6SA3QsC1JEo2LHR0dQ0NDPJpMJrNlyxZZll0uF7lNp9ORSESW5YGBAW6mTvT09EiSFIlEaICMRCKMMZIXkiRFo38funj+K6+8wounUilNdYqiDA4OulyuRCJBmoYbU6Kjo4Mxxt1Go1ESHD/+8Y/JoLm5WVEUjRSgW6lUqq2tjTHm9Xq5fJmYmCAlxKHl8KAJhjHW39/f09NDPicmJhRFSafTVHsikci6r7WtrU0URZJE3FiW5QMHDuj9I2dxBG675eblunUV2Im6OJgoZV4ChRUs8GZhAvn+QUl/04dCIQ0Tl8sliqJadtDPg/+Nzu1JZLhcLp5DOkAQBD4A0y3+Z7f6z3caR9XjPRnzqQseA+VTwF6vl1dHekhfnSRJpBu4klAUhYozxmgg5054bDxHrzDoFlWnbi/lp9NpQsQnQubzoBEu6XSa4lQ3inwODg6ST3UTKMflcmmmKygwNVt1W4qXzvcrV7xIiuF56vLnb330CU1jHPv4k9OfXSxGLfAJAmUkgDkM+kcV/y0WgXfeeUfzCH9ycnJ8fLylpYWqPHLkCGNMEISnnnpKEwTlSJLEl268+OKLjDG/3+90OtXGPp+P6wZ1PqV9Pp8683vf+x7VyGOguw899BBj7Ny5c9yYFivs3r1bU53T6aRB9+DBg9yYEm63u7GxUZ15xx130CVvhfquJu1yuX7yk59oMh0OhybH4OWRI0dkWWaM0TSGulRLSwtJotdee02dzxjbtWuXpsY777yTVo1oLHG5FALCTTd+6/av/+vaNf+6ds231n4dO1GXAhNlzUkACsOc/WKFqLZv384YGxgYuOuuu3w+XywWUz8Z4S08evQoY2zjxo2aUY0x5nA4SDccO3aM7BOJBGPsO9/5Di/OE6Qb+CVP8KkFnkOJjRs3anI0l8lkkobnBx54QHOLMfbggw9mHXRJpqjtm5ub1Zc50r29vZOTk+3t7WQTj8eHhoa6u7ubmpoo5/33389RXH/r5MmTtGREo5DIkkIdHR3VFNS3V5+jKYLLxRGwMXbrihU4QWtx9FDK/ASwW9X8fVSpEfp8vnfffbenp4cWSdA6CZfLtWvXrm3btvExj+YMaFdF7qaqd2TktizI3QsXLpCfzZs3F8ShESepVGr//v2jo6OSJOntp6en9Zk5ct555x0Sallt7rvvPsYYqaisBsgEARAAgaUQwBzGUuih7AIE9u3bR6ss+USCJEnBYFAUxbzkAo2UfMhfoNbS3s6rIblDGxoacrlcfX19kiS5XC63293V1RWJRCYmJnIXXNzd8+fPL64gSoEACICAEQKYwzBCCTaLJ+B0OtuvfxhjQ0NDv//97/v6+mRZ3rlz5/j4OPfr9/t7e3v5ZdaE8ccNWYsvOlOSJD7jsmgnRgo+/vjj9FBjYGCgIDVu2LBhYGBAsw6GR0LPULLureU2SIAACIDAoglgDmPR6FAwF4FUKuXxeOx2u3rtRUtLS29vL98qSeXnWw2Q1Tsty3j99df1d19++WV95lJyuKA5fvz4UvwYLBuPx+mBxQsvvKCRF2qGBr2R2bp16xhjiUQiqwci9sgjj+TlE8YgAAIgYJAAFIZBUDDLj4DT6Tx+/Lgsy+rjJchFbW0tbeWgy23bttEBFf39/Zo6ksmkzWZraGjgR2LQcs6+vj7N3+XxeJwWgWo8LPGSNozs3btXUx1jjI4f1exSWWJ18xWf78is+ex5/sMPP0xTFHoPQ0NDROy73/0ut0cCBEAABApIAAqjgDDh6isEdu/ezRgLBoPhcJj+hs5kMrFYbM+ePbTjlKydTieNfx0dHRrLLVu2MMampqY2bdpExjt27HC5XLIsb9q0iWQH+dy6detX6i7QxdNPP02HYXg8Hq5yksmkz+ejhasbNmxYSlWTk5Ncu9xzzz3k6sc//jFf2zE0NOTxeOY7TIwxpvagj8ThcDz33HO0o4cfXp7JZPr7+/kTmdKIJH1syAEBELA8AazDsHwXl62BnZ2dL7/8ciKRCF7/qOMQRVF9+sW+fftkWe7r69NbCoJw9OhRvpHV4XC8+uqrW7ZskSSptbWV+6RTw3OMxNwyr4TT6Tx69OiWLVsSiYS6OnLi9/s7OzvzcsiN6fmFJEn19fV0arjD4QiFQsFgMJFIrF+/nluSGkulUiMjI+q9JHoP6iI83d7efvLkyZ6enoHrH55PCz7wFlY1EKRBAAQKSwBzGIXlCW9fITA+Ph6NRvlGEnpvSDQaHR8f56KBCvT29kajUa/XyxceiqIYCoVOnDihOcCqsbExkUj4/X6yFATB7/cnEgm73c4Yq6ur+0oES76g6tQvNyE1MzY2tuDS1ByV+3y+UCjEG0uHcXV2dg4ODnJc1DSq6NFHH6WpCO4zqwd+V53Yt2/f2NiYmq3b7c7aC+pSSIMACIDAEgnYFEVZogsUrxIC4cNfvt6z87GvnJJpBgIej2dkZCQUCi16XsEMrUAMagIm/8qpQ0UaBEBATwBzGHomyDEvAY/H09TUpF8TSg8RGGN0ipR5G4DIQAAEQKBqCEBhVE1XW6Khjz76aCKR6Ojo6O/v52skh4aG6DWnLpdL86oRSzQajQABEACBiiQAhVGR3Va1QW/bto2OxOjo6Kivr7dd/7S2tiYSCZfL9eqrr1YtGTQcBEAABMxGAArDbD2CeHIRcDgc+tWjtCb07bff1qwJzeUI90AABEAABIpMALtViwwY7otAwHf9UwTHcAkCIAACIFAwApjDKBhKOAIBEAABEAABEOAEoDA4CiRAAARAAARAAAQKRgAKo2Ao4QgEQAAEQAAEQIATgMLgKJAAARAAARAAARAoGAEojIKhhCMQAAEQAAEQAAFOAAqDo0ACBEAABEAABECgYASgMAqGEo5AAARAAARAAAQ4ASgMjgIJEAABEAABEACBghGAwigYSjgCARAAARAAARDgBKAwOAokQAAEQAAEQAAECkYACqNgKOEIBEAABEAABECAE4DC4CiQAAEQAAEQAAEQKBgBKIyCoYQjEAABEAABEAABTgAKg6NAAgRAAARAAARAoGAEoDAKhhKOQAAEQAAEQAAEOAEoDI4CCRAAARAAARAAgYIRqCmYJziqJgLhw7Fqai7aCgIgAAIgkDcBzGHkjQwFQAAEQAAEQAAEFiQAhbEgIhiAAAiAAAiAAAjkTQAKI29kKAACIAACIAACILAgAZuiKAsawQAEQAAEQAAEQAAE8iKAOYy8cMEYBEAABEAABEDAEAEoDEOYYAQCIAACIAACIJAXASiMvHDBGARAAARAAARAwBABKAxDmGAEAiAAAiAAAiCQFwEojLxwwRgEQAAEQAAEQMAQASgMQ5hgBAIgAAIgAAIgkBcBKIy8cMEYBEAABEAABEDAEAEoDEOYYAQCIAACIAACIJAXASiMvHDBGARAAARAAARAwBABKAxDmGAEAiAAAiAAAiCQFwEojLxwwRgEQAAEQAAEQMAQASgMQ5hgBAIgAAIgAAIgkBcBKIy8cMEYBEAABEAABEDAEAEoDEOYYAQCIAACIAACIJAXASiMvHDBGARAAARAAARAwBABKAxDmGAEAiAAAiAAAiCQFwEojLxwwRgEQAAEQAAEQMAQASgMQ5hgBAIgAAIgAAIgkBcBKIy8cMEYBEAABEAABEDAEAEoDEOYYAQCIAACIAACIJAXASiMvHDBGARAAARAAARAwBABKAxDmGAEAiAAAiAAAiCQFwEojLxwwRgEQAAEQAAEQMAQASgMQ5hgBAIgAAIgAAIgkBcBKIy8cMEYBEAABEAABEDAEAEoDEOYYAQCIAACIAACIJAXASiMvHDBGARAAARAAARAwBABKAxDmGAEAiAAAiAAAiCQFwEojLxwwRgEQAAEQAAEQMAQASgMQ5hgBAIgAAIgAAIgkBcBKIy8cMEYBEAABEAABEDAEAEoDEOYYAQCIAACIAACIJAXASiMvHDBGARAAARAAARAwBABKAxDmGAEAiAAAiAAAiCQFwEojLxwwRgEQAAEQAAEQMAQASgMQ5hgBAIgAAIgAAIgkBcBKIy8cMEYBEAABEAABEDAEAEoDEOYYAQCIAACIAACIJAXgf8HDJMCpfUkkBAAAAAASUVORK5CYII="}}},{"metadata":{"trusted":true},"cell_type":"code","source":"import cv2\nimport collections\nimport os\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport random\n\nfrom PIL import Image, ImageFile\nimport pandas as pd\nimport torch.nn as nn\nfrom tqdm import tqdm_notebook as tqdm\nfrom ipywidgets import IntProgress\nfrom torch.autograd import Variable\n\nimport torch.utils.data\nfrom torchvision import models\nfrom torchvision import transforms\nimport torchvision\nfrom torch.utils.data import DataLoader, Dataset, sampler\n#torch.backends.cudnn.enabled = False","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Checking python versions\nimport platform\nprint(f'Python version: {platform.python_version()}')\nprint(f'PyTorch version: {torch.__version__}')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Read csv\ntrain = pd.read_csv(\"/kaggle/input/severstal-steel-defect-detection/train.csv\")\ntrain.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_defect = train[train['EncodedPixels'].notnull()]\ntrain_defect = train_defect.reset_index(drop=True)\ntrain_defect = train_defect[:2000] # Loading only 2000 samples to check whether flow is correct\ntrain_defect.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Mask Decoder :\nTo mask the images from the given encodings "},{"metadata":{"trusted":true},"cell_type":"code","source":"def rle2mask(rle, imgshape):\n    width = imgshape[0]\n    height= imgshape[1]\n    \n    mask= np.zeros( width*height ).astype(np.uint8)\n    \n    array = np.asarray([int(x) for x in rle.split()])\n    starts = array[0::2]\n    lengths = array[1::2]\n\n    current_position = 0\n    for index, start in enumerate(starts):\n        mask[int(start):int(start+lengths[index])] = 1\n        current_position += lengths[index]\n        \n    return np.flipud( np.rot90( mask.reshape(height, width), k=1 ) )","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Visualize data "},{"metadata":{"trusted":true},"cell_type":"code","source":"columns = 1\nrows = 4\nfig = plt.figure(figsize=(20,columns*rows+6))\nfor i in range(1,columns*rows+1):\n    fn = train_defect['ImageId_ClassId'].str[:-2].iloc[i]\n    fig.add_subplot(rows, columns, i).set_title(fn)\n    img = cv2.imread( '/kaggle/input/severstal-steel-defect-detection/train_images/'+fn )\n    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n    mask = rle2mask(train_defect['EncodedPixels'].iloc[i], (256, 1600))\n    img[mask==1,0] = 255\n    plt.imshow(img)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### DataSet and DataLoader"},{"metadata":{"trusted":true},"cell_type":"code","source":"class ImageData(Dataset):\n    def __init__(self, df, transform, subset=\"train\"):\n        super().__init__()\n        self.df = df\n        self.transform = transform\n        self.subset = subset\n        \n        if self.subset == \"train\":\n            self.data_path = '/kaggle/input/severstal-steel-defect-detection/train_images/'\n        elif self.subset == \"test\":\n            self.data_path = \"/kaggle/input/severstal-steel-defect-detection/test_images/\"\n            \n    def __len__(self):\n        return len(self.df)\n    \n    def __getitem__(self, index):\n        fn = self.df[\"ImageId_ClassId\"].iloc[index].split('_')[0]\n        img = Image.open(self.data_path + fn)\n        img = self.transform(img)\n        \n        if self.subset == \"train\":\n            mask = rle2mask(self.df[\"EncodedPixels\"].iloc[index], (256, 1600))\n            mask = transforms.ToPILImage()(mask)\n            mask = self.transform(mask)\n            return img,mask\n        else:\n            mask = None\n            return img","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"data_transf = transforms.Compose([\n                                    transforms.Resize((256,800)),\n                                    transforms.ToTensor(),\n                                    transforms.Normalize(mean=[0.485], std=[0.229])\n])\ntrain_data = ImageData(df= train_defect, transform = data_transf )\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### Loss Function :\nSince they mentioned to use dice loss explicitly we are using it for the segmentation"},{"metadata":{"trusted":true},"cell_type":"code","source":"class DiceLoss(torch.nn.Module):\n    def __init__(self):\n        super(DiceLoss, self).__init__()\n        \n    def forward(self, logits, targets):\n        \n        N = targets.size(0)\n        preds = torch.sigmoid(logits)\n        \n        EPSILON = 1e-7\n        \n        # Flattening the final layer \n        preds_flat = preds.view(N,-1)\n        targets_flat = targets.view(N, -1)\n        \n        intersection = (preds_flat * targets_flat).sum()\n        union = (preds_flat + targets_flat).sum()\n        \n        loss = (2.0 * intersection + EPSILON) / (union + EPSILON)\n        loss = 1 - loss / N\n        return loss","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Train Function"},{"metadata":{"trusted":true},"cell_type":"code","source":"# To start with optmized learning rate\ndef warmup_lr_scheduler(optimizer, warmup_iters, warmup_factor):\n\n    def f(x):\n        if x >= warmup_iters:\n            return 1\n        alpha = float(x) / warmup_iters\n        return warmup_factor * (1 - alpha) + alpha\n\n    return torch.optim.lr_scheduler.LambdaLR(optimizer, f)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# train function\ndef train(model, optimizer, data_loader, device, epoch):\n    model.train()\n    loss_func = DiceLoss() #nn.BCEWithLogitsLoss() \n\n    lr_scheduler = None\n    if epoch == 0:\n        warmup_factor = 1. / 1000\n        warmup_iters = min(1000, len(data_loader) - 1)\n\n        lr_scheduler = warmup_lr_scheduler(optimizer, warmup_iters, warmup_factor)\n\n    lossf=None\n    inner_tq = tqdm(data_loader, total=len(data_loader), leave=False, desc= f'Epoch: {epoch+1}')\n    for images, masks in inner_tq:\n        y_preds = model.forward(images.to(device))\n        # For Segnet\n        y_preds = y_preds[:, 1, :, :] \n        # For DeepLab V3\n#         y_preds = y_preds['out'][:, 1, :, :] \n\n        loss = loss_func(y_preds, masks.to(device))\n\n        if torch.cuda.device_count() > 1:\n            loss = loss.mean() # mean() to average on multi-gpu.\n\n        loss.backward()\n        optimizer.step()\n        optimizer.zero_grad()\n\n        if lr_scheduler is not None:\n            lr_scheduler.step()\n\n        if lossf:\n            lossf = 0.98*lossf+0.02*loss.item()\n        else:\n            lossf = loss.item()\n        inner_tq.set_postfix(loss = lossf)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"!wget 'https://download.pytorch.org/models/vgg19_bn-c79401a0.pth'","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Model Definition\n\ndef initialize_weights(*models):\n    for model in models:\n        for module in model.modules():\n            if isinstance(module, nn.Conv2d) or isinstance(module, nn.Linear):\n                nn.init.kaiming_normal(module.weight)\n                if module.bias is not None:\n                    module.bias.data.zero_()\n            elif isinstance(module, nn.BatchNorm2d):\n                module.weight.data.fill_(1)\n                module.bias.data.zero_()\n                \nclass _DecoderBlock(nn.Module):\n    def __init__(self, in_channels, out_channels, num_conv_layers):\n        super(_DecoderBlock, self).__init__()\n        middle_channels = int(in_channels / 2)\n        layers = [\n            nn.ConvTranspose2d(in_channels, in_channels, kernel_size=2, stride=2),\n            nn.Conv2d(in_channels, middle_channels, kernel_size=3, padding=1),\n            nn.BatchNorm2d(middle_channels),\n            nn.ReLU(inplace=True)\n        ]\n        layers += [\n                      nn.Conv2d(middle_channels, middle_channels, kernel_size=3, padding=1),\n                      nn.BatchNorm2d(middle_channels),\n                      nn.ReLU(inplace=True),\n                  ] * (num_conv_layers - 2)\n        layers += [\n            nn.Conv2d(middle_channels, out_channels, kernel_size=3, padding=1),\n            nn.BatchNorm2d(out_channels),\n            nn.ReLU(inplace=True),\n        ]\n        self.decode = nn.Sequential(*layers)\n\n    def forward(self, x):\n        return self.decode(x)\n\n\nclass SegNet(nn.Module):\n    def __init__(self, num_classes, pretrained=True):\n        super(SegNet, self).__init__()\n        vgg = models.vgg19_bn()\n        vgg19_bn_path = \"vgg19_bn-c79401a0.pth\"\n        if pretrained:\n            vgg.load_state_dict(torch.load(vgg19_bn_path))\n        vgg = vgg.cuda()\n        features = list(vgg.features.children())\n        self.enc1 = nn.Sequential(*features[0:7])\n        self.enc2 = nn.Sequential(*features[7:14])\n        self.enc3 = nn.Sequential(*features[14:27])\n        self.enc4 = nn.Sequential(*features[27:40])\n        self.enc5 = nn.Sequential(*features[40:])\n\n        self.dec5 = nn.Sequential(\n            *([nn.ConvTranspose2d(512, 512, kernel_size=2, stride=2)] +\n              [nn.Conv2d(512, 512, kernel_size=3, padding=1),\n               nn.BatchNorm2d(512),\n               nn.ReLU(inplace=True)] * 4)\n        )\n        self.dec4 = _DecoderBlock(1024, 256, 4)\n        self.dec3 = _DecoderBlock(512, 128, 4)\n        self.dec2 = _DecoderBlock(256, 64, 2)\n        self.dec1 = _DecoderBlock(128, num_classes, 2)\n        initialize_weights(self.dec5, self.dec4, self.dec3, self.dec2, self.dec1)\n\n    def forward(self, x):\n        enc1 = self.enc1(x)\n        enc2 = self.enc2(enc1)\n        enc3 = self.enc3(enc2)\n        enc4 = self.enc4(enc3)\n        enc5 = self.enc5(enc4)\n\n        dec5 = self.dec5(enc5)\n        dec4 = self.dec4(torch.cat([enc4, dec5], 1))\n        dec3 = self.dec3(torch.cat([enc3, dec4], 1))\n        dec2 = self.dec2(torch.cat([enc2, dec3], 1))\n        dec1 = self.dec1(torch.cat([enc1, dec2], 1))\n        return nn.Parameter(dec1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Model Declaration \n#SegNet\nmodel_ft = SegNet(num_classes=4,pretrained=False)\n\n# DeeplabV3\n# model_ft = torchvision.models.segmentation.deeplabv3_resnet101(pretrained=False, num_classes=4)\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n_ = model_ft.to(device)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# train DataLoader\nNUM_GPUS = torch.cuda.device_count()\nif NUM_GPUS > 1:\n    model_ft = torch.nn.DataParallel(model_ft)\ndata_loader = DataLoader(train_data, batch_size = 8, shuffle = True,\n                         num_workers = NUM_GPUS)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Optimizer \nparams = [p for p in model_ft.parameters() if p.requires_grad]\noptimizer = torch.optim.SGD(params, lr = 0.001, momentum = 0.9 , weight_decay = 0.0005)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Learning Scheduler\nlr_scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size = 5, gamma = 0.1 )","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"num_epochs = 2\nfor epoch in range(num_epochs):\n    train(model_ft, optimizer, data_loader, device, epoch)\n    lr_scheduler.step()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Save the model \ntorch.save(model_ft.state_dict(),\"model_seg_last.pth\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Loading and check the model\n# SegNet\nmodel = SegNet(num_classes = 4)\n\n# DeepLabV3\n# model = torchvision.models.segmentation.deeplabv3_resnet101(pretrained=False, num_classes=4)\n\nmodel.load_state_dict(torch.load(\"model_seg_last.pth\"))\nmodel.eval()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"A Huge thanks to the author of  https://www.kaggle.com/mobassir/deeplabv3-resnet101-for-severstal-sdd ."}],"metadata":{"kernelspec":{"display_name":"pook","language":"python","name":"pook"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.6.8"}},"nbformat":4,"nbformat_minor":1}