Openstreetmap (2).ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Wrangle-OpenStreetMap-Data\n",
    "\n",
    "On the particular project, I am using data mungling techniques to assess the quality of OpenStreetMap’s (OSM) data for the mumbai city regarding their consistency and uniformity. The data wrangling takes place programmatically, using Python for the most of the process and SQL for items that need further attention.\n",
    "\n",
    "The dataset describes the city of mumbai.Mumbai,India is the closest thing I have to a hometown in the India as I lived there for a good chunk of my childhood, so I was keen to take a look at it in this new, OpenStreetMap-based lens. The size of the dataset is 66 MB and can can be downloaded from here: https://mapzen.com/data/metro-extracts/metro/mumbai_india/\n",
    "\n",
    "About the project\n",
    "\n",
    "Scope\n",
    "\n",
    "OpenStreetMap (OSM) is a collaborative project to create a free editable map of the world. The creation and growth of OSM have been motivated by restrictions on use or availability of map information across much of the world, and the advent of inexpensive portable satellite navigation devices.\n",
    "\n",
    "On the specific project, I am using data from https://www.openstreetmap.org/node/16173235 and data mungling techniques, to assess the quality of their validity, accuracy, completeness, consistency and uniformity. The biggest part of the wrangling takes place programmatically using Python and then the dataset is entered into a SQL database for further examination of any remaining elements that need attention. Finally, I perform some basic exploration and express some ideas for additional improvements.\n",
    "\n",
    "Skills demonstrated\n",
    "\n",
    "Assessment of the quality of data for validity, accuracy, completeness, consistency and uniformity. Parsing and gathering data from popular file formats such as .xml and .csv. Processing data from very large files that cannot be cleaned with spreadsheet programs. Storing, querying, and aggregating data using SQL. Mumbai, India\n",
    "\n",
    "https://www.openstreetmap.org/node/16173235 https://mapzen.com/data/metro-extracts/metro/mumbai_india/\n",
    "\n",
    "Problems Encountered in the Map\n",
    "\n",
    "Problems Encountered in the Map Once the location was decided, I downloaded the full extract of the region and ran Python code to investigate any issues with the data. The following problems were discovered:\n",
    "\n",
    "Street Names: Incomplete ('hanuman raod ___') or incorrect names ('Zhopadpatti'), along with street abbreviations ('rd.' instead of 'Road')\n",
    "\n",
    "Postal Codes: Inconsistent postal code formats ('500023' and '120045') and incorrect post codes ('123')\n",
    "\n",
    "To tackle these issues, I had to create python scripts to clean each respective category of data. Auditing part is explained in Openstreetmap.ipynb notebook"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I have created mumbai_sample file which is part of mumbai_india file and it can be used for various experiments before using those on main osm file, we can also get the idea about the format of osm file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#!/usr/bin/env python\n",
    "# -*- coding: utf-8 -*-\n",
    "\n",
    "import xml.etree.ElementTree as ET  # Use cElementTree or lxml if too slow\n",
    "\n",
    "OSM_FILE = \"mumbai_india.osm\"  # Replace this with your osm file\n",
    "SAMPLE_FILE = \"mumbai_sample.osm\"\n",
    "\n",
    "k = 500 # Parameter: take every k-th top level element\n",
    "\n",
    "def get_element(osm_file, tags=('node', 'way', 'relation')):\n",
    "    \"\"\"Yield element if it is the right type of tag\n",
    "\n",
    "    Reference:\n",
    "    http://stackoverflow.com/questions/3095434/inserting-newlines-in-xml-file-generated-via-xml-etree-elementtree-in-python\n",
    "    \"\"\"\n",
    "    context = iter(ET.iterparse(osm_file, events=('start', 'end')))\n",
    "    _, root = next(context)\n",
    "    for event, elem in context:\n",
    "        if event == 'end' and elem.tag in tags:\n",
    "            yield elem\n",
    "            root.clear()\n",
    "\n",
    "\n",
    "with open(SAMPLE_FILE, 'wb') as output:\n",
    "    output.write('<?xml version=\"1.0\" encoding=\"UTF-8\"?>\\n')\n",
    "    output.write('<osm>\\n  ')\n",
    "\n",
    "    # Write every kth top level element\n",
    "    for i, element in enumerate(get_element(OSM_FILE)):\n",
    "        if i % k == 0:\n",
    "            output.write(ET.tostring(element, encoding='utf-8'))\n",
    "\n",
    "    output.write('</osm>')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Different types of tags mumbai_india file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'bounds': 1,\n",
      " 'member': 13085,\n",
      " 'nd': 2356083,\n",
      " 'node': 2051208,\n",
      " 'osm': 1,\n",
      " 'relation': 3994,\n",
      " 'tag': 391731,\n",
      " 'way': 283963}\n"
     ]
    }
   ],
   "source": [
    "import xml.etree.cElementTree as ET\n",
    "import pprint\n",
    "tags={}\n",
    "def count_tags(sample_file):\n",
    "    for event, elem in ET.iterparse(sample_file, events=(\"start\",)):\n",
    "        if elem.tag in tags.keys():\n",
    "            tags[elem.tag] += 1\n",
    "        else:\n",
    "            tags[elem.tag] = 1\n",
    "        #print tags\n",
    "    return tags\n",
    "tags = count_tags('mumbai_india.osm')\n",
    "pprint.pprint(tags)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Tag types \n",
    "\n",
    "  \"lower\", for tags that contain only lowercase letters and are valid,\n",
    "  \n",
    "  \"lower_colon\", for otherwise valid tags with a colon in their names,\n",
    "  \n",
    "  \"problemchars\", for tags with problematic characters, \n",
    "  \n",
    "  \"other\", for other tags that do not fall into the other three categories."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'lower': 375605, 'lower_colon': 15589, 'other': 530, 'problemchars': 7}\n"
     ]
    }
   ],
   "source": [
    "import xml.etree.cElementTree as ET\n",
    "import pprint\n",
    "import re\n",
    "lower = re.compile(r'^([a-z]|_)*$')\n",
    "lower_colon = re.compile(r'^([a-z]|_)*:([a-z]|_)*$')\n",
    "problemchars = re.compile(r'[=\\+/&<>;\\'\"\\?%#$@\\,\\. \\t\\r\\n]')\n",
    "\n",
    "\n",
    "def key_type(element, keys):\n",
    "    if element.tag == \"tag\":\n",
    "            if lower.search(element.attrib['k']):\n",
    "              keys[\"lower\"] += 1\n",
    "            elif lower_colon.search(element.attrib['k']):\n",
    "              keys[\"lower_colon\"] += 1\n",
    "            elif problemchars.search(element.attrib['k']):\n",
    "              keys[\"problemchars\"] += 1\n",
    "            else:\n",
    "              keys[\"other\"] += 1\n",
    "    return keys\n",
    "\n",
    "\n",
    "\n",
    "def process_map(filename):\n",
    "    keys = {\"lower\": 0, \"lower_colon\": 0, \"problemchars\": 0, \"other\": 0}\n",
    "    for _, element in ET.iterparse(filename):\n",
    "        keys = key_type(element, keys)\n",
    "\n",
    "    return keys\n",
    "keys = process_map('mumbai_india.osm')\n",
    "pprint.pprint(keys)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To find number of unique users in mumbai_india osm file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1749"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import xml.etree.cElementTree as ET\n",
    "import pprint\n",
    "import re\n",
    "def get_user(element):\n",
    "     if \"uid\" in element.attrib:\n",
    "        return element.attrib[\"uid\"]\n",
    "\n",
    "\n",
    "def process_map(filename):\n",
    "    users = set()\n",
    "    for _, element in ET.iterparse(filename):\n",
    "       users.add(get_user(element))\n",
    "       users.discard(None)\n",
    "    return users\n",
    "users = process_map('mumbai_india.osm')\n",
    "#pprint.pprint(users)\n",
    "len(users)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Street Names\n",
    "There are two things the auditing function needed to accomplish:\n",
    "Replace any abbreviations of street types with the type completely spelled out.\n",
    "  1. 'Veer Savarkar Rd' -> 'Veer Savarkar Road'\n",
    "  \n",
    "Replace incorrect or incomplete street names with the corrected/complete counterparts.\n",
    "\n",
    "  2. 'jhopadpatti' -> 'Slums'\n",
    "\n",
    "In order to discover problematic street names, I first had to use the regular expression (re) module to locate street types at the end of an address."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "OSMFILE = \"mumbai_india.osm\"\n",
    "street_type_re = re.compile(r'\\b\\S+\\.?$', re.IGNORECASE)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Afterwards, I matched them against a list of acceptable street types. If they weren't in the list of expected types, they would be added to a dictionary as keys, with the addresses that contain the problematic cases as the values.\n",
    "\n",
    "Having this overview allowed me to determine what my auditing function needed to accomplish. I created a dictionaries for mapping/correcting purposes - 'mapping'. If my function came across a problematic street type, it would refer to that dictionaries for the corrected version to be replaced with.\n",
    "\n",
    "An analysis of the XML data, along with outside research on Google Maps and OpenStreetMaps, was needed to identify the missing street types for the incomplete street names.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Universal Mobile,DVD and Travelling shop => Universal Mobile,DVD and Travelling shop\n",
      "Vidhan Bhavan => Vidhan Bhavan\n",
      "kurla => kurla\n",
      "lalluabai compound => lalluabai compound\n",
      "THAKKAR BAPPA COLONY => THAKKAR BAPPA COLONY\n",
      "DATAR COLONY => DATAR COLONY\n",
      "P.L. Lokhande Marg,MHADA COLONY => P.L. Lokhande Marg,MHADA COLONY\n",
      "Thakurli Station Rd => Thakurli Station Road\n",
      "Veer Savarkar Rd => Veer Savarkar Road\n",
      "Bhandup Village Rd => Bhandup Village Road\n",
      "Sambhaji Nagar Rd => Sambhaji Nagar Road\n",
      "Pipeline Rd => Pipeline Road\n",
      "Vasai Station Rd => Vasai Station Road\n",
      "NL Rd => NL Road\n",
      "Marol Village Rd => Marol Village Road\n",
      "Vishnu Nagar Rd => Vishnu Nagar Road\n",
      "Yashavant Nagar Rd => Yashavant Nagar Road\n",
      "vishnu Nagar Rd => vishnu Nagar Road\n",
      "Balraj Sahni Rd => Balraj Sahni Road\n",
      "Wodehouse Rd => Wodehouse Road\n",
      "Koparkhairane Village Rd => Koparkhairane Village Road\n",
      "Datta Mandir Rd => Datta Mandir Road\n",
      "Belpada Rd => Belpada Road\n",
      "Goregaon Mulund Link Rd => Goregaon Mulund Link Road\n",
      "Agrawal Rd => Agrawal Road\n",
      "Carmichael Rd => Carmichael Road\n",
      "Sane Guruji Rd => Sane Guruji Road\n",
      "Shahid Mangal Pandey Rd => Shahid Mangal Pandey Road\n",
      "Mavla Maratha Rd => Mavla Maratha Road\n",
      "Off New Link Rd => Off New Link Road\n",
      "Station Rd => Station Road\n",
      "35th Rd => 35th Road\n",
      "Kansai Road, Ambedkar chowk, => Kansai Road, Ambedkar chowk,\n",
      "CITIZEN STREET => CITIZEN STREET\n",
      "Hiranandani Estate Branch => Hiranandani Estate Branch\n",
      "Road No 25 => Road No 25\n",
      "Road No 26 => Road No 26\n",
      "Sector 14, CBD Belapur => Sector 14, CBD Belapur\n",
      "Sona Villa Sector 21 => Sona Villa Sector 21\n",
      "Sector 21 => Sector 21\n",
      "Sector 23 => Sector 23\n",
      "Main St, Hiranandani Gardens, => Main St, Hiranandani Gardens,\n",
      "thane => thane\n",
      "Infinite Corridoor => Infinite Corridoor\n",
      "avanthi vihar colony labhandi raipur => avanthi vihar colony labhandi raipur\n",
      "Kirol Road, Off LBS Marg, Patelwadi => Kirol Road, Off LBS Marg, Patelwadi\n",
      "National Highway 4 => National Highway 4\n",
      "Road No 4 => Road No 4\n",
      "Road Number 4 => Road Number 4\n",
      "RCF Colony Type 4 => RCF Colony Type 4\n",
      "R.S.C. Road Number 4 => R.S.C. Road Number 4\n",
      "Chhatrapati Shivaji Road Number 4 => Chhatrapati Shivaji Road Number 4\n",
      "GV Scheme Road 4 => GV Scheme Road 4\n",
      "MIDC Internal => MIDC Internal\n",
      "Essel World => Essel World\n",
      "A Wing, Abhishek Co Operative Housing Society, Santacruz Eas => A Wing, Abhishek Co Operative Housing Society, Santacruz Eas\n",
      "borivali east => borivali east\n",
      "Peltopherum Road (N S Road No 2) => Peltopherum Road (N S Road No 2)\n",
      "SION TROMBAY ROAD => SION TROMBAY ROAD\n",
      "KOPAR ROAD => KOPAR ROAD\n",
      "MUMBAI VASHI LINK ROAD => MUMBAI VASHI LINK ROAD\n",
      "hiranandani garden => hiranandani garden\n",
      "Road no 9 baingawadi => Road no 9 baingawadi\n",
      "Cama Industrial Estate,GoreGaon(E) => Cama Industrial Estate,GoreGaon(E)\n",
      "bardaan galli => bardaan galli\n",
      "Leela Cliff CHS => Leela Cliff CHS\n",
      "Gandhi Chauk => Gandhi Chauk\n",
      "Road No. 21, Opposite Passport Seva Kendra, => Road No. 21, Opposite Passport Seva Kendra,\n",
      "Dapoda Road, Amita complex, Opposite laxmi Shop => Dapoda Road, Amita complex, Opposite laxmi Shop\n",
      "Prabhat Colony => Prabhat Colony\n",
      "RCF Colony => RCF Colony\n",
      "R.C.F. Colony => R.C.F. Colony\n",
      "Mysore Colony => Mysore Colony\n",
      "mankhurd => mankhurd\n",
      "Plot no.3, Sector 2, Kharghar => Plot no.3, Sector 2, Kharghar\n",
      "Sector 3, Kharghar => Sector 3, Kharghar\n",
      "Sector 50 => Sector 50\n",
      "Lokmanya Tilak Nagar => Lokmanya Tilak Nagar\n",
      "Satya Nagar => Satya Nagar\n",
      "Shirdi Nagar => Shirdi Nagar\n",
      "Gandhi Nagar => Gandhi Nagar\n",
      "General Arun Kumar Vaidya Nagar => General Arun Kumar Vaidya Nagar\n",
      "Manish Nagar => Manish Nagar\n",
      "Road no. 19/20, Opp Sankalp School, Veer Savarkar Nagar => Road no. 19/20, Opp Sankalp School, Veer Savarkar Nagar\n",
      "Laxmi Nagar => Laxmi Nagar\n",
      "pirojshanagar godrej => pirojshanagar godrej\n",
      "jnpt => jnpt\n",
      "Jain Derasar Road, Wadala => Jain Derasar Road, Wadala\n",
      "MIDC RABALE => MIDC RABALE\n",
      "International Airport Sahar => International Airport Sahar\n",
      "Sahar => Sahar\n",
      "Road no.1 => Road no.1\n",
      "PERARAWADI => PERARAWADI\n",
      "IIT Bombay => IIT Bombay\n",
      "Military Road, Marol. Andheri => Military Road, Marol. Andheri\n",
      "Ghodbunder ROad => Ghodbunder Road\n",
      "Road No. 8 => Road No. 8\n",
      "M. G. Road,Bazar Plot => M. G. Road,Bazar Plot\n",
      "Dr Madhukar B. Raut Marg => Dr Madhukar B. Raut Marg\n",
      "Guru Gobind Singh Marg => Guru Gobind Singh Marg\n",
      "Shahid Bhagat Singh Marg => Shahid Bhagat Singh Marg\n",
      "R N Goenka Marg => R N Goenka Marg\n",
      "Srirang Sabde Marg => Srirang Sabde Marg\n",
      "Pirojsha Godrej Marg => Pirojsha Godrej Marg\n",
      "171 Lal Bahadur Shastri Marg => 171 Lal Bahadur Shastri Marg\n",
      "G Talwarkar Marg => G Talwarkar Marg\n",
      "Amit Keshav Nayak Marg => Amit Keshav Nayak Marg\n",
      "Lokmanya Tilak Marg => Lokmanya Tilak Marg\n",
      "Gen A K Vaidya Marg => Gen A K Vaidya Marg\n",
      "Ganapatrao Kadam Marg => Ganapatrao Kadam Marg\n",
      "Palm Beach Marg => Palm Beach Marg\n",
      "KC Marg => KC Marg\n",
      "Bhikoba Waman Pathare Marg => Bhikoba Waman Pathare Marg\n",
      "V. N. Purav Marg => V. N. Purav Marg\n",
      "Rajendra Venkateshwara Rao Marg => Rajendra Venkateshwara Rao Marg\n",
      "N Dharmadhikari Marg => N Dharmadhikari Marg\n",
      "Pandit Madan Mohan Malviya Marg => Pandit Madan Mohan Malviya Marg\n",
      "Waman Tukaram Patil Marg => Waman Tukaram Patil Marg\n",
      "Dadasaheb Phalke Marg => Dadasaheb Phalke Marg\n",
      "Veer Jeejamata Bhosle Marg => Veer Jeejamata Bhosle Marg\n",
      "Nirmaladevi Marg => Nirmaladevi Marg\n",
      "Balshet Madurkar Marg => Balshet Madurkar Marg\n",
      "Cosmopolitan Education Society Marg => Cosmopolitan Education Society Marg\n",
      "rani Sati Marg => rani Sati Marg\n",
      "Jeejabai Bhosle Marg => Jeejabai Bhosle Marg\n",
      "R Thakur Marg => R Thakur Marg\n",
      "Javji Dadaji Marg => Javji Dadaji Marg\n",
      "Ahinsa Marg => Ahinsa Marg\n",
      "Jagannath Shankarsheth Marg => Jagannath Shankarsheth Marg\n",
      "G.D. Ambedkar Marg => G.D. Ambedkar Marg\n",
      "Abaji Marg => Abaji Marg\n",
      "Tukaram Javji Marg => Tukaram Javji Marg\n",
      "D. Pendse Marg => D. Pendse Marg\n",
      "Prakash Thorat Marg => Prakash Thorat Marg\n",
      "Saibaba Mandir Marg => Saibaba Mandir Marg\n",
      "Pradip Dattatraya Samant Marg => Pradip Dattatraya Samant Marg\n",
      "D Joshi Marg => D Joshi Marg\n",
      "Gopal Krushna Gokhale Marg => Gopal Krushna Gokhale Marg\n",
      "Adi Shankaracharya Marg => Adi Shankaracharya Marg\n",
      "P Tandon Marg => P Tandon Marg\n",
      "Sardar Gurucharan Singh Kocher Marg => Sardar Gurucharan Singh Kocher Marg\n",
      "Shankar Ghanekar Marg => Shankar Ghanekar Marg\n",
      "Swatantrya Veer Savarkar Marg => Swatantrya Veer Savarkar Marg\n",
      "Mahatma Phule Marg => Mahatma Phule Marg\n",
      "Nirmaladevi Arunkumar Ahuja Marg => Nirmaladevi Arunkumar Ahuja Marg\n",
      "Lal Bahadur Shastri Marg => Lal Bahadur Shastri Marg\n",
      "Vidyalaya Marg => Vidyalaya Marg\n",
      "Dr Babasaheb Ambedkar Marg => Dr Babasaheb Ambedkar Marg\n",
      "Captain Prakash Pethe Marg => Captain Prakash Pethe Marg\n",
      "Bhakti Vedanta Swami Marg => Bhakti Vedanta Swami Marg\n",
      "Atmaram Bhoir Marg => Atmaram Bhoir Marg\n",
      "A H Wadia Marg => A H Wadia Marg\n",
      "Kadeshwari Marg => Kadeshwari Marg\n",
      "Acharya Anand Rushi Marg => Acharya Anand Rushi Marg\n",
      "P Ramabai Marg => P Ramabai Marg\n",
      "Mahatma Gandhi Marg => Mahatma Gandhi Marg\n",
      "Pradip Dattatra Samant Marg => Pradip Dattatra Samant Marg\n",
      "D K Sandu Marg => D K Sandu Marg\n",
      "Govindji Shroff Marg => Govindji Shroff Marg\n",
      "Mavli Mandal Marg => Mavli Mandal Marg\n",
      "Mahapalika Bhavan Marg => Mahapalika Bhavan Marg\n",
      "Narendra Ballal Marg => Narendra Ballal Marg\n",
      "Khandelwal Marg => Khandelwal Marg\n",
      "Pandurang Budhkar Marg => Pandurang Budhkar Marg\n",
      "N.M Joshi Marg => N.M Joshi Marg\n",
      "Vidya Prasarak High School Marg => Vidya Prasarak High School Marg\n",
      "Ramakrishna Chemburkar Marg => Ramakrishna Chemburkar Marg\n",
      "Maharana Pratap Marg => Maharana Pratap Marg\n",
      "Keshavrao Khadye Marg => Keshavrao Khadye Marg\n",
      "Kale Marg => Kale Marg\n",
      "Ramabai Chemburkar Marg => Ramabai Chemburkar Marg\n",
      "Vithalbhai Patel Marg => Vithalbhai Patel Marg\n",
      "Datta Jagdamba Mandir Marg => Datta Jagdamba Mandir Marg\n",
      "Dharmaveer Marg => Dharmaveer Marg\n",
      "Gopi Tank Marg => Gopi Tank Marg\n",
      "Adi Sankaracharya Marg => Adi Sankaracharya Marg\n",
      "B D Patil Marg => B D Patil Marg\n",
      "SL Raheja Hospital Marg => SL Raheja Hospital Marg\n",
      "Appasaheb Marathe Marg => Appasaheb Marathe Marg\n",
      "P.L. Lokhande Marg => P.L. Lokhande Marg\n",
      "K C Marg => K C Marg\n",
      "LBS Marg => LBS Marg\n",
      "Lions Sol Marg => Lions Sol Marg\n",
      "V.N. Purav Marg => V.N. Purav Marg\n",
      "Laxman Umaji Gadkari Marg => Laxman Umaji Gadkari Marg\n",
      "Lockmanya Talik Marg => Lockmanya Talik Marg\n",
      "Krishnalal Marva Marg => Krishnalal Marva Marg\n",
      "V N Purav Marg => V N Purav Marg\n",
      "Sakharam Patil Marg => Sakharam Patil Marg\n",
      "Sulochana Shetty Marg => Sulochana Shetty Marg\n",
      "Vijay Marg => Vijay Marg\n",
      "M. V. Pandloskar Marg => M. V. Pandloskar Marg\n",
      "V B Phadke Marg => V B Phadke Marg\n",
      "Madhusudan Kalekar Marg => Madhusudan Kalekar Marg\n",
      "m Ambedkar Marg => m Ambedkar Marg\n",
      "Sivram Seth Amritwar Marg => Sivram Seth Amritwar Marg\n",
      "Vidyanidhi Marg => Vidyanidhi Marg\n",
      "Swami Gyan Jiwandas Marg => Swami Gyan Jiwandas Marg\n",
      "N M Joshi Marg => N M Joshi Marg\n",
      "Maulana Shaukat Ali Marg => Maulana Shaukat Ali Marg\n",
      "Krishna Changa Naik Marg => Krishna Changa Naik Marg\n",
      "Rao Khadye Marg => Rao Khadye Marg\n",
      "Sant Gadge Maharaj Marg => Sant Gadge Maharaj Marg\n",
      "Lokhandwala Marg => Lokhandwala Marg\n",
      "Sant Savta Mali Marg => Sant Savta Mali Marg\n",
      "N Bharucha Marg => N Bharucha Marg\n",
      "Mumbai Samachar Marg => Mumbai Samachar Marg\n",
      "Gurunanak Marg => Gurunanak Marg\n",
      "M G Ramchandran Marg => M G Ramchandran Marg\n",
      "Navghar Marg => Navghar Marg\n",
      "PG Kher Marg => PG Kher Marg\n",
      "Pravesh Marg => Pravesh Marg\n",
      "Vihar Lake Marg => Vihar Lake Marg\n",
      "Madhusudhan Kalelkar Marg => Madhusudhan Kalelkar Marg\n",
      "Sharadbhau Acharya Marg => Sharadbhau Acharya Marg\n",
      "Rajarambane Marg => Rajarambane Marg\n",
      "B D Savant Marg => B D Savant Marg\n",
      "Senapati Bapat Marg => Senapati Bapat Marg\n",
      "Aramnagar Part 1, Versova => Aramnagar Part 1, Versova\n",
      "St John's Road, Bandra West => St John's Road, Bandra West\n",
      "JP Road, Versova, Andheri West => JP Road, Versova, Andheri West\n",
      "Pali Market Road, Bandra West => Pali Market Road, Bandra West\n",
      ", Rizvi Park, S V Road, Santacruz West => , Rizvi Park, S V Road, Santacruz West\n",
      "St John's Street, Bandra West => St John's Street, Bandra West\n",
      "Oshiwara, Andheri West => Oshiwara, Andheri West\n",
      "Pali Junction, Dr BR Ambedkar Road, Bandra West => Pali Junction, Dr BR Ambedkar Road, Bandra West\n",
      "16th Road, Bandra West => 16th Road, Bandra West\n",
      "Link Road Extension, Oshiwara, Andheri West => Link Road Extension, Oshiwara, Andheri West\n",
      "Off Link Road, Andheri West => Off Link Road, Andheri West\n",
      "Behind Goregaon Sports Club, Link Road, Malad West => Behind Goregaon Sports Club, Link Road, Malad West\n",
      "J.P Road, 7 Bungalows, Andheri West => J.P Road, 7 Bungalows, Andheri West\n",
      "Goregaon West => Goregaon West\n",
      "St Alexius Road, Bandra West => St Alexius Road, Bandra West\n",
      "Mahatma Phule Rd, Naupada, Thane West => Mahatma Phule Rd, Naupada, Thane West\n",
      "Interface Heights, Off link Road, Malad West => Interface Heights, Off link Road, Malad West\n",
      "Dr B.R Ambedkar Road, Bandra West => Dr B.R Ambedkar Road, Bandra West\n",
      "N Dutta Road, Four Bungalows, Andheri West => N Dutta Road, Four Bungalows, Andheri West\n",
      "JVPD Scheme Circle, Vile Parle West => JVPD Scheme Circle, Vile Parle West\n",
      "Dr BR Ambedkar Road, Bandra West => Dr BR Ambedkar Road, Bandra West\n",
      "Bank Of Maharashtra Lane, Seven Bungalows, Andheri West => Bank Of Maharashtra Lane, Seven Bungalows, Andheri West\n",
      "Gulmohar Road, Juhu Scheme, Vile Parle West => Gulmohar Road, Juhu Scheme, Vile Parle West\n",
      "Maitri Tower,Louiswadi,Thane West => Maitri Tower,Louiswadi,Thane West\n",
      "Gymkhana Road, Borivali West => Gymkhana Road, Borivali West\n",
      "#A-141 Four Bungalows,Lokhandwala Complex Road, Ambovali Village, Andheri West => #A-141 Four Bungalows,Lokhandwala Complex Road, Ambovali Village, Andheri West\n",
      "Dr Ambedkar Road, Bandra West => Dr Ambedkar Road, Bandra West\n",
      "Jankalyan Nagar, Malad West => Jankalyan Nagar, Malad West\n",
      "S.V Road, Goregaon West => S.V Road, Goregaon West\n",
      "Besant Road, Vile Parle West => Besant Road, Vile Parle West\n",
      "St Cyril's Road, Bandra West => St Cyril's Road, Bandra West\n",
      "Four Bungalows, Opposite Bon Bon, JP Road, Andheri West => Four Bungalows, Opposite Bon Bon, JP Road, Andheri West\n",
      "Gulmohar X Road, JVPD Scheme, Vile Parle West => Gulmohar X Road, JVPD Scheme, Vile Parle West\n",
      "4, Lokhandwala Complex Rd, Shastri Nagar, Andheri West => 4, Lokhandwala Complex Rd, Shastri Nagar, Andheri West\n",
      "New Link Road, Andheri West => New Link Road, Andheri West\n",
      "Jai Prakash Road, Versova, Andheri West => Jai Prakash Road, Versova, Andheri West\n",
      "Hill Road, Bandra West => Hill Road, Bandra West\n",
      "New Link Road Andheri West => New Link Road Andheri West\n",
      "Veera Desai Road, Andheri West => Veera Desai Road, Andheri West\n",
      "SAB TV Lane, Off New Link Road, Andheri West => SAB TV Lane, Off New Link Road, Andheri West\n",
      "Panch Marg, off Yari Road, Versova, Andheri West => Panch Marg, off Yari Road, Versova, Andheri West\n",
      "Seven Bungalows Andheri West => Seven Bungalows Andheri West\n",
      "Plot No 24, Off J.P Road, 7 Bungalows, Andheri West => Plot No 24, Off J.P Road, 7 Bungalows, Andheri West\n",
      "Shastrinagar, Andheri West => Shastrinagar, Andheri West\n",
      "Linking Road, Bandra West => Linking Road, Bandra West\n",
      "Panch Marg, Off Yari Road, Versova, Andheri West => Panch Marg, Off Yari Road, Versova, Andheri West\n",
      "G-13, Pranik Chambers, Near Saki Naka Junction, Saki Vihar Road, Marol, Andheri East, Sag Baug, Maro => G-13, Pranik Chambers, Near Saki Naka Junction, Saki Vihar Road, Marol, Andheri East, Sag Baug, Maro\n",
      "Vangani, Nere, Panvel Matheran Road, Wangani Tarf Waje, => Vangani, Nere, Panvel Matheran Road, Wangani Tarf Waje,\n",
      "Dr N A Purandare Marg (Chowpatty Sea Face) => Dr N A Purandare Marg (Chowpatty Sea Face)\n",
      "Chaiti nagar => Chaiti nagar\n",
      "shastri nagar => shastri nagar\n",
      "millat nagar => millat nagar\n",
      "shahkar nagar => shahkar nagar\n",
      "prem nagar => prem nagar\n",
      "residential road of sethiya nagar => residential road of sethiya nagar\n",
      "kamraj nagar => kamraj nagar\n",
      "Behind Vasant Sagar Complex => Behind Vasant Sagar Complex\n",
      "Wockhardt Towers, East Wing, 5th Floor, Bandra Kurla Complex => Wockhardt Towers, East Wing, 5th Floor, Bandra Kurla Complex\n",
      "amruta => amruta\n",
      "# 2556, E Block, Sahakarnagar, Sahakara Nagar, Near Childrens park, Next to Sathya Heritage => # 2556, E Block, Sahakarnagar, Sahakara Nagar, Near Childrens park, Next to Sathya Heritage\n",
      ", Shubhada Building, Firstfloor, Sir Pochkhanwala Road, Worli New => , Shubhada Building, Firstfloor, Sir Pochkhanwala Road, Worli New\n",
      "mohammed ali chawl => mohammed ali chawl\n",
      "Shivaji Chowk Ambernath Station Rd. => Shivaji Chowk Ambernath Station Road\n",
      "Veer Savarkar Flyover;Goregaon Mulund Link Rd. => Veer Savarkar Flyover;Goregaon Mulund Link Road\n",
      "Ambarnath Station Rd. => Ambarnath Station Road\n",
      "Lawn Pathway => Lawn Pathway\n",
      "Seawoods Estates Lawn 3 Pathway => Seawoods Estates Lawn 3 Pathway\n",
      "YMCA Road (Club Back Road) => YMCA Road (Club Back Road)\n",
      "Bhikaji Damaji Patil Marg (Corridor Road) => Bhikaji Damaji Patil Marg (Corridor Road)\n",
      "Magan Nathuram Road (M N Road / Andheri Kurla Road) => Magan Nathuram Road (M N Road / Andheri Kurla Road)\n",
      "Sardar Vallabhbhai Patel Marg (Sandhurst Road) => Sardar Vallabhbhai Patel Marg (Sandhurst Road)\n",
      "V Makarand Ghnekar Marg (Azad Road) => V Makarand Ghnekar Marg (Azad Road)\n",
      "Mathuradas Vasanji Road (Andheri Kurla Road) => Mathuradas Vasanji Road (Andheri Kurla Road)\n",
      "J Boman Behram Marg (Bellasis Road) => J Boman Behram Marg (Bellasis Road)\n",
      "PD Hinduja Road (Khar Pali Road) => PD Hinduja Road (Khar Pali Road)\n",
      "Sitram Jadhav Marg (Sun Mill Road) => Sitram Jadhav Marg (Sun Mill Road)\n",
      "N G Acharya Marg (Chembur Govandi Road) => N G Acharya Marg (Chembur Govandi Road)\n",
      "S B Dhody Marg (Nesbit Road) => S B Dhody Marg (Nesbit Road)\n",
      "Boman Behram Road (Mere Weather Road) => Boman Behram Road (Mere Weather Road)\n",
      "Maratha Mandir Marg (Club Road) => Maratha Mandir Marg (Club Road)\n",
      "V N Purav Marg (Sion Trombay Road) => V N Purav Marg (Sion Trombay Road)\n",
      "Dr KS Hedgewar Road (30th Road) => Dr KS Hedgewar Road (30th Road)\n",
      "Allibhai Premji Marg (Foras Road) => Allibhai Premji Marg (Foras Road)\n",
      "Dr Babasaheb Ambedkar Marg (Vincent Road) => Dr Babasaheb Ambedkar Marg (Vincent Road)\n",
      "Krishna Chandra Road (Hill Road) => Krishna Chandra Road (Hill Road)\n",
      "N M Joshi Marg (Delsle Road) => N M Joshi Marg (Delsle Road)\n",
      "Shahid Bhagat Singh Marg (Lower Colaba Road) => Shahid Bhagat Singh Marg (Lower Colaba Road)\n",
      "Nanabhai Moos Marg (Upper Colaba Road) => Nanabhai Moos Marg (Upper Colaba Road)\n",
      "Nathalal M. Parikh Road (Wodehouse Road) => Nathalal M. Parikh Road (Wodehouse Road)\n",
      "Ramjibhai Kamani Road (Nichol Road) => Ramjibhai Kamani Road (Nichol Road)\n",
      "Dr Annie Besant Road (Love Grove Road) => Dr Annie Besant Road (Love Grove Road)\n",
      "Ramakrishna Chemburkar Marg (Ghatkopar Mahul Road) => Ramakrishna Chemburkar Marg (Ghatkopar Mahul Road)\n",
      "Jai Prakash Road (Versova Road) => Jai Prakash Road (Versova Road)\n",
      "Dr Baba Saheb Ambedkar Marg (Victoria Garden Road) => Dr Baba Saheb Ambedkar Marg (Victoria Garden Road)\n",
      "Master Vinayak Road (Perry Road) => Master Vinayak Road (Perry Road)\n",
      "Ramdas Naik Road (Hill Road) => Ramdas Naik Road (Hill Road)\n",
      "Senapati Bapat Marg (Tulsi Pipe Road) => Senapati Bapat Marg (Tulsi Pipe Road)\n",
      "Patthe Bapurao Marg (Falkland Road) => Patthe Bapurao Marg (Falkland Road)\n",
      "Raja Ram Mohan Roy Road (Charni Road) => Raja Ram Mohan Roy Road (Charni Road)\n",
      "RK Patkar Marg (Waterfield Road) => RK Patkar Marg (Waterfield Road)\n",
      "Baji Pasalkar Marg (Saki Vihar Road) => Baji Pasalkar Marg (Saki Vihar Road)\n",
      "Choitram Gidwani Marg (Kolwada Borla Road) => Choitram Gidwani Marg (Kolwada Borla Road)\n",
      "Saras Baug => Saras Baug\n",
      "Next to Andhei subway, Old Nagardas Road, Andheri (E) => Next to Andhei subway, Old Nagardas Road, Andheri (E)\n",
      "Paranjpe 'B' Scheme road number 1, Vile Parle (E) => Paranjpe 'B' Scheme road number 1, Vile Parle (E)\n",
      "Sector- 06, plot 04 => Sector- 06, plot 04\n",
      "SVP Nagar, Lokhandwala Complex Road, 4 Bungalows,Andheri Lokhandwala => SVP Nagar, Lokhandwala Complex Road, 4 Bungalows,Andheri Lokhandwala\n",
      "Reliance Corporate Park => Reliance Corporate Park\n",
      "Lazarus rd => Lazarus rd\n",
      "Powai => Powai\n",
      "Supreme Business Park, South Avenue Hiranandani Powai => Supreme Business Park, South Avenue Hiranandani Powai\n",
      "Road No 3 => Road No 3\n",
      "Road Number 3 => Road Number 3\n",
      "Irani Wadi Road No 3 => Irani Wadi Road No 3\n",
      "Cross Road No 3 => Cross Road No 3\n",
      "ashok nagar hill no 3 => ashok nagar hill no 3\n",
      "Tejpal Scheme Road 3 => Tejpal Scheme Road 3\n",
      "Marigold Apt. , Panchpakhdi, Opp Nitin co & Honda showroom, => Marigold Apt. , Panchpakhdi, Opp Nitin co & Honda showroom,\n",
      "vasansha market => vasansha market\n",
      "Sector 7 => Sector 7\n",
      "sector 7 => sector 7\n",
      "Road Number 7 => Road Number 7\n",
      "St. George's Hospital Compound => St. George's Hospital Compound\n",
      "Dada Rege Marg aka. Shivaji Park Road no.3 => Dada Rege Marg aka. Shivaji Park Road no.3\n",
      "AGNIMATA TEMPLE => AGNIMATA TEMPLE\n",
      "Chendani koliwada => Chendani koliwada\n",
      "B C Chatterjee Marg (Main Avenue) => B C Chatterjee Marg (Main Avenue)\n",
      "D Saraswati Marg (Central Avenue) => D Saraswati Marg (Central Avenue)\n",
      "Kamalakarpant L Walawalkar Mg => Kamalakarpant L Walawalkar Mg\n",
      "Retibandar, Ghodbunder, Mira Bhayandar => Retibandar, Ghodbunder, Mira Bhayandar\n",
      "J.v.link.rod => J.v.link.rod\n",
      "Jaishankar Yagnik Road (Flank Road West) => Jaishankar Yagnik Road (Flank Road West)\n",
      "Shivaji Nager,T.P.Road,Bhandup (West) => Shivaji Nager,T.P.Road,Bhandup (West)\n",
      "Evershine Nagar, Malad (West) => Evershine Nagar, Malad (West)\n",
      "Nere-Chipale, => Nere-Chipale,\n",
      "Sector 20 => Sector 20\n",
      "Nehru Road Ext => Nehru Road Ext\n",
      "Ghatkopar FOB => Ghatkopar FOB\n",
      "Govandi FOB => Govandi FOB\n",
      "Ashok Kedare Chowk => Ashok Kedare Chowk\n",
      "Chintamani Chowk => Chintamani Chowk\n",
      "Doctor Shyama Prasad Mukherjee Chowk => Doctor Shyama Prasad Mukherjee Chowk\n",
      "datta mandhiar foot path => datta mandhiar foot path\n",
      "90/ A, Near Holy Family Hospital, Hill Rd, Bandra West, Mumbai, Maharashtra => 90/ A, Near Holy Family Hospital, Hill Rd, Bandra West, Mumbai, Maharashtra\n",
      "Sector 21, Nerul, Navi Mumbai, Maharashtra => Sector 21, Nerul, Navi Mumbai, Maharashtra\n",
      "govandi => govandi\n",
      "Nariman Point => Nariman Point\n",
      "Backbay Reclamation, Nariman Point => Backbay Reclamation, Nariman Point\n",
      "9 A nerul, Uran Road, Sector 19A, Nerul, Navi Mumbai, => 9 A nerul, Uran Road, Sector 19A, Nerul, Navi Mumbai\n",
      "JK Sawant Marg, => JK Sawant Marg\n",
      "Karjat, MH => Karjat, MH\n",
      "Paud Lonavla Road, Mulshi => Paud Lonavla Road, Mulshi\n",
      "Dr. Dadabhai Naoroji => Dr. Dadabhai Naoroji\n",
      "l.j.road => l.j.road\n",
      "Sector 6 Market => Sector 6 Market\n",
      "malad west => malad west\n",
      "Sumer castle => Sumer castle\n",
      "M.G.Road => M.G.Road\n",
      "Ramchandra Gali => Ramchandra Gali\n",
      "new link road => new link Road\n",
      "st anthony church  road => st anthony church  Road\n",
      "mohd ali road => mohd ali Road\n",
      "ghatkapar/mankhurd link road => ghatkapar/mankhurd link Road\n",
      "jay bhavani mata road => jay bhavani mata Road\n",
      "LBS Marg 90 feet road => LBS Marg 90 feet Road\n",
      "2nd floor swadesi market kalbadevi road => 2nd floor swadesi market kalbadevi Road\n",
      "saba road => saba Road\n",
      "video road => video Road\n",
      "Shankar Pawse road => Shankar Pawse Road\n",
      "devidas ext road => devidas ext Road\n",
      "tanki road => tanki Road\n",
      "father peter periera road => father peter periera Road\n",
      "24th road => 24th Road\n",
      "Godrej Hill road => Godrej Hill Road\n",
      "4th road => 4th Road\n",
      "Gograswadi, Patharli road => Gograswadi, Patharli Road\n",
      "Kamothe main road => Kamothe main Road\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Barave gaon road => Barave gaon Road\n",
      "Sher-e-Punjab road => Sher-e-Punjab Road\n",
      "90 feet road => 90 feet Road\n",
      "M. G. cross road => M. G. cross Road\n",
      "Mulshi road => Mulshi Road\n",
      "l.u.gadkari road => l.u.gadkari Road\n",
      "90ft road => 90ft Road\n",
      "Manish Market service road => Manish Market service Road\n",
      "kirol road => kirol Road\n",
      "mohmmad road => mohmmad Road\n",
      "Raghunath Vihar road => Raghunath Vihar Road\n",
      "bharda wadi road => bharda wadi Road\n",
      "off Ghodhbunder road => off Ghodhbunder Road\n",
      "Vashi station road => Vashi station Road\n",
      "NMMT road => NMMT Road\n",
      "bhivandi-kalher road => bhivandi-kalher Road\n",
      "Harbour Heights service road => Harbour Heights service Road\n",
      "amrutnager road => amrutnager Road\n",
      "st anthony church road => st anthony church Road\n",
      "Juhu Tara road => Juhu Tara Road\n",
      "market road => market Road\n",
      "Father peter periera road => Father peter periera Road\n",
      "road => Road\n",
      "Khokhani Estate, Opp. Neelkanth Industrial Estate, Near Saibaba Mandir, Sativali.Palgarh: => Khokhani Estate, Opp. Neelkanth Industrial Estate, Near Saibaba Mandir, Sativali.Palgarh:\n",
      "Parvati Cross => Parvati Cross\n",
      "Prabhat colony => Prabhat colony\n",
      "railway colony => railway colony\n",
      "Datar colony => Datar colony\n",
      "sec-2, Khanda colony => sec-2, Khanda colony\n",
      "Amar Mahal , Chembur => Amar Mahal , Chembur\n",
      "Tilak Nagar, Chembur => Tilak Nagar, Chembur\n",
      "Azad Nagar, Sandoz Baug Post, => Azad Nagar, Sandoz Baug Post,\n",
      "Bora Bazaar Street, Siddhi Vinayak CHS, Fort, CST => Bora Bazaar Street, Siddhi Vinayak CHS, Fort, CST\n",
      "Indradarshan Crossroad => Indradarshan Crossroad\n",
      "Vashi => Vashi\n",
      "m ward marg => m ward marg\n",
      "vidyarthi marg => vidyarthi marg\n",
      "Kailash Esplanade, LBS marg => Kailash Esplanade, LBS marg\n",
      "sitaram palturam murai marg => sitaram palturam murai marg\n",
      "H.N.A azmi marg => H.N.A azmi marg\n",
      "P L Lokhande marg => P L Lokhande marg\n",
      "Near dwarka shop p.l.lokhande marg => Near dwarka shop p.l.lokhande marg\n",
      "Hathayogi nikam guruji marg => Hathayogi nikam guruji marg\n",
      "gokul marg => gokul marg\n",
      "Gokhale Road North => Gokhale Road North\n",
      "Near Dukes Factory, Waman Tukaram Patil Marg (Ghatkopar Sewri Wadala Road), Deonar => Near Dukes Factory, Waman Tukaram Patil Marg (Ghatkopar Sewri Wadala Road), Deonar\n",
      "Sion Trombay Road, Deonar => Sion Trombay Road, Deonar\n",
      "ashirwad hospital => ashirwad hospital\n",
      "sant nirankari => sant nirankari\n",
      "Baudi Lane, Orlem Malad => Baudi Lane, Orlem Malad\n",
      "Dominic Lane, Orlem Malad => Dominic Lane, Orlem Malad\n",
      "Veer Vithaldas Chandan Street (De Souza Street) => Veer Vithaldas Chandan Street (De Souza Street)\n",
      "Road No 2 => Road No 2\n",
      "Pokhran Road 2 => Pokhran Road 2\n",
      "D.P. road No. 2 => D.P. road No. 2\n",
      "Road Number 2 => Road Number 2\n",
      "Cross Road No 2 => Cross Road No 2\n",
      "Hanuman Cross Rd. No. 2 => Hanuman Cross Rd. No. 2\n",
      "L. J. Cross Road No. 2 => L. J. Cross Road No. 2\n",
      "Gawde Road 2 => Gawde Road 2\n",
      "Sector 6 => Sector 6\n",
      "J.V.P.D Scheme Road 6 => J.V.P.D Scheme Road 6\n",
      "Chitalsar => Chitalsar\n",
      "Hendrapada => Hendrapada\n",
      "Kalbadevi => Kalbadevi\n",
      "Nana Shankar Rd, Retibandar, Ghodbunder, Mira Bhayandar, => Nana Shankar Rd, Retibandar, Ghodbunder, Mira Bhayandar,\n",
      "Sodawala Lane => Sodawala Lane\n",
      "Ice Factory Lane => Ice Factory Lane\n",
      "St. Anne's Lane => St. Anne's Lane\n",
      "Khetwadi 12th Lane => Khetwadi 12th Lane\n",
      "1st Pasta Lane => 1st Pasta Lane\n",
      "Sadashiva Cross Lane => Sadashiva Cross Lane\n",
      "Dominic Lane => Dominic Lane\n",
      "Juhu Lane => Juhu Lane\n",
      "Dontad Cross Lane => Dontad Cross Lane\n",
      "Khetwadi 5th Lane => Khetwadi 5th Lane\n",
      "14th Lane => 14th Lane\n",
      "Sindhi Lane => Sindhi Lane\n",
      "Goshala Lane => Goshala Lane\n",
      "Baudi Lane => Baudi Lane\n",
      "Thanawala Lane => Thanawala Lane\n",
      "4th Pasta Lane => 4th Pasta Lane\n",
      "Shankar Lane => Shankar Lane\n",
      "Cama Lane => Cama Lane\n",
      "Raviraj Oberoi Complex, Inside SAB TV Lane => Raviraj Oberoi Complex, Inside SAB TV Lane\n",
      "sector 34 B => sector 34 B\n",
      "Zainee Shopping Centre, Opposite Akash Talkies => Zainee Shopping Centre, Opposite Akash Talkies\n",
      "Gokuldham => Gokuldham\n",
      "Satelliete Garden, Gokuldham => Satelliete Garden, Gokuldham\n",
      "HATTIMOHALLA => HATTIMOHALLA\n",
      "Niedersachsenring => Niedersachsenring\n",
      "T. T. C Industrial Area, M. I. D. C, Rabale, Navi Mumbai - 400701, Amrut Dham, Gothivali Village, Sector 30, Ghansoli => T. T. C Industrial Area, M. I. D. C, Rabale, Navi Mumbai - 400701, Amrut Dham, Gothivali Village, Sector 30, Ghansoli\n",
      "Ghansoli => Ghansoli\n",
      "Evershine Nagar, Malad (W) => Evershine Nagar, Malad (W)\n",
      "King's Circle Flyover => King's Circle Flyover\n",
      "Marine Lines Flyover => Marine Lines Flyover\n",
      "Majiwada Flyover => Majiwada Flyover\n",
      "Karjat => Karjat\n",
      "Tilakwadi, Belagavi Belgaum => Tilakwadi, Belagavi Belgaum\n",
      "Road No.6 => Road No.6\n",
      "Additional Kalyan Bhiwandi MIDC Industrial Area, Plot No. 1, Village Kone, => Additional Kalyan Bhiwandi MIDC Industrial Area, Plot No. 1, Village Kone,\n",
      "Road No.4 => Road No.4\n",
      "aqsa masjid => aqsa masjid\n",
      "Plot No.2 => Plot No.2\n",
      "Hanuman Cross Rd. No.1 => Hanuman Cross Rd. No.1\n",
      "L.J. Road No.1 => L.J. Road No.1\n",
      "Nagdevi Path => Nagdevi Path\n",
      "D N Path => D N Path\n",
      "Shivaji Path => Shivaji Path\n",
      "Adi Murzban Path => Adi Murzban Path\n",
      "Lt Prakash Kotnis Path => Lt Prakash Kotnis Path\n",
      "shree satidham => shree satidham\n",
      "neelam nagar, phase-2 => neelam nagar, phase-2\n",
      "jacob circle => jacob circle\n",
      "Bandra Kurla Complex Road => Bandra Kurla Complex Road\n",
      "Wing Mess Road => Wing Mess Road\n",
      "Dharavi Main Road => Dharavi Main Road\n",
      "Saraswati Road => Saraswati Road\n",
      "Ganesh Gawde Road => Ganesh Gawde Road\n",
      "Rambhau Salgaonkar Road => Rambhau Salgaonkar Road\n",
      "Maharshi Karve Road => Maharshi Karve Road\n",
      "Off Veera Desai Road => Off Veera Desai Road\n",
      "Balaji Mandir Road => Balaji Mandir Road\n",
      "33rd Road => 33rd Road\n",
      "Raja Shivaji School Road => Raja Shivaji School Road\n",
      "90 Feet Road => 90 Feet Road\n",
      "Powai Road => Powai Road\n",
      "Jame Jamshed Road => Jame Jamshed Road\n",
      "Pipe Line Road => Pipe Line Road\n",
      "Postal Colony Road => Postal Colony Road\n",
      "Bhau Daji Road => Bhau Daji Road\n",
      "Chhatrapati Shivaji Maharaj Road => Chhatrapati Shivaji Maharaj Road\n",
      "Charat Singh Road => Charat Singh Road\n",
      "Thane Ghodbunder Road => Thane Ghodbunder Road\n",
      "Agasan Road => Agasan Road\n",
      "Mohammed Ali Road => Mohammed Ali Road\n",
      "Zaver Road => Zaver Road\n",
      "Matheran Road => Matheran Road\n",
      "Bhandarkar Road => Bhandarkar Road\n",
      "Deonar Road => Deonar Road\n",
      "Mahakali Caves Road => Mahakali Caves Road\n",
      "Sarojini Naidu Road => Sarojini Naidu Road\n",
      "VIjay Raut Road => VIjay Raut Road\n",
      "Bharucha Road => Bharucha Road\n",
      "Mumbai - Pune Road => Mumbai - Pune Road\n",
      "Kopar Road => Kopar Road\n",
      "Dr BR Ambedkar Road => Dr BR Ambedkar Road\n",
      "Poddar Road => Poddar Road\n",
      "60 Feet Road => 60 Feet Road\n",
      "Navghar Road => Navghar Road\n",
      "Lajpatrai Road => Lajpatrai Road\n",
      "Dr. Peter Dias Road => Dr. Peter Dias Road\n",
      "Behind M.G. Road => Behind M.G. Road\n",
      "St Dominic Street;7th Road => St Dominic Street;7th Road\n",
      "St Louis Convent Road => St Louis Convent Road\n",
      "Chendani Bunder Road => Chendani Bunder Road\n",
      "Chaphekar Road => Chaphekar Road\n",
      "LT Road => LT Road\n",
      "Vichumbe Road => Vichumbe Road\n",
      "Kherwadi Road => Kherwadi Road\n",
      "Purushottam Kheraj (Dumping) Road => Purushottam Kheraj (Dumping) Road\n",
      "Gyan Vikas Road => Gyan Vikas Road\n",
      "Uran-Karanja Road => Uran-Karanja Road\n",
      "Mia Mohd Chhotani Road => Mia Mohd Chhotani Road\n",
      "Fisheries Institute Road => Fisheries Institute Road\n",
      "R H Bhojraj Road => R H Bhojraj Road\n",
      "Juhu Versova Link Road => Juhu Versova Link Road\n",
      "Oberoi Complex, Off New Link Road => Oberoi Complex, Off New Link Road\n",
      "Miragaon Road => Miragaon Road\n",
      "Dattapada Road => Dattapada Road\n",
      "Bhandup Station Road => Bhandup Station Road\n",
      "Kalyan Badlapur Road => Kalyan Badlapur Road\n",
      "Torana Apartments Sahar Road => Torana Apartments Sahar Road\n",
      "D P Road => D P Road\n",
      "Napean Sea Road => Napean Sea Road\n",
      "Valvan-Bhangarvadi Road => Valvan-Bhangarvadi Road\n",
      "Khatau Road => Khatau Road\n",
      "B Road => B Road\n",
      "Ghodbandar Road => Ghodbandar Road\n",
      "JNPT Road => JNPT Road\n",
      "3rd Cross Road => 3rd Cross Road\n",
      "Maroshi Village Road => Maroshi Village Road\n",
      "4, Lolkhandwala Complex Road => 4, Lolkhandwala Complex Road\n",
      "Veer Nariman Road => Veer Nariman Road\n",
      "7, jamshedji Tata Road => 7, jamshedji Tata Road\n",
      "Cawasji Jahangir Road => Cawasji Jahangir Road\n",
      "Ghansham Gupte Road => Ghansham Gupte Road\n",
      "Shivtirth Co Operative Housing Society Ltd, Madhukar Kalekar Road => Shivtirth Co Operative Housing Society Ltd, Madhukar Kalekar Road\n",
      "CD Barfiwala Road => CD Barfiwala Road\n",
      "Golibar Road => Golibar Road\n",
      "Fatak Road => Fatak Road\n",
      "B Cabin Road => B Cabin Road\n",
      "Almeda Road => Almeda Road\n",
      "Asha Nagar Road => Asha Nagar Road\n",
      "Vangani, Nere, Panvel Matheran Road => Vangani, Nere, Panvel Matheran Road\n",
      "Jogeshwari Vikhroli Link Road => Jogeshwari Vikhroli Link Road\n",
      "Shanti Vidya Nagri Road => Shanti Vidya Nagri Road\n",
      "Datta Mandir Road => Datta Mandir Road\n",
      "Hazarimal Somani Road => Hazarimal Somani Road\n",
      "Goregaon Mulund Link Road => Goregaon Mulund Link Road\n",
      "Vithabhai Patel Road => Vithabhai Patel Road\n",
      "Eksar Road => Eksar Road\n",
      "Ghodbundar Varsarva Naka Road => Ghodbundar Varsarva Naka Road\n",
      "St. Theresa's Road => St. Theresa's Road\n",
      "Jagnnath Mandir Road => Jagnnath Mandir Road\n",
      "Tilak Vidyalaya Road => Tilak Vidyalaya Road\n",
      "Parsik Hill Road => Parsik Hill Road\n",
      "Shrikant Dhadwe Road => Shrikant Dhadwe Road\n",
      "Darave Road => Darave Road\n",
      "Naya Nagar Road => Naya Nagar Road\n",
      "Opposite Oshiwara Bus Depot, New Link Road => Opposite Oshiwara Bus Depot, New Link Road\n",
      "TEC Niwas Sthan Road => TEC Niwas Sthan Road\n",
      "Jivdani Road => Jivdani Road\n",
      "Rahul Estate Road => Rahul Estate Road\n",
      "Adarsh Nagar, New Link Road => Adarsh Nagar, New Link Road\n",
      "Kharghar Station Road => Kharghar Station Road\n",
      "Upvan Lake Road => Upvan Lake Road\n",
      "Raheja Vihar Circular Road => Raheja Vihar Circular Road\n",
      "Dr E Moses Road => Dr E Moses Road\n",
      "Jain Mandir Road => Jain Mandir Road\n",
      "N.S.S Road => N.S.S Road\n",
      "Sewri-Chembur Road => Sewri-Chembur Road\n",
      "Mount Mary Road => Mount Mary Road\n",
      "Tulsidham Road => Tulsidham Road\n",
      "M D Keni Road => M D Keni Road\n",
      "Veera Desai Road => Veera Desai Road\n",
      "Jehangir Vimadalal Road => Jehangir Vimadalal Road\n",
      "Pandit Jawaharlal Nehru Road => Pandit Jawaharlal Nehru Road\n",
      "Dr KS Hedgewar Road => Dr KS Hedgewar Road\n",
      "Cawasji Patel Road => Cawasji Patel Road\n",
      "Deonar Govandi Road => Deonar Govandi Road\n",
      "Vijay Raut Road => Vijay Raut Road\n",
      "Swami Vivekanand Road => Swami Vivekanand Road\n",
      "Mulund Court Road => Mulund Court Road\n",
      "Mira Bhayandar Road => Mira Bhayandar Road\n",
      "Arthur Bunder Road => Arthur Bunder Road\n",
      "Sewaram Lalwani Road => Sewaram Lalwani Road\n",
      "Sai Baba Mandir Road => Sai Baba Mandir Road\n",
      "Juhu Tara Road => Juhu Tara Road\n",
      "Subhash Chandra Bose Road => Subhash Chandra Bose Road\n",
      "Garden Road => Garden Road\n",
      "Gokhale Road => Gokhale Road\n",
      "Tula Road => Tula Road\n",
      "Kherani Road => Kherani Road\n",
      "Karjat Road => Karjat Road\n",
      "Dr. Raghunath Road => Dr. Raghunath Road\n",
      "Lady Jamshedji Road => Lady Jamshedji Road\n",
      "N.S.S. Road => N.S.S. Road\n",
      "Uttan Road => Uttan Road\n",
      "D Road => D Road\n",
      "Barrister Rajni Patel Road => Barrister Rajni Patel Road\n",
      "St John Baptist Road => St John Baptist Road\n",
      "Service Road => Service Road\n",
      "Mathurdas Road => Mathurdas Road\n",
      "Pillai College Road => Pillai College Road\n",
      "New Link Road => New Link Road\n",
      "Kalamboli Link Road => Kalamboli Link Road\n",
      "A Merchant Road => A Merchant Road\n",
      "Panvel Creek Bridge Road => Panvel Creek Bridge Road\n",
      "Saint George Road => Saint George Road\n",
      "Nanabhai Moos Road => Nanabhai Moos Road\n",
      "Shantilal Modi Road => Shantilal Modi Road\n",
      "Kanjur Village Road => Kanjur Village Road\n",
      "Chatrapati Shivaji Road => Chatrapati Shivaji Road\n",
      "Andheri Link Road => Andheri Link Road\n",
      "Narayan Nagar Road => Narayan Nagar Road\n",
      "Dadasaheb Gaikwad Road => Dadasaheb Gaikwad Road\n",
      "Laxmi Plaza,  Laxmi Industrial Estate,  New Link Road => Laxmi Plaza,  Laxmi Industrial Estate,  New Link Road\n",
      "M G Road => M G Road\n",
      "Sir Dorab Tata Road => Sir Dorab Tata Road\n",
      "Vasai Station Road => Vasai Station Road\n",
      "JB Nagar Road => JB Nagar Road\n",
      "Senapati Bapat Road => Senapati Bapat Road\n",
      "Chuim Village Road => Chuim Village Road\n",
      "Malad Link Road => Malad Link Road\n",
      "Shahid Bhagat Singh Road => Shahid Bhagat Singh Road\n",
      "Cheeta Camp Market Road => Cheeta Camp Market Road\n",
      "Vaikunthlal Mehta Road => Vaikunthlal Mehta Road\n",
      "S Central Road => S Central Road\n",
      "St.Martin's Road => St.Martin's Road\n",
      ", Naveketan Industrial Estate, Mahakali Caves Road => , Naveketan Industrial Estate, Mahakali Caves Road\n",
      "Chapel Road => Chapel Road\n",
      "Bal Govind Das Road => Bal Govind Das Road\n",
      "Linking Road => Linking Road\n",
      "Boran Road => Boran Road\n",
      "Madame Cama Road => Madame Cama Road\n",
      "Yari Road => Yari Road\n",
      "Gladys Alwares Road => Gladys Alwares Road\n",
      "Vidhan Bhavan Road => Vidhan Bhavan Road\n",
      "Parsi Wada Road => Parsi Wada Road\n",
      "New Golden Nest Road => New Golden Nest Road\n",
      "Ranade Road => Ranade Road\n",
      "Charat Singh Colony Road => Charat Singh Colony Road\n",
      "28th Road => 28th Road\n",
      "Ramratan Trivedi Road => Ramratan Trivedi Road\n",
      "29th Road => 29th Road\n",
      "16th Road => 16th Road\n",
      "Lady Jehangir Road => Lady Jehangir Road\n",
      "R B Kadam Road => R B Kadam Road\n",
      "Bapu Bagve Road => Bapu Bagve Road\n",
      "17th Road => 17th Road\n",
      "Thakur complex Road => Thakur complex Road\n",
      "Ram Mandir Road => Ram Mandir Road\n",
      "Jarimari Road => Jarimari Road\n",
      "Central Hospital Road => Central Hospital Road\n",
      "Mulshi Road => Mulshi Road\n",
      "Sayed Abdullah Brevi Road => Sayed Abdullah Brevi Road\n",
      "Panvel Bypass Uran Road => Panvel Bypass Uran Road\n",
      "Dr. Annie Besant Road => Dr. Annie Besant Road\n",
      "Sardar Vallabhbhai Patel Road => Sardar Vallabhbhai Patel Road\n",
      "Mandlik Road => Mandlik Road\n",
      "19th Road => 19th Road\n",
      "25th Road => 25th Road\n",
      "Global Prgoda Road => Global Prgoda Road\n",
      "Apeejay Road => Apeejay Road\n",
      "Gorai Borivali Road => Gorai Borivali Road\n",
      "Kores Road => Kores Road\n",
      "Church Road => Church Road\n",
      "7th Cross Road => 7th Cross Road\n",
      "Giriz Salbhat Road => Giriz Salbhat Road\n",
      "Wodehouse Road => Wodehouse Road\n",
      "Jamnalal Bajaj Road => Jamnalal Bajaj Road\n",
      "Kohinoor - Vidyavihar Road => Kohinoor - Vidyavihar Road\n",
      "212 Pali Market Road => 212 Pali Market Road\n",
      "Chandavarkar Road => Chandavarkar Road\n",
      "Raja Bahadur Compound Road => Raja Bahadur Compound Road\n",
      "Goshala Road => Goshala Road\n",
      "M Khana Road => M Khana Road\n",
      "Palm Hill Resort Road => Palm Hill Resort Road\n",
      "Swimming Pool Road => Swimming Pool Road\n",
      "Veronica Road => Veronica Road\n",
      "Shankar Mandir Road => Shankar Mandir Road\n",
      "Parulekar Road => Parulekar Road\n",
      "S.V. Road => S.V. Road\n",
      "Naupada Vidya Vihar Road => Naupada Vidya Vihar Road\n",
      "New Kasheli Road => New Kasheli Road\n",
      "Tikujini Wadi Road => Tikujini Wadi Road\n",
      "JP Nagar Road => JP Nagar Road\n",
      "Ram Singh Road => Ram Singh Road\n",
      "9th Road => 9th Road\n",
      "St. Martin's Road => St. Martin's Road\n",
      "Tungareshwar Road => Tungareshwar Road\n",
      "150 Feet Road => 150 Feet Road\n",
      "31st Road => 31st Road\n",
      "Walkeswar Road => Walkeswar Road\n",
      "Veer Savarkar Road => Veer Savarkar Road\n",
      "Byramji Jeejeebhoy Road => Byramji Jeejeebhoy Road\n",
      "Dayaldas Road => Dayaldas Road\n",
      "Mahatma Gandhi Road => Mahatma Gandhi Road\n",
      "Saint Andrew Road => Saint Andrew Road\n",
      "Mithaghar Road => Mithaghar Road\n",
      "Chunabhatti Road => Chunabhatti Road\n",
      "Railway Colony Road => Railway Colony Road\n",
      "Oshiwara Link Road => Oshiwara Link Road\n",
      "Andheri-Kurla Road => Andheri-Kurla Road\n",
      "Hanuman Road => Hanuman Road\n",
      "M Mehta Road => M Mehta Road\n",
      "Gorai Link Road => Gorai Link Road\n",
      "Baptista Road => Baptista Road\n",
      "IA Project Road => IA Project Road\n",
      "Cardinal Gracias Road => Cardinal Gracias Road\n",
      "6th Road => 6th Road\n",
      "Anik Wadala Link Road => Anik Wadala Link Road\n",
      "Hill Road => Hill Road\n",
      "Tembhipada Road => Tembhipada Road\n",
      "Panvel Uran Road => Panvel Uran Road\n",
      "Mount Carmel Road => Mount Carmel Road\n",
      "Hanuman Road,M.g. Road => Hanuman Road,M.g. Road\n",
      "Gulmohar Road => Gulmohar Road\n",
      "Kirol Road => Kirol Road\n",
      "Besant Road => Besant Road\n",
      "Barkat Ali Dargah Road => Barkat Ali Dargah Road\n",
      "Santa Cruz – Chembur Link Road => Santa Cruz – Chembur Link Road\n",
      "TL Wasvani Road => TL Wasvani Road\n",
      "Central Avenue New Mandala Road => Central Avenue New Mandala Road\n",
      "Ramakant Kadam Road => Ramakant Kadam Road\n",
      "22nd Road => 22nd Road\n",
      "Sonawala Road => Sonawala Road\n",
      "Hari Shankar Joshi Road => Hari Shankar Joshi Road\n",
      "Main Gate Road => Main Gate Road\n",
      "Shell Colony Road => Shell Colony Road\n",
      "Dr Ambedkar Road => Dr Ambedkar Road\n",
      "4th Road => 4th Road\n",
      "Vithalbhai Patel Road => Vithalbhai Patel Road\n",
      "1st Road => 1st Road\n",
      "Naupada Road => Naupada Road\n",
      "T Jivraj Road => T Jivraj Road\n",
      "Walji Ladha Road => Walji Ladha Road\n",
      "M.G.Rd. & Hanuman Road => M.G.Rd. & Hanuman Road\n",
      "Saint Leos Road => Saint Leos Road\n",
      "Manson Road => Manson Road\n",
      "Chinchowali Bandar Road => Chinchowali Bandar Road\n",
      "30th Road => 30th Road\n",
      "Link Road => Link Road\n",
      "Manipada Road => Manipada Road\n",
      "Grand Road => Grand Road\n",
      "Sea Face Road => Sea Face Road\n",
      "Station Road => Station Road\n",
      "Azad Road => Azad Road\n",
      "Rebello Road => Rebello Road\n",
      "Mathuradas Vasanji Road => Mathuradas Vasanji Road\n",
      "Saat Bangla Road => Saat Bangla Road\n",
      "Agra Road => Agra Road\n",
      "Railway fatak Road => Railway fatak Road\n",
      "P.D'Mello Road => P.D'Mello Road\n",
      "Panday Road => Panday Road\n",
      "Temba Road => Temba Road\n",
      "Park Avenue Road => Park Avenue Road\n",
      "GD Somani Road => GD Somani Road\n",
      "Sheikh Misari Road => Sheikh Misari Road\n",
      "Kolshet Road => Kolshet Road\n",
      "Bhandup Village Road => Bhandup Village Road\n",
      "Kanheri Caves Road => Kanheri Caves Road\n",
      "Salt Pan Road => Salt Pan Road\n",
      "Doctor Homi Bhaba Road => Doctor Homi Bhaba Road\n",
      "15th Road => 15th Road\n",
      "Tullock Road => Tullock Road\n",
      "Ghodbunder Road => Ghodbunder Road\n",
      "7th Road => 7th Road\n",
      "Off Palm Beach Road => Off Palm Beach Road\n",
      "Dinshaw Wacha Road => Dinshaw Wacha Road\n",
      "Mamledar wadi Road => Mamledar wadi Road\n",
      "Kalba Devi Road => Kalba Devi Road\n",
      "23rd Road => 23rd Road\n",
      "Tata Nagar Road => Tata Nagar Road\n",
      "Thane Belapur Road => Thane Belapur Road\n",
      "JBM Road => JBM Road\n",
      "Brahmagiri Road => Brahmagiri Road\n",
      "14th Road => 14th Road\n",
      "Ramdas Naik Road => Ramdas Naik Road\n",
      "Almeida Park Road => Almeida Park Road\n",
      "11 Altamount Road => 11 Altamount Road\n",
      "Boraspada Road => Boraspada Road\n",
      "21st Road => 21st Road\n",
      "Sahar Elevated Road => Sahar Elevated Road\n",
      "New Hall Road => New Hall Road\n",
      "Dr SM Pandit Road => Dr SM Pandit Road\n",
      "Off Yari Road => Off Yari Road\n",
      "Andheri Kurla Road => Andheri Kurla Road\n",
      "Deonar Farm Road => Deonar Farm Road\n",
      "CIVIC Central Road => CIVIC Central Road\n",
      "Lokmanya Tilak Road => Lokmanya Tilak Road\n",
      "Tejpal Scheme Road => Tejpal Scheme Road\n",
      "5th Road => 5th Road\n",
      "St, Martin's Road => St, Martin's Road\n",
      "Parel Tank Road => Parel Tank Road\n",
      "Mumba Devi Road => Mumba Devi Road\n",
      "Dr. Moose Road => Dr. Moose Road\n",
      "Om Nagar Road => Om Nagar Road\n",
      "16 Sprott Road => 16 Sprott Road\n",
      "Off New Link Road => Off New Link Road\n",
      "Shiv Mandir Road => Shiv Mandir Road\n",
      "Carter Road => Carter Road\n",
      "Dr Rajendra Prasad Road => Dr Rajendra Prasad Road\n",
      "Ceasar Road => Ceasar Road\n",
      "Lala Devidayal Road => Lala Devidayal Road\n",
      "Ghatkopar Mahul Road => Ghatkopar Mahul Road\n",
      "3rd Road => 3rd Road\n",
      "Juhu Road => Juhu Road\n",
      "Bellasis Road => Bellasis Road\n",
      "Dinshaw Wachha Road => Dinshaw Wachha Road\n",
      "Chembur Station Road => Chembur Station Road\n",
      "Babanrao Kulkarni Road => Babanrao Kulkarni Road\n",
      "Purushottam Kheraj Road => Purushottam Kheraj Road\n",
      "Manuel Gonsalves Road => Manuel Gonsalves Road\n",
      "Vasai-Gass Road => Vasai-Gass Road\n",
      "I A Project Road => I A Project Road\n",
      "General Jagannath Bhosle Road => General Jagannath Bhosle Road\n",
      "C S Link Road => C S Link Road\n",
      "Dr Dadabhai Naoroji Road => Dr Dadabhai Naoroji Road\n",
      "Ghodbunder Fort Road => Ghodbunder Fort Road\n",
      "LBS Road => LBS Road\n",
      "Paud Lonavala Road, Hadshi => Paud Lonavala Road, Hadshi\n",
      "RSC 11 => RSC 11\n",
      "Road Number 11 => Road Number 11\n",
      "Sector 10 => Sector 10\n",
      "Road No 10 => Road No 10\n",
      "N S Road No. 13 => N S Road No. 13\n",
      "road No. 13 => road No. 13\n",
      "Sector 13 => Sector 13\n",
      "Road No. 13 => Road No. 13\n",
      "Road No 13 => Road No 13\n",
      "Sector 12 => Sector 12\n",
      "Road Number 16 => Road Number 16\n",
      "Sector 19 => Sector 19\n",
      "Road No 19 => Road No 19\n",
      "Sector 18 => Sector 18\n",
      "chembur stn => chembur stn\n",
      "Hill Road, Ranwar => Hill Road, Ranwar\n",
      "Sindhi Society => Sindhi Society\n",
      "45, Tarun Bharat Co Op Society => 45, Tarun Bharat Co Op Society\n",
      "Tarun Bharat Society => Tarun Bharat Society\n",
      "Police Officers Progressive Co-operative Housing Society => Police Officers Progressive Co-operative Housing Society\n",
      "c.s.t street => c.s.t street\n",
      "Yogiraj street => Yogiraj street\n",
      "khandia street => khandia street\n",
      "Zaoba Wadi => Zaoba Wadi\n",
      "Kotachi Wadi => Kotachi Wadi\n",
      "Kele Wadi => Kele Wadi\n",
      "Navi Wadi => Navi Wadi\n",
      "str => str\n",
      "Road Number 32 => Road Number 32\n",
      "Gulmohar Road (NS Road No 1) => Gulmohar Road (NS Road No 1)\n",
      "26th Road (Sumatilal Manilal Shah Marg) => 26th Road (Sumatilal Manilal Shah Marg)\n",
      "27th Road (Pandurang Ashram Marg) => 27th Road (Pandurang Ashram Marg)\n",
      "Turner Road (Gurunanak Marg) => Turner Road (Gurunanak Marg)\n",
      "Sion Panvel Highway (Yashwantrao Chavan Marg) => Sion Panvel Highway (Yashwantrao Chavan Marg)\n",
      "Sutar Chawl => Sutar Chawl\n",
      "Teen Hath Naka => Teen Hath Naka\n",
      "Saki Naka => Saki Naka\n",
      "Central Avenue => Central Avenue\n",
      "South Avenue => South Avenue\n",
      "North Avenue => North Avenue\n",
      "Khadilkar Marg (Kandewadi Lane) => Khadilkar Marg (Kandewadi Lane)\n",
      "Green City => Green City\n",
      "Sector 35D => Sector 35D\n",
      "PARVATI CROSS => PARVATI CROSS\n",
      "Hindcycle Road, Worli, Mumbai => Hindcycle Road, Worli, Mumbai\n",
      "Janki Kutir Juhu Church Road, Mumbai => Janki Kutir Juhu Church Road, Mumbai\n",
      "Liberty Tower, Plot No. K 10, Behind Reliable Plaza, Thane-Belapur Road, Airoli Navi Mumbai => Liberty Tower, Plot No. K 10, Behind Reliable Plaza, Thane-Belapur Road, Airoli Navi Mumbai\n",
      "Opposite Bon Bon, Four Bungalows, Andheri West, Mumbai => Opposite Bon Bon, Four Bungalows, Andheri West, Mumbai\n",
      "Lokhandawala Road, 4 Bungalows, Andheri (West), Mumbai => Lokhandawala Road, 4 Bungalows, Andheri (West), Mumbai\n",
      "New Link Road, Andheri West, Mumbai => New Link Road, Andheri West, Mumbai\n",
      "Rd S => Rd S\n",
      "Kolsewadi => Kolsewadi\n",
      "Asalpha => Asalpha\n",
      "Veer Savarkar Rd (gavthan road) => Veer Savarkar Rd (gavthan road)\n",
      "Pawana Nagar Taluka Maval => Pawana Nagar Taluka Maval\n",
      "Creak Crest, Model Town, Four Bunglow => Creak Crest, Model Town, Four Bunglow\n",
      "Jagnnath Mandir Road,Satyanagar, Sakinaka => Jagnnath Mandir Road,Satyanagar, Sakinaka\n",
      "Satyanagar, Sakinaka => Satyanagar, Sakinaka\n",
      "Gokhale Road North, Amarhind Mandal => Gokhale Road North, Amarhind Mandal\n",
      "Eastern Expressway => Eastern Expressway\n",
      "Caser Mill Circle => Caser Mill Circle\n",
      "Harinivas Circle => Harinivas Circle\n",
      "Western Express Highway => Western Express Highway\n",
      "Sion Panvel Highway => Sion Panvel Highway\n",
      "Eastern Express Highway => Eastern Express Highway\n",
      "old Mumbai pune Highway => old Mumbai pune Highway\n",
      "eastern Express Highway => eastern Express Highway\n",
      "Bandra Kurla Complex, Off Western Express Highway => Bandra Kurla Complex, Off Western Express Highway\n",
      "Mumbai Agra National Highway => Mumbai Agra National Highway\n",
      "Erangal Beach, Madh Island, Marve Road, Malad East, => Erangal Beach, Madh Island, Marve Road, Malad East,\n",
      "186, New Andheri Link Road, Bhagat Singh Colony,, Andheri East, => 186, New Andheri Link Road, Bhagat Singh Colony,, Andheri East,\n",
      "5th Floor, Chakala Pragati CHS, J B Nagar, Andheri East => 5th Floor, Chakala Pragati CHS, J B Nagar, Andheri East\n",
      "Kher Nagar, Bandra East => Kher Nagar, Bandra East\n",
      "Ashok Nagar, Marol Naka, Off Military Road, Andheri East => Ashok Nagar, Marol Naka, Off Military Road, Andheri East\n",
      "Tucker Heights, Plot No 197, Andheri East => Tucker Heights, Plot No 197, Andheri East\n",
      "Dattapada Road, Borivali East => Dattapada Road, Borivali East\n",
      "Near Sai Baba Mandir, Pathanwadi Bus Stand, Malad East => Near Sai Baba Mandir, Pathanwadi Bus Stand, Malad East\n",
      "Com. Harbanslal Marg, Sion East => Com. Harbanslal Marg, Sion East\n",
      "d-sector, f/line, cheeta camp, => d-sector, f/line, cheeta camp,\n",
      "Unique Garden => Unique Garden\n",
      "Rd5-Rd6 Connector => Rd5-Rd6 Connector\n",
      "JVLR => JVLR\n",
      "Walawalkar Marg, Andheri(West), => Walawalkar Marg, Andheri(West),\n",
      "Aramnagar Part 1 => Aramnagar Part 1\n",
      "Pokhran Road 1 => Pokhran Road 1\n",
      "Road No 1 => Road No 1\n",
      "Koldongri 1 => Koldongri 1\n",
      "Aramnagar 1 => Aramnagar 1\n",
      "Gawde Road 1 => Gawde Road 1\n",
      "Road Number 1 => Road Number 1\n",
      "Bandra (East) => Bandra (East)\n",
      "Koldongri => Koldongri\n",
      "Ring Road 5 => Ring Road 5\n",
      "Gulmohar Cross Road No 5 => Gulmohar Cross Road No 5\n",
      "Tejpal Scheme Road 5 => Tejpal Scheme Road 5\n",
      "Road Number 5 => Road Number 5\n",
      "Road No 5 => Road No 5\n",
      "hill road,Sion-chunabhatti => hill road,Sion-chunabhatti\n",
      "Road No. 9 => Road No. 9\n",
      "Gulmohar Cross Road No 9 => Gulmohar Cross Road No 9\n",
      "Ghodbunder Jhopadpatti => Ghodbunder Slums\n",
      "District Raigad => District Raigad\n",
      "Vengaon Road,Dahivali, Karjat, Dist Raigad ,Raigad => Vengaon Road,Dahivali, Karjat, Dist Raigad ,Raigad\n",
      "Jn. of 28th Road & St. Theresa's Road, T.P.S. III => Jn. of 28th Road & St. Theresa's Road, T.P.S. III\n",
      "23rd Road, T.P.S. III => 23rd Road, T.P.S. III\n",
      "28th Road,  T.P.S. III => 28th Road,  T.P.S. III\n",
      "28th Road, T.P.S. III => 28th Road, T.P.S. III\n",
      "28th Road, TPS III => 28th Road, TPS III\n",
      "Chandivali Farm Road , Chandivali => Chandivali Farm Road , Chandivali\n",
      "Nerul, Navi Mumbai,  Sector No 19 A => Nerul, Navi Mumbai,  Sector No 19 A\n",
      "Artist Village => Artist Village\n",
      "Ghodbunder Baug Road. Ghodbunder Village => Ghodbunder Baug Road. Ghodbunder Village\n",
      "Mohili Village => Mohili Village\n",
      "10E, Kopri Village => 10E, Kopri Village\n",
      "Ghodbunder Baug Road, Ghodbunder Village => Ghodbunder Baug Road, Ghodbunder Village\n",
      "Thakur Village => Thakur Village\n",
      "Thakur Cinema, Thakur Village => Thakur Cinema, Thakur Village\n",
      "New Marine Lines => New Marine Lines\n",
      "Marine Drive => Marine Drive\n",
      "Sector-6 => Sector-6\n",
      "Deshmukh Residency => Deshmukh Residency\n",
      "BELAPUR => BELAPUR\n",
      ", Off Hemu Kalani Marg, Chembur East, Mumbai - 400071, Behind Maruti Suzuki Showroom, Behind Ankit Chinese Restauran => , Off Hemu Kalani Marg, Chembur East, Mumbai - 400071, Behind Maruti Suzuki Showroom, Behind Ankit Chinese Restauran\n",
      "Bhandup => Bhandup\n",
      "Hanuman Raod => Hanuman Road\n",
      "Shastri Nagar, Dombivli West Dombivli, Maharashtra 421202 => Shastri Nagar, Dombivli West Dombivli, Maharashtra 421202\n",
      "road no.13 => road no.13\n",
      "A Irani Bridge => A Irani Bridge\n",
      "Elphinstone Bridge => Elphinstone Bridge\n",
      "Sahakar Nagar lane => Sahakar Nagar lane\n",
      "Devidas lane => Devidas lane\n",
      "jivedaya lane => jivedaya lane\n",
      "vallabaugh lane => vallabaugh lane\n",
      "Lokhandwala, Shastri Nagar, 2nd lane => Lokhandwala, Shastri Nagar, 2nd lane\n",
      "Vallabh Baug lane => Vallabh Baug lane\n",
      "T.H. Karatia Marg, Mogul lane => T.H. Karatia Marg, Mogul lane\n",
      "Bhagat lane => Bhagat lane\n",
      "Scheme 57 Road no.10 => Scheme 57 Road no.10\n",
      "Aarye Milk Colony, Royal Palms => Aarye Milk Colony, Royal Palms\n",
      "Central Avenue, Hiranandani Estate => Central Avenue, Hiranandani Estate\n",
      "Sport Road, Ballard Estate => Sport Road, Ballard Estate\n",
      "Rahul Estate => Rahul Estate\n",
      "Hiranandani Estate => Hiranandani Estate\n",
      "Kannamwar Nagar 1 , Vikhroli (east) => Kannamwar Nagar 1 , Vikhroli (east)\n",
      "Vidyavihar skywalk => Vidyavihar skywalk\n",
      "Hatutma Sitaram Ghadi Gaonkar Street => Hatutma Sitaram Ghadi Gaonkar Street\n",
      "Kalyan Street => Kalyan Street\n",
      "St Dominic Street => St Dominic Street\n",
      "Sarang Street => Sarang Street\n",
      "Tantanpura Street => Tantanpura Street\n",
      "High Street => High Street\n",
      "Yogiraj Street => Yogiraj Street\n",
      "Main Street => Main Street\n",
      "Technology Street => Technology Street\n",
      "Wallace Street => Wallace Street\n",
      "Cawasji Patel Street => Cawasji Patel Street\n",
      "St. Anthony Street => St. Anthony Street\n",
      "Saint John's Street => Saint John's Street\n",
      "Rope Walk Street => Rope Walk Street\n",
      "Perin Nariman Street => Perin Nariman Street\n",
      "Dalal Street => Dalal Street\n",
      "St John's Street => St John's Street\n",
      "Lamington Street => Lamington Street\n",
      "Dadiseth Agiary Street => Dadiseth Agiary Street\n",
      "Forest Street => Forest Street\n",
      "Forjet Street => Forjet Street\n",
      "Nowroji Ferdonji Street => Nowroji Ferdonji Street\n",
      "TTC Ind. Area, Mahape (Pawane) => TTC Ind. Area, Mahape (Pawane)\n",
      "Ground Floor, Thakur Shopping Mall & Multiplex => Ground Floor, Thakur Shopping Mall & Multiplex\n",
      "1st Floor, Thakur Shopping Mall & Multiplex => 1st Floor, Thakur Shopping Mall & Multiplex\n",
      "shiwaji chowk => shiwaji chowk\n",
      "Road no 33 => Road no 33\n"
     ]
    }
   ],
   "source": [
    "import xml.etree.cElementTree as ET\n",
    "from collections import defaultdict\n",
    "import re\n",
    "import pprint\n",
    "\n",
    "OSMFILE = \"mumbai_india.osm\"\n",
    "street_type_re = re.compile(r'\\b\\S+\\.?$', re.IGNORECASE)\n",
    "\n",
    "#expected = [\"Slums\", \"Avenue\", \"Boulevard\", \"Drive\", \"Court\", \"Place\", \"Square\", \"Lane\", \"Road\", \n",
    "            \n",
    "# UPDATE THIS VARIABLE\n",
    "mapping = { \"Jhopadpatti\": \"Slums\",\n",
    "            \"Marg,\": \"Marg\",\n",
    "            \"Mumbai,\":\"Mumbai\",\n",
    "            \"ROad\":\"Road\",\n",
    "            \"Rd\":\"Road\",\n",
    "            \"Raod\":\"Road\",\n",
    "            \"Rd.\":\"Road\",\n",
    "            \"road\":\"Road\"\n",
    "            }\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "# UPDATE THIS VARIABLE\n",
    "\n",
    "\n",
    "def audit_street_type(street_types, street_name):\n",
    "    m = street_type_re.search(street_name)\n",
    "    if m:\n",
    "        street_type = m.group()\n",
    "        #if street_type not in expected:\n",
    "        street_types[street_type].add(street_name)\n",
    "\n",
    "\n",
    "def is_street_name(elem):\n",
    "    return (elem.attrib['k'] == \"addr:street\")\n",
    "\n",
    "\n",
    "def audit(osmfile):\n",
    "    osm_file = open(osmfile, \"r\")\n",
    "    street_types = defaultdict(set)\n",
    "    for event, elem in ET.iterparse(osm_file, events=(\"start\",)):\n",
    "\n",
    "        if elem.tag == \"node\" or elem.tag == \"way\":\n",
    "            for tag in elem.iter(\"tag\"):\n",
    "                if is_street_name(tag):\n",
    "                    audit_street_type(street_types, tag.attrib['v'])\n",
    "    osm_file.close()\n",
    "    return street_types\n",
    "\n",
    "def update_name(name, mapping):\n",
    "    #print '\\nSTART'\n",
    "    #print name\n",
    "    m = street_type_re.search(name)\n",
    "    if m.group() in mapping.keys():        \n",
    "        name=re.sub(street_type_re,mapping[m.group()],name)\n",
    "\n",
    "    # YOUR CODE HERE\n",
    "\n",
    "    return name\n",
    "\n",
    "st_types = audit(OSMFILE)\n",
    "#pprint.pprint(dict(st_types))\n",
    "\n",
    "for st_type, ways in st_types.iteritems():\n",
    "    for name in ways:\n",
    "        better_name = update_name(name, mapping)\n",
    "        print name, \"=>\", better_name  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "# Postal Codes\n",
    "\n",
    "Postal codes of Mumbai all start with '40' and all are of 6 digits therefore for auditing postcodes it was important to ensure the postal codes are in the 6-digit format\n",
    "To locate and extract the 6-digit zip code in cases where there are irrelevant or too much information - i.e. random white spaces or an additional strings after the zip code.\n",
    "Example conversions:\n",
    "1. 400076,India\n",
    "\n",
    "To identify problems with postal codes, I used a function that would search through the XML data for every instance of a postal code, and add it to a set of unique postal codes if it satisfies above mentioned conditions.\n",
    "The audit function can be found below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'400001': set(['400001']),\n",
      " '400002': set(['400002']),\n",
      " '400003': set(['400003']),\n",
      " '400004': set(['400004']),\n",
      " '400005': set(['400005']),\n",
      " '400007': set(['400007']),\n",
      " '400008': set(['400008']),\n",
      " '400009': set(['400009']),\n",
      " '400010': set(['400010']),\n",
      " '400011': set(['400011']),\n",
      " '400012': set(['400012']),\n",
      " '400013': set(['400013']),\n",
      " '400014': set(['400014']),\n",
      " '400015': set(['400015']),\n",
      " '400016': set(['400016']),\n",
      " '400017': set(['400017']),\n",
      " '400018': set(['400018']),\n",
      " '400019': set(['400019']),\n",
      " '400020': set(['400020']),\n",
      " '400021': set(['400021']),\n",
      " '400022': set(['400022']),\n",
      " '400023': set(['400023']),\n",
      " '400024': set(['400024']),\n",
      " '400025': set(['400025']),\n",
      " '400027': set(['400027']),\n",
      " '400028': set(['400028']),\n",
      " '400029': set(['400029']),\n",
      " '400030': set(['400030']),\n",
      " '400031': set(['400031']),\n",
      " '400033': set(['400033']),\n",
      " '400034': set(['400034']),\n",
      " '400035': set(['400035']),\n",
      " '400036': set(['400036']),\n",
      " '400037': set(['400037']),\n",
      " '400038': set(['400038']),\n",
      " '400039': set(['400039']),\n",
      " '400042': set(['400042']),\n",
      " '400043': set(['400043']),\n",
      " '400047': set(['400047']),\n",
      " '400049': set(['400049']),\n",
      " '400050': set(['400050']),\n",
      " '400051': set(['400051']),\n",
      " '400052': set(['400052']),\n",
      " '400053': set(['400053']),\n",
      " '400054': set(['400054']),\n",
      " '400055': set(['400055']),\n",
      " '400056': set(['400056']),\n",
      " '400057': set(['400057']),\n",
      " '400058': set(['400058']),\n",
      " '400059': set(['400059']),\n",
      " '400060': set(['400060']),\n",
      " '400061': set(['400061']),\n",
      " '400062': set(['400062']),\n",
      " '400063': set(['400063']),\n",
      " '400064': set(['400064']),\n",
      " '400065': set(['400065']),\n",
      " '400066': set(['400066']),\n",
      " '400067': set(['400067']),\n",
      " '400068': set(['400068']),\n",
      " '400069': set(['400069']),\n",
      " '400070': set(['400070']),\n",
      " '400071': set(['400071']),\n",
      " '400072': set(['400072']),\n",
      " '400074': set(['400074']),\n",
      " '400075': set(['400075']),\n",
      " '400076': set(['400076']),\n",
      " '400077': set(['400077']),\n",
      " '400078': set(['400078']),\n",
      " '400079': set(['400079']),\n",
      " '400080': set(['400080']),\n",
      " '400081': set(['400081']),\n",
      " '400082': set(['400082']),\n",
      " '400083': set(['400083']),\n",
      " '400084': set(['400084']),\n",
      " '400086': set(['400086']),\n",
      " '400087': set(['400087']),\n",
      " '400088': set(['400088']),\n",
      " '400089': set(['400089']),\n",
      " '400091': set(['400091']),\n",
      " '400092': set(['400092']),\n",
      " '400093': set(['400093']),\n",
      " '400094': set(['400094']),\n",
      " '400095': set(['400095']),\n",
      " '400096': set(['400096']),\n",
      " '400097': set(['400097']),\n",
      " '400098': set(['400098']),\n",
      " '400099': set(['400099']),\n",
      " '400101': set(['400101']),\n",
      " '400102': set(['400102']),\n",
      " '400103': set(['400103']),\n",
      " '400104': set(['400104']),\n",
      " '400601': set(['400601']),\n",
      " '400602': set(['400602']),\n",
      " '400605': set(['400605']),\n",
      " '400606': set(['400606']),\n",
      " '400607': set(['400607']),\n",
      " '400610': set(['400610']),\n",
      " '400612': set(['400612']),\n",
      " '400614': set(['400614']),\n",
      " '400615': set(['400615']),\n",
      " '400701': set(['400701']),\n",
      " '400702': set(['400702']),\n",
      " '400703': set(['400703']),\n",
      " '400705': set(['400705']),\n",
      " '400706': set(['400706']),\n",
      " '400708': set(['400708']),\n",
      " '400709': set(['400709']),\n",
      " '400710': set(['400710']),\n",
      " '400950': set(['400950']),\n",
      " '401101': set(['401101']),\n",
      " '401104': set(['401104']),\n",
      " '401105': set(['401105']),\n",
      " '401107': set(['401107']),\n",
      " '401201': set(['401201']),\n",
      " '401202': set(['401202']),\n",
      " '401203': set(['401203']),\n",
      " '401209': set(['401209']),\n",
      " '401303': set(['401303']),\n",
      " '401305': set(['401305']),\n",
      " '402201': set(['402201']),\n",
      " '402205': set(['402205'])}\n"
     ]
    }
   ],
   "source": [
    "import xml.etree.cElementTree as ET\n",
    "from collections import defaultdict\n",
    "import re\n",
    "import pprint\n",
    "\n",
    "OSMFILE = \"mumbai_india.osm\"\n",
    "tree = ET.parse(OSMFILE)\n",
    "root = tree.getroot()\n",
    "\n",
    "postal_type_re = re.compile('^(4)(0)\\d{4}$')\n",
    "\n",
    "\n",
    "\n",
    "def audit_postal_code(postal_types,postal_value):\n",
    "    m = postal_type_re.search(postal_value)\n",
    "    if m:\n",
    "        postal_type=m.group()\n",
    "        postal_types[postal_type].add(postal_value)\n",
    "\n",
    "def is_postal_code(elem):\n",
    "    return (elem.attrib['k'] == \"addr:postcode\")\n",
    "\n",
    "\n",
    "def audit(osmfile):\n",
    "    osm_file = open(osmfile, \"r\")\n",
    "    postal_types=defaultdict(set)\n",
    "    for event, elem in ET.iterparse(osm_file, events=(\"start\",)):\n",
    "\n",
    "        if elem.tag in[\"node\", \"way\", \"relation\"] :\n",
    "            for tag in elem.iter(\"tag\"):\n",
    "                if is_postal_code(tag):\n",
    "                    audit_postal_code(postal_types,tag.attrib['v'])\n",
    "    osm_file.close()\n",
    "    return postal_types\n",
    "\n",
    "\n",
    "#audit(SAMPLE_FILE)\n",
    "#audit(OSMFILE)\n",
    "pprint.pprint(dict(audit(OSMFILE)))\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Code below shows the bad postal codes which are either not of mumbai or are in incorrect format:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['421501', '421503', '410206', '410210', '410210', '410210', '410210', '412108', '410701', '421202', '421202', '4000607', '40058', '421501', '63103', '421501', '421501', '410201', '410201', '410201', '4000082', '400059.', '410210', '110092', '123', '410209', '410209', '410209', '421302', '410206', '4000072', '410206', '421301', '421301', '4000072', '421306', '410210', '410210', '410210', '410210', '410210', '410210', '410210', '410210', '410210', '410210', '410210', '410210', '410210', '410210', '410210', '410210', '410210', '410210', '410208', '421501', '410206', '40076', '560023', '590006', '492006', '410210', '410206', '560092', '410206', '410206', '410206', '410206', '410206', '410206', '410206', '410206', '410206', '410221', '410102', '410206', '421601', '410206', '421005', '410206', '421501', '410221', '410206', '410206', '48147', '410218', '421201', '421302', '410206', '412108', '412108', '421501', '421501', '410206', '421311', '400076India', '400076India', '410210', '40081', '410210', '410201', '410201', '40001', '40049', '421501', '410406', '412108', '400076India', '421302', '500053', '421201', '410206', '43', '40051', '400076India', '421202', '410206', '410210', '421302', '560103', '500054', '410210', '410210', '410210', '410210', '410218', '421301', '410210', '410210', '410210', '410210', '410210', '410210', '410210', '410210', '410210', '410206', '410206', '410210', '410210', '421301', '410210', '421002', '421203', '410206', '410209', '421306', '421202', '421202', '421503', '410401', '410210', '410210', '410210', '410210', '410210', '410210', '410210', '410208', '410208', '410209', '410210', '410206', '412108', '421004', '410206', '421201']\n",
      "167\n"
     ]
    }
   ],
   "source": [
    "import xml.etree.ElementTree as ET\n",
    "import re\n",
    "\n",
    "tree = ET.parse(OSMFILE)\n",
    "root = tree.getroot()\n",
    "\n",
    "counttotal = 0\n",
    "count = 0\n",
    "wp = []\n",
    "regex = re.compile('^(4)(0)\\d{4}$')\n",
    "for i in tree.getiterator('tag'):\n",
    "    k1 = i.get(\"k\")\n",
    "    if k1 == \"addr:postcode\":\n",
    "        v1 = i.get(\"v\")\n",
    "        m1 = regex.match(v1)\n",
    "        if not m1:\n",
    "            counttotal = counttotal +1\n",
    "            if len(v1) <> 6:\n",
    "                v1 = v1.replace(\" \",\"\")\n",
    "                v1 = v1.replace(\",\",\"\")\n",
    "                v1 = v1.replace(\"-\",\"\")\n",
    "                v1 = v1.replace('\"',\"\")\n",
    "                v1 = v1.replace('55',\"5\")\n",
    "                m2 = regex.match(v1)\n",
    "                if not m2:\n",
    "                    wp.append(v1)\n",
    "                    count = count +1\n",
    "            elif len(v1) == 6:\n",
    "                wp.append(v1)\n",
    "                count = count +1\n",
    "\n",
    "print wp\n",
    "print count\n",
    "#print counttotal\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This function is use to update the postal codes by which all postal codes from bad postals mentioned above will be replaced by '000000'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "000000\n",
      "000000\n",
      "000000\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400615\n",
      "400614\n",
      "000000\n",
      "400005\n",
      "000000\n",
      "400020\n",
      "400086\n",
      "400601\n",
      "400022\n",
      "400606\n",
      "400039\n",
      "400005\n",
      "400001\n",
      "400039\n",
      "400020\n",
      "000000\n",
      "400039\n",
      "000000\n",
      "400039\n",
      "400607\n",
      "400020\n",
      "400069\n",
      "400703\n",
      "400610\n",
      "400016\n",
      "400071\n",
      "400067\n",
      "400005\n",
      "400049\n",
      "400072\n",
      "400710\n",
      "400093\n",
      "400054\n",
      "400052\n",
      "400088\n",
      "000000\n",
      "400001\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400607\n",
      "400607\n",
      "400607\n",
      "400607\n",
      "400607\n",
      "400607\n",
      "400607\n",
      "400607\n",
      "400607\n",
      "400607\n",
      "400607\n",
      "400607\n",
      "400607\n",
      "400607\n",
      "400607\n",
      "400607\n",
      "400607\n",
      "400607\n",
      "400607\n",
      "400607\n",
      "400607\n",
      "400607\n",
      "400607\n",
      "400607\n",
      "400607\n",
      "400607\n",
      "400057\n",
      "400007\n",
      "400050\n",
      "400076\n",
      "400706\n",
      "400074\n",
      "400614\n",
      "400614\n",
      "400071\n",
      "400076\n",
      "400016\n",
      "400066\n",
      "400007\n",
      "400005\n",
      "400005\n",
      "400103\n",
      "400103\n",
      "400005\n",
      "000000\n",
      "400022\n",
      "400049\n",
      "400049\n",
      "400049\n",
      "400057\n",
      "400057\n",
      "400088\n",
      "400606\n",
      "400606\n",
      "400606\n",
      "400066\n",
      "400053\n",
      "400091\n",
      "400091\n",
      "400601\n",
      "400601\n",
      "400601\n",
      "400001\n",
      "400038\n",
      "400019\n",
      "400034\n",
      "400005\n",
      "400709\n",
      "400705\n",
      "400053\n",
      "400033\n",
      "400080\n",
      "400080\n",
      "400061\n",
      "400061\n",
      "400053\n",
      "400053\n",
      "400053\n",
      "400053\n",
      "400061\n",
      "400061\n",
      "400061\n",
      "400053\n",
      "400053\n",
      "000000\n",
      "400053\n",
      "400053\n",
      "400053\n",
      "400053\n",
      "400053\n",
      "400058\n",
      "400053\n",
      "400053\n",
      "400053\n",
      "400053\n",
      "400053\n",
      "400053\n",
      "400053\n",
      "400053\n",
      "400053\n",
      "400047\n",
      "400047\n",
      "400058\n",
      "400058\n",
      "400058\n",
      "400607\n",
      "400001\n",
      "400001\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "401203\n",
      "000000\n",
      "000000\n",
      "400043\n",
      "400077\n",
      "400071\n",
      "400071\n",
      "400024\n",
      "400016\n",
      "400068\n",
      "000000\n",
      "400067\n",
      "400053\n",
      "400050\n",
      "400080\n",
      "400080\n",
      "400071\n",
      "400018\n",
      "400087\n",
      "400072\n",
      "400074\n",
      "400074\n",
      "400074\n",
      "400053\n",
      "400053\n",
      "400053\n",
      "400053\n",
      "400053\n",
      "400053\n",
      "400053\n",
      "400053\n",
      "400053\n",
      "400061\n",
      "400061\n",
      "400049\n",
      "400064\n",
      "400062\n",
      "400601\n",
      "400061\n",
      "400061\n",
      "400061\n",
      "400601\n",
      "400601\n",
      "000000\n",
      "400058\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "400080\n",
      "400078\n",
      "401202\n",
      "400072\n",
      "400053\n",
      "400061\n",
      "400061\n",
      "400028\n",
      "400060\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "400703\n",
      "400706\n",
      "400602\n",
      "400075\n",
      "000000\n",
      "400005\n",
      "400703\n",
      "400601\n",
      "400016\n",
      "400057\n",
      "400071\n",
      "400071\n",
      "400093\n",
      "400097\n",
      "400013\n",
      "400101\n",
      "400101\n",
      "400094\n",
      "400049\n",
      "400607\n",
      "400607\n",
      "400022\n",
      "400709\n",
      "400003\n",
      "400709\n",
      "400002\n",
      "400004\n",
      "400004\n",
      "400092\n",
      "400092\n",
      "400703\n",
      "400074\n",
      "400089\n",
      "400043\n",
      "400088\n",
      "000000\n",
      "400089\n",
      "400089\n",
      "400089\n",
      "400077\n",
      "400024\n",
      "400089\n",
      "400089\n",
      "400024\n",
      "400074\n",
      "400074\n",
      "400074\n",
      "400077\n",
      "400043\n",
      "400070\n",
      "400070\n",
      "400070\n",
      "400070\n",
      "400070\n",
      "400070\n",
      "400614\n",
      "400614\n",
      "400614\n",
      "400088\n",
      "000000\n",
      "400067\n",
      "000000\n",
      "400093\n",
      "400086\n",
      "400703\n",
      "400088\n",
      "400088\n",
      "400088\n",
      "400088\n",
      "400074\n",
      "400074\n",
      "400074\n",
      "400074\n",
      "400071\n",
      "400071\n",
      "400071\n",
      "400071\n",
      "400018\n",
      "400064\n",
      "400064\n",
      "400067\n",
      "400058\n",
      "400067\n",
      "400058\n",
      "000000\n",
      "400053\n",
      "400053\n",
      "400053\n",
      "400064\n",
      "400061\n",
      "000000\n",
      "400043\n",
      "400089\n",
      "400089\n",
      "400089\n",
      "400043\n",
      "400043\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "400077\n",
      "400089\n",
      "400043\n",
      "000000\n",
      "400089\n",
      "400043\n",
      "400088\n",
      "400043\n",
      "400077\n",
      "400071\n",
      "400043\n",
      "400074\n",
      "400043\n",
      "400089\n",
      "400088\n",
      "400043\n",
      "400089\n",
      "400089\n",
      "400043\n",
      "400043\n",
      "400089\n",
      "400089\n",
      "000000\n",
      "400089\n",
      "400089\n",
      "400043\n",
      "400043\n",
      "000000\n",
      "000000\n",
      "400043\n",
      "400071\n",
      "400043\n",
      "400043\n",
      "000000\n",
      "400088\n",
      "400043\n",
      "400043\n",
      "400081\n",
      "400043\n",
      "400043\n",
      "400043\n",
      "400043\n",
      "400043\n",
      "400043\n",
      "400043\n",
      "400071\n",
      "400071\n",
      "400089\n",
      "400089\n",
      "000000\n",
      "400089\n",
      "400043\n",
      "400043\n",
      "400043\n",
      "400043\n",
      "400043\n",
      "400043\n",
      "400043\n",
      "400043\n",
      "400043\n",
      "400088\n",
      "400088\n",
      "400089\n",
      "400089\n",
      "400043\n",
      "400089\n",
      "400043\n",
      "400088\n",
      "400089\n",
      "400089\n",
      "400088\n",
      "400043\n",
      "400043\n",
      "400043\n",
      "000000\n",
      "000000\n",
      "400088\n",
      "400088\n",
      "000000\n",
      "400043\n",
      "400043\n",
      "400043\n",
      "400043\n",
      "400089\n",
      "400043\n",
      "400043\n",
      "400088\n",
      "400043\n",
      "400071\n",
      "400089\n",
      "400089\n",
      "400043\n",
      "400043\n",
      "400009\n",
      "400043\n",
      "400043\n",
      "400043\n",
      "400053\n",
      "400081\n",
      "400084\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "400080\n",
      "400080\n",
      "400080\n",
      "400080\n",
      "400081\n",
      "400043\n",
      "400043\n",
      "400043\n",
      "400043\n",
      "400043\n",
      "400043\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "400080\n",
      "400081\n",
      "400080\n",
      "400080\n",
      "400080\n",
      "400043\n",
      "400043\n",
      "400043\n",
      "400043\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "400080\n",
      "400602\n",
      "400080\n",
      "400080\n",
      "400080\n",
      "400080\n",
      "400080\n",
      "400080\n",
      "400080\n",
      "400080\n",
      "400080\n",
      "400080\n",
      "400080\n",
      "400080\n",
      "400080\n",
      "400049\n",
      "400053\n",
      "400080\n",
      "400102\n",
      "400070\n",
      "400070\n",
      "400070\n",
      "400077\n",
      "400005\n",
      "400005\n",
      "400063\n",
      "400013\n",
      "400094\n",
      "400078\n",
      "400614\n",
      "000000\n",
      "400706\n",
      "400706\n",
      "400097\n",
      "400020\n",
      "400020\n",
      "400001\n",
      "400023\n",
      "400065\n",
      "400064\n",
      "400092\n",
      "400101\n",
      "400057\n",
      "400001\n",
      "000000\n",
      "000000\n",
      "400043\n",
      "400056\n",
      "400080\n",
      "400078\n",
      "000000\n",
      "400078\n",
      "400072\n",
      "400072\n",
      "400072\n",
      "400072\n",
      "400601\n",
      "400072\n",
      "400072\n",
      "400072\n",
      "400072\n",
      "000000\n",
      "400076\n",
      "400076\n",
      "400072\n",
      "400606\n",
      "400051\n",
      "400606\n",
      "400051\n",
      "400612\n",
      "400028\n",
      "400028\n",
      "400088\n",
      "400088\n",
      "400088\n",
      "400088\n",
      "400070\n",
      "400070\n",
      "400043\n",
      "400043\n",
      "400043\n",
      "400043\n",
      "400074\n",
      "400705\n",
      "400705\n",
      "400089\n",
      "400089\n",
      "400043\n",
      "400043\n",
      "400043\n",
      "400079\n",
      "400072\n",
      "400079\n",
      "400059\n",
      "000000\n",
      "400086\n",
      "400060\n",
      "400053\n",
      "400081\n",
      "400083\n",
      "400071\n",
      "400005\n",
      "000000\n",
      "400072\n",
      "400022\n",
      "000000\n",
      "400060\n",
      "402201\n",
      "400066\n",
      "400013\n",
      "400076\n",
      "400066\n",
      "400099\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "400078\n",
      "400063\n",
      "400081\n",
      "400081\n",
      "400605\n",
      "000000\n",
      "400007\n",
      "400007\n",
      "401202\n",
      "400049\n",
      "400050\n",
      "400050\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400602\n",
      "400053\n",
      "400013\n",
      "400081\n",
      "400005\n",
      "400005\n",
      "400001\n",
      "400001\n",
      "400001\n",
      "400001\n",
      "400001\n",
      "400601\n",
      "400061\n",
      "400061\n",
      "400706\n",
      "400007\n",
      "400007\n",
      "400076\n",
      "400705\n",
      "400004\n",
      "400004\n",
      "400004\n",
      "400004\n",
      "400004\n",
      "400104\n",
      "400021\n",
      "400005\n",
      "400052\n",
      "400052\n",
      "400050\n",
      "000000\n",
      "400049\n",
      "400051\n",
      "000000\n",
      "400058\n",
      "400058\n",
      "400081\n",
      "400038\n",
      "400089\n",
      "400002\n",
      "400027\n",
      "400021\n",
      "400027\n",
      "400001\n",
      "400005\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400028\n",
      "400013\n",
      "400080\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400072\n",
      "400072\n",
      "400072\n",
      "400099\n",
      "400055\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400067\n",
      "400016\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400002\n",
      "400002\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400101\n",
      "400052\n",
      "400050\n",
      "400004\n",
      "400059\n",
      "400007\n",
      "400077\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400066\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "000000\n",
      "400089\n",
      "400076\n",
      "000000\n",
      "400076\n",
      "401201\n",
      "400051\n",
      "400053\n",
      "400001\n",
      "400001\n",
      "400602\n",
      "000000\n",
      "400014\n",
      "400001\n",
      "000000\n",
      "000000\n",
      "400079\n",
      "000000\n",
      "400062\n",
      "400033\n",
      "000000\n",
      "400601\n",
      "400042\n",
      "401202\n",
      "400701\n",
      "000000\n",
      "401107\n",
      "400003\n",
      "400030\n",
      "400071\n",
      "400051\n",
      "400050\n",
      "400029\n",
      "400049\n",
      "400054\n",
      "400093\n",
      "400093\n",
      "400093\n",
      "000000\n",
      "400601\n",
      "401209\n",
      "400095\n",
      "000000\n",
      "401305\n",
      "400086\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "400057\n",
      "400601\n",
      "400043\n",
      "400602\n",
      "400092\n",
      "400092\n",
      "400064\n",
      "400064\n",
      "400102\n",
      "401107\n",
      "401107\n",
      "000000\n",
      "400020\n",
      "400016\n",
      "000000\n",
      "400012\n",
      "000000\n",
      "400016\n",
      "400016\n",
      "400089\n",
      "400089\n",
      "400089\n",
      "400089\n",
      "400012\n",
      "400012\n",
      "000000\n",
      "401107\n",
      "400082\n",
      "400080\n",
      "400068\n",
      "400076\n",
      "400076\n",
      "400076\n",
      "401101\n",
      "400001\n",
      "400101\n",
      "400080\n",
      "400082\n",
      "401105\n",
      "400005\n",
      "400021\n",
      "400005\n",
      "400095\n",
      "400017\n",
      "400075\n",
      "400022\n",
      "000000\n",
      "000000\n",
      "400702\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400007\n",
      "401101\n",
      "400614\n",
      "400061\n",
      "400061\n",
      "400097\n",
      "400086\n",
      "400706\n",
      "400047\n",
      "400706\n",
      "000000\n",
      "400614\n",
      "000000\n",
      "400605\n",
      "000000\n",
      "400706\n",
      "400007\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "400080\n",
      "400080\n",
      "400080\n",
      "400054\n",
      "400705\n",
      "400706\n",
      "400703\n",
      "400703\n",
      "400081\n",
      "400709\n",
      "400701\n",
      "400614\n",
      "400614\n",
      "400614\n",
      "400614\n",
      "401107\n",
      "400059\n",
      "400019\n",
      "400080\n",
      "400072\n",
      "400050\n",
      "400038\n",
      "400050\n",
      "400009\n",
      "000000\n",
      "400017\n",
      "400051\n",
      "000000\n",
      "400052\n",
      "400059\n",
      "000000\n",
      "400101\n",
      "400059\n",
      "400023\n",
      "400080\n",
      "400080\n",
      "400080\n",
      "400080\n",
      "400081\n",
      "400080\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "400016\n",
      "000000\n",
      "400080\n",
      "400080\n",
      "400081\n",
      "400080\n",
      "400081\n",
      "400080\n",
      "400606\n",
      "000000\n",
      "401209\n",
      "400053\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "000000\n",
      "400016\n",
      "400093\n",
      "400067\n",
      "400064\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "401303\n",
      "000000\n",
      "400007\n",
      "400013\n",
      "401107\n",
      "400036\n",
      "400007\n",
      "400080\n",
      "400080\n",
      "400080\n",
      "400007\n",
      "400081\n",
      "400081\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "000000\n",
      "400007\n",
      "000000\n",
      "400070\n",
      "400070\n",
      "400070\n",
      "400049\n",
      "400070\n",
      "400070\n",
      "400706\n",
      "400706\n",
      "400070\n",
      "400706\n",
      "400034\n",
      "400049\n",
      "400005\n",
      "400020\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400099\n",
      "400018\n",
      "400076\n",
      "000000\n",
      "000000\n",
      "400076\n",
      "400076\n",
      "400001\n",
      "400088\n",
      "000000\n",
      "400063\n",
      "400607\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "400014\n",
      "400014\n",
      "400031\n",
      "400016\n",
      "400102\n",
      "400070\n",
      "400057\n",
      "400074\n",
      "400064\n",
      "400072\n",
      "400016\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "000000\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400057\n",
      "400028\n",
      "400086\n",
      "400052\n",
      "400070\n",
      "400070\n",
      "000000\n",
      "400031\n",
      "400064\n",
      "400098\n",
      "400050\n",
      "400018\n",
      "400055\n",
      "400606\n",
      "000000\n",
      "000000\n",
      "400601\n",
      "400016\n",
      "400053\n",
      "400053\n",
      "400053\n",
      "400053\n",
      "400053\n",
      "400053\n",
      "400053\n",
      "400076\n",
      "400061\n",
      "400061\n",
      "400061\n",
      "400062\n",
      "000000\n",
      "400053\n",
      "400061\n",
      "400013\n",
      "400102\n",
      "400053\n",
      "400053\n",
      "400053\n",
      "400053\n",
      "400053\n",
      "400049\n",
      "400053\n",
      "400049\n",
      "400053\n",
      "400053\n",
      "400054\n",
      "400050\n",
      "400050\n",
      "400061\n",
      "400053\n",
      "000000\n",
      "400061\n",
      "400030\n",
      "400030\n",
      "400061\n",
      "400058\n",
      "400053\n",
      "400047\n",
      "400053\n",
      "400053\n",
      "400053\n",
      "400053\n",
      "400053\n",
      "400001\n",
      "400001\n",
      "000000\n",
      "400001\n",
      "400001\n",
      "400053\n",
      "400053\n",
      "400053\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400001\n",
      "400021\n",
      "400054\n",
      "400706\n",
      "400706\n",
      "400072\n",
      "400706\n",
      "400706\n",
      "400080\n",
      "400071\n",
      "400089\n",
      "400071\n",
      "400071\n",
      "400071\n",
      "400071\n",
      "400071\n",
      "400071\n",
      "400071\n",
      "400089\n",
      "400076\n",
      "400053\n",
      "400053\n",
      "400035\n",
      "400053\n",
      "402205\n",
      "400053\n",
      "400102\n",
      "400050\n",
      "400064\n",
      "400053\n",
      "400061\n",
      "400061\n",
      "400061\n",
      "400061\n",
      "400021\n",
      "400021\n",
      "400021\n",
      "400021\n",
      "400021\n",
      "400021\n",
      "400062\n",
      "400089\n",
      "400091\n",
      "000000\n",
      "400049\n",
      "400058\n",
      "400050\n",
      "400057\n",
      "000000\n",
      "400096\n",
      "400053\n",
      "400053\n",
      "400037\n",
      "400078\n",
      "400053\n",
      "400053\n",
      "400053\n",
      "400053\n",
      "400053\n",
      "400005\n",
      "400088\n",
      "400088\n",
      "400088\n",
      "400088\n",
      "400088\n",
      "400053\n",
      "000000\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400056\n",
      "400049\n",
      "400104\n",
      "400080\n",
      "400076\n",
      "400050\n",
      "400606\n",
      "400070\n",
      "400005\n",
      "400005\n",
      "400058\n",
      "400091\n",
      "000000\n",
      "400076\n",
      "000000\n",
      "400076\n",
      "000000\n",
      "400055\n",
      "400093\n",
      "400097\n",
      "400097\n",
      "400097\n",
      "400013\n",
      "400013\n",
      "400013\n",
      "400094\n",
      "400094\n",
      "400094\n",
      "400094\n",
      "400094\n",
      "400094\n",
      "400094\n",
      "400092\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400607\n",
      "401104\n",
      "400011\n",
      "400701\n",
      "400028\n",
      "400024\n",
      "400002\n",
      "400002\n",
      "400002\n",
      "400002\n",
      "400002\n",
      "400002\n",
      "400002\n",
      "400002\n",
      "400002\n",
      "400002\n",
      "400002\n",
      "400002\n",
      "400004\n",
      "400004\n",
      "400004\n",
      "400004\n",
      "400004\n",
      "400004\n",
      "400004\n",
      "400004\n",
      "400004\n",
      "400004\n",
      "400004\n",
      "400002\n",
      "400002\n",
      "400002\n",
      "400002\n",
      "400002\n",
      "400002\n",
      "400002\n",
      "400002\n",
      "400002\n",
      "400002\n",
      "400002\n",
      "400004\n",
      "400004\n",
      "400089\n",
      "400043\n",
      "400043\n",
      "400014\n",
      "400024\n",
      "400024\n",
      "400092\n",
      "400051\n",
      "400607\n",
      "400607\n",
      "400607\n",
      "400064\n",
      "400064\n",
      "400064\n",
      "400064\n",
      "400053\n",
      "000000\n",
      "400053\n",
      "400601\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400008\n",
      "000000\n",
      "400014\n",
      "400014\n",
      "400014\n",
      "400025\n",
      "400037\n",
      "400088\n",
      "400088\n",
      "400088\n",
      "400088\n",
      "400074\n",
      "400093\n",
      "400071\n",
      "400018\n",
      "400072\n",
      "400071\n",
      "400071\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400067\n",
      "400001\n",
      "000000\n",
      "400077\n",
      "400071\n",
      "400043\n",
      "400043\n",
      "400089\n",
      "400088\n",
      "400088\n",
      "400043\n",
      "000000\n",
      "000000\n",
      "400051\n",
      "400051\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400043\n",
      "400071\n",
      "400089\n",
      "400088\n",
      "400088\n",
      "400089\n",
      "400043\n",
      "400088\n",
      "400043\n",
      "400043\n",
      "400088\n",
      "400089\n",
      "400088\n",
      "400071\n",
      "400071\n",
      "400071\n",
      "000000\n",
      "400080\n",
      "400064\n",
      "400064\n",
      "400080\n",
      "400080\n",
      "400080\n",
      "400080\n",
      "400080\n",
      "400607\n",
      "400606\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "400093\n",
      "400080\n",
      "400081\n",
      "400081\n",
      "400080\n",
      "400080\n",
      "400080\n",
      "400080\n",
      "400081\n",
      "400070\n",
      "400070\n",
      "400070\n",
      "400074\n",
      "400074\n",
      "400074\n",
      "400074\n",
      "400074\n",
      "400074\n",
      "400074\n",
      "400074\n",
      "400074\n",
      "400074\n",
      "400074\n",
      "400074\n",
      "400074\n",
      "400074\n",
      "400074\n",
      "400074\n",
      "400074\n",
      "400074\n",
      "400074\n",
      "400074\n",
      "400074\n",
      "400074\n",
      "400074\n",
      "400074\n",
      "400074\n",
      "400708\n",
      "000000\n",
      "000000\n",
      "400104\n",
      "400093\n",
      "000000\n",
      "400074\n",
      "400072\n",
      "400072\n",
      "000000\n",
      "000000\n",
      "400042\n",
      "400072\n",
      "400089\n",
      "400071\n",
      "400088\n",
      "400093\n",
      "400089\n",
      "000000\n",
      "400088\n",
      "000000\n",
      "400072\n",
      "400015\n",
      "400022\n",
      "400022\n",
      "401101\n",
      "400067\n",
      "400067\n",
      "400067\n",
      "400067\n",
      "400067\n",
      "400067\n",
      "400067\n",
      "400067\n",
      "400067\n",
      "400067\n",
      "400064\n",
      "400064\n",
      "400068\n",
      "400063\n",
      "400067\n",
      "000000\n",
      "400605\n",
      "401107\n",
      "400607\n",
      "400093\n",
      "400078\n",
      "400080\n",
      "400081\n",
      "400080\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "400072\n",
      "400076\n",
      "400103\n",
      "400064\n",
      "400078\n",
      "400078\n",
      "400080\n",
      "400080\n",
      "400068\n",
      "400092\n",
      "400070\n",
      "400070\n",
      "400062\n",
      "400104\n",
      "400075\n",
      "400075\n",
      "400081\n",
      "400070\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "400104\n",
      "400058\n",
      "400076\n",
      "400070\n",
      "400070\n",
      "400070\n",
      "400070\n",
      "400070\n",
      "400042\n",
      "400080\n",
      "400067\n",
      "400069\n",
      "400099\n",
      "400028\n",
      "400070\n",
      "400051\n",
      "400051\n",
      "400051\n",
      "400051\n",
      "400051\n",
      "400076\n",
      "400101\n",
      "400066\n",
      "400072\n",
      "400080\n",
      "400080\n",
      "400080\n",
      "400058\n",
      "400098\n",
      "400051\n",
      "400051\n",
      "400099\n",
      "400078\n",
      "400081\n",
      "400103\n",
      "400081\n",
      "400094\n",
      "400034\n",
      "400037\n",
      "400086\n",
      "400097\n",
      "400052\n",
      "400005\n",
      "400080\n",
      "400013\n",
      "400601\n",
      "400601\n",
      "400601\n",
      "400601\n",
      "400060\n",
      "400001\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400071\n",
      "000000\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400063\n",
      "400710\n",
      "400703\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400081\n",
      "000000\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "400705\n",
      "400089\n",
      "400089\n",
      "400088\n",
      "401105\n",
      "400089\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "400005\n",
      "000000\n",
      "000000\n",
      "400069\n",
      "400008\n",
      "400081\n",
      "000000\n",
      "400706\n",
      "400056\n",
      "400089\n",
      "000000\n",
      "400012\n",
      "400012\n",
      "400018\n",
      "400018\n",
      "400028\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "400064\n",
      "400064\n",
      "400064\n",
      "400064\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "400064\n",
      "400093\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "400705\n",
      "400056\n",
      "000000\n",
      "400079\n",
      "400093\n",
      "000000\n",
      "000000\n",
      "400062\n",
      "000000\n",
      "400093\n",
      "400093\n",
      "400602\n",
      "400042\n",
      "000000\n",
      "000000\n",
      "401202\n",
      "401202\n",
      "401201\n",
      "401201\n",
      "000000\n",
      "401202\n",
      "400057\n",
      "400089\n",
      "400089\n",
      "400089\n",
      "400075\n",
      "401303\n",
      "400081\n",
      "400081\n",
      "401303\n",
      "401303\n",
      "400094\n",
      "400094\n",
      "400028\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "000000\n",
      "000000\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "400089\n",
      "400025\n",
      "000000\n",
      "401209\n",
      "000000\n",
      "400070\n",
      "400601\n",
      "400601\n",
      "400005\n",
      "400012\n",
      "400012\n",
      "400012\n",
      "400053\n",
      "400053\n",
      "400021\n",
      "400007\n",
      "400012\n",
      "400012\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400016\n",
      "400031\n",
      "400012\n",
      "400070\n",
      "400070\n",
      "400086\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400012\n",
      "400093\n",
      "400101\n",
      "400077\n",
      "400086\n",
      "400059\n",
      "400022\n",
      "400101\n",
      "400022\n",
      "400052\n",
      "400099\n",
      "400093\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400076\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400950\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400016\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400051\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400095\n",
      "400093\n",
      "400093\n",
      "400051\n",
      "000000\n",
      "400021\n",
      "400021\n",
      "400021\n",
      "400002\n",
      "000000\n",
      "000000\n",
      "400077\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400067\n",
      "000000\n",
      "400015\n",
      "400706\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400005\n",
      "400038\n",
      "400093\n",
      "400062\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "400086\n",
      "400708\n",
      "400001\n",
      "400050\n",
      "000000\n",
      "000000\n",
      "400053\n",
      "400061\n",
      "400061\n",
      "400058\n",
      "400028\n",
      "400082\n",
      "400043\n",
      "400001\n",
      "400051\n",
      "400005\n",
      "400020\n",
      "400005\n",
      "400018\n",
      "400013\n",
      "400001\n",
      "400001\n",
      "400088\n",
      "400005\n",
      "400708\n",
      "400709\n",
      "400705\n",
      "400701\n",
      "000000\n",
      "400089\n",
      "400071\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "400005\n",
      "400081\n",
      "400080\n",
      "000000\n",
      "000000\n",
      "400086\n",
      "400070\n",
      "400001\n",
      "400071\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400051\n",
      "400051\n",
      "400051\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400080\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400080\n",
      "400081\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "000000\n",
      "000000\n",
      "400070\n",
      "400010\n",
      "400079\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400051\n",
      "401303\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400612\n",
      "401201\n",
      "401201\n",
      "400075\n",
      "400075\n",
      "400075\n",
      "400075\n",
      "400050\n",
      "400705\n",
      "400099\n",
      "401107\n",
      "401107\n",
      "401107\n",
      "401107\n",
      "400612\n",
      "400050\n",
      "400050\n",
      "401101\n",
      "400086\n",
      "400706\n",
      "400706\n",
      "400706\n",
      "400706\n",
      "400706\n",
      "400706\n",
      "400064\n",
      "400064\n",
      "000000\n",
      "000000\n",
      "000000\n",
      "400081\n",
      "400094\n",
      "400094\n",
      "400050\n",
      "400076\n",
      "400076\n",
      "400080\n",
      "400022\n",
      "400052\n",
      "400067\n",
      "400050\n",
      "400050\n",
      "400050\n",
      "400606\n",
      "401303\n",
      "400097\n",
      "400050\n",
      "400050\n",
      "000000\n",
      "400706\n",
      "400070\n",
      "400070\n",
      "400070\n",
      "400070\n",
      "400070\n",
      "400070\n",
      "400070\n",
      "400070\n",
      "400070\n",
      "400070\n",
      "400070\n",
      "400070\n",
      "400070\n",
      "400070\n",
      "400070\n",
      "400050\n",
      "400706\n",
      "400070\n",
      "400086\n",
      "400086\n",
      "400077\n",
      "400070\n",
      "400007\n",
      "400094\n",
      "400094\n",
      "400094\n",
      "400094\n",
      "400005\n",
      "400080\n",
      "400005\n",
      "400005\n",
      "400706\n",
      "000000\n",
      "400706\n",
      "000000\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "401303\n",
      "400086\n",
      "400086\n",
      "400077\n",
      "400092\n",
      "400067\n",
      "400081\n",
      "400081\n",
      "400081\n",
      "400067\n",
      "400072\n",
      "400081\n"
     ]
    }
   ],
   "source": [
    "def update_postcode(postcode):\n",
    "    search = re.search(r'^\\D*(\\d{6}).*',postcode)\n",
    "    search1 = re.search('^(4)(0)\\d{4}$',postcode)\n",
    "    if search1:\n",
    "        return search.group(1) \n",
    "    else :\n",
    "        return '000000'\n",
    "for i in tree.getiterator('tag'):\n",
    "    k1 = i.get(\"k\")\n",
    "    if k1 == \"addr:postcode\":\n",
    "        v1 = i.get(\"v\")\n",
    "        print  update_postcode(v1)\n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Parsing the XML File and Writing the Data to a CSV\n",
    "\n",
    "In order to begin moving our data towards a SQL database for analysis, I first had to parse the XML file and transform it from a document format to a tabular format. I was then able to write the data across multiple (5) .csv files, allowing us to easily import it to a SQL database. It is in this script where the data cleaning functions are applied.\n",
    "\n",
    "Below is the data. In this code portion of the shape_element function, handles the bulk of the format shaping and utilizes the audit functions I displayed above to clean the data as well. This portion focuses on shaping way_tags and nodes_tags from the XML file into a Python dictionary.Process_map function is used to write data into csv file.In shape_element function update_postcode and update_name functions are called."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import csv\n",
    "import codecs\n",
    "import pprint\n",
    "import re\n",
    "import xml.etree.cElementTree as ET\n",
    "\n",
    "import cerberus\n",
    "\n",
    "import schema\n",
    "\n",
    "OSM_PATH = \"mumbai_india.osm\"\n",
    "\n",
    "NODES_PATH = \"nodes.csv\"\n",
    "NODE_TAGS_PATH = \"nodes_tags.csv\"\n",
    "WAYS_PATH = \"ways.csv\"\n",
    "WAY_NODES_PATH = \"ways_nodes.csv\"\n",
    "WAY_TAGS_PATH = \"ways_tags.csv\"\n",
    "\n",
    "LOWER_COLON = re.compile(r'^([a-z]|_)+:([a-z]|_)+')\n",
    "PROBLEMCHARS = re.compile(r'[=\\+/&<>;\\'\"\\?%#$@\\,\\. \\t\\r\\n]')\n",
    "\n",
    "SCHEMA = schema.schema\n",
    "\n",
    "# Make sure the fields order in the csvs matches the column order in the sql table schema\n",
    "NODE_FIELDS = ['id', 'lat', 'lon', 'user', 'uid', 'version', 'changeset', 'timestamp']\n",
    "NODE_TAGS_FIELDS = ['id', 'key', 'value', 'type']\n",
    "WAY_FIELDS = ['id', 'user', 'uid', 'version', 'changeset', 'timestamp']\n",
    "WAY_TAGS_FIELDS = ['id', 'key', 'value', 'type']\n",
    "WAY_NODES_FIELDS = ['id', 'node_id', 'position']\n",
    "def shape_element(element, node_attr_fields=NODE_FIELDS, way_attr_fields=WAY_FIELDS,\n",
    "                  problem_chars=PROBLEMCHARS, default_tag_type='regular'):\n",
    "    \"\"\"Clean and shape node or way XML element to Python dict\"\"\"\n",
    "    node_attribs = {}\n",
    "    way_attribs = {}\n",
    "    way_nodes = []\n",
    "    tags = []  # Handle secondary tags the same way for both node and way elements\n",
    "  \n",
    "    # YOUR CODE HERE\n",
    "    \n",
    "    if element.tag == 'node':\n",
    "        for i in NODE_FIELDS:\n",
    "            node_attribs[i] = element.attrib[i]\n",
    "        for tag in element.iter(\"tag\"):\n",
    "            tag_dict= {}\n",
    "            # Calling the cleaning function\n",
    "            # replace name with the value of the attribute\n",
    "            if tag.attrib['k'] == 'addr:street':\n",
    "                tag.attrib['v'] = update_name(tag.attrib['v'], mapping)\n",
    "            if tag.attrib[\"k\"] == 'addr:postcode':\n",
    "                tag.attrib[\"v\"] = update_postcode(tag.attrib[\"v\"])\n",
    "            \n",
    "            tag_dict['id'] = node_attribs['id']\n",
    "            key = tag.attrib['k']\n",
    "            \n",
    "            if re.search(PROBLEMCHARS, tag.attrib['k']):\n",
    "                pass\n",
    "            if re.search(LOWER_COLON, tag.attrib['k']):\n",
    "                pass\n",
    "            if ':' in tag.attrib['k']:\n",
    "                type = key[: key.index(':')]\n",
    "                key = key[key.index(':')+1 :]   \n",
    "            else:\n",
    "                type = 'regular'   \n",
    "            \n",
    "            tag_dict['key'] = key\n",
    "            tag_dict['value'] = tag.attrib['v']\n",
    "            tag_dict['type'] = type\n",
    "            tags.append(tag_dict)\n",
    "            \n",
    "            #pprint.pprint(dict(tag_dict))\n",
    "            \n",
    "        return {'node': node_attribs, 'node_tags': tags}\n",
    "    \n",
    "    elif element.tag == 'way':\n",
    "        for way in WAY_FIELDS:\n",
    "            way_attribs[way] = element.attrib[way]\n",
    "        for tag in element.iter(\"tag\"):\n",
    "            tag_dict1= {}\n",
    "\n",
    "            if tag.attrib[\"k\"] == 'addr:postcode':\n",
    "                tag.attrib[\"v\"] = update_postcode(tag.attrib[\"v\"])\n",
    "            tag_dict1['id'] = way_attribs['id']\n",
    "            key = tag.attrib['k']\n",
    "            \n",
    "            if re.search(PROBLEMCHARS, tag.attrib['k']):\n",
    "                pass\n",
    "            if re.search(LOWER_COLON, tag.attrib['k']):\n",
    "                pass\n",
    "            if ':' in tag.attrib['k']:\n",
    "                type = key[: key.index(':')]\n",
    "                key = key[key.index(':')+1 :]\n",
    "            else:\n",
    "                type = 'regular' \n",
    "            tag_dict1['key'] = key\n",
    "            tag_dict1['value'] = tag.attrib['v']\n",
    "            tag_dict1['type'] = type\n",
    "            tags.append(tag_dict1) \n",
    "            \n",
    "        i= 0\n",
    "        for tag in element.iter(\"nd\"):\n",
    "            way_dict = {}\n",
    "            way_dict[\"id\"] = way_attribs[\"id\"]\n",
    "            way_dict[\"node_id\"] = tag.attrib[\"ref\"]\n",
    "            way_dict[\"position\"] = i\n",
    "            way_nodes.append(way_dict)\n",
    "            i +=1    \n",
    "        \n",
    "        # pprint.pprint(dict(way_dict)) \n",
    "            \n",
    "        return {'way': way_attribs, 'way_nodes': way_nodes, 'way_tags': tags}\n",
    "\n",
    "# ================================================== #\n",
    "#               Helper Functions                     #\n",
    "# ================================================== #\n",
    "def get_element(osm_file, tags=('node', 'way', 'relation')):\n",
    "    \"\"\"Yield element if it is the right type of tag\"\"\"\n",
    "\n",
    "    context = ET.iterparse(osm_file, events=('start', 'end'))\n",
    "    _, root = next(context)\n",
    "    for event, elem in context:\n",
    "        if event == 'end' and elem.tag in tags:\n",
    "            yield elem\n",
    "            root.clear()\n",
    "\n",
    "\n",
    "def validate_element(element, validator, schema=SCHEMA):\n",
    "    \"\"\"Raise ValidationError if element does not match schema\"\"\"\n",
    "    if validator.validate(element, schema) is not True:\n",
    "        field, errors = next(validator.errors.iteritems())\n",
    "        message_string = \"\\nElement of type '{0}' has the following errors:\\n{1}\"\n",
    "        error_string = pprint.pformat(errors)\n",
    "        \n",
    "        raise Exception(message_string.format(field, error_string))\n",
    "\n",
    "\n",
    "class UnicodeDictWriter(csv.DictWriter, object):\n",
    "    \"\"\"Extend csv.DictWriter to handle Unicode input\"\"\"\n",
    "\n",
    "    def writerow(self, row):\n",
    "        super(UnicodeDictWriter, self).writerow({\n",
    "            k: (v.encode('utf-8') if isinstance(v, unicode) else v) for k, v in row.iteritems()\n",
    "        })\n",
    "\n",
    "    def writerows(self, rows):\n",
    "        for row in rows:\n",
    "            self.writerow(row)\n",
    "\n",
    "\n",
    "# ================================================== #\n",
    "#               Main Function                        #\n",
    "# ================================================== #\n",
    "def process_map(file_in, validate):\n",
    "    \"\"\"Iteratively process each XML element and write to csv(s)\"\"\"\n",
    "\n",
    "    with codecs.open(NODES_PATH, 'wb') as nodes_file, \\\n",
    "         codecs.open(NODE_TAGS_PATH, 'wb') as nodes_tags_file, \\\n",
    "         codecs.open(WAYS_PATH, 'wb') as ways_file, \\\n",
    "         codecs.open(WAY_NODES_PATH, 'wb') as way_nodes_file, \\\n",
    "         codecs.open(WAY_TAGS_PATH, 'wb') as way_tags_file:\n",
    "\n",
    "        nodes_writer = UnicodeDictWriter(nodes_file, NODE_FIELDS)\n",
    "        node_tags_writer = UnicodeDictWriter(nodes_tags_file, NODE_TAGS_FIELDS)\n",
    "        ways_writer = UnicodeDictWriter(ways_file, WAY_FIELDS)\n",
    "        way_nodes_writer = UnicodeDictWriter(way_nodes_file, WAY_NODES_FIELDS)\n",
    "        way_tags_writer = UnicodeDictWriter(way_tags_file, WAY_TAGS_FIELDS)\n",
    "\n",
    "        nodes_writer.writeheader()\n",
    "        node_tags_writer.writeheader()\n",
    "        ways_writer.writeheader()\n",
    "        way_nodes_writer.writeheader()\n",
    "        way_tags_writer.writeheader()\n",
    "\n",
    "        validator = cerberus.Validator()\n",
    "\n",
    "        for element in get_element(file_in, tags=('node', 'way')):\n",
    "            el = shape_element(element)\n",
    "            if el:\n",
    "                if validate is True:\n",
    "                    validate_element(el, validator)\n",
    "\n",
    "                if element.tag == 'node':\n",
    "                    nodes_writer.writerow(el['node'])\n",
    "                    node_tags_writer.writerows(el['node_tags'])\n",
    "                elif element.tag == 'way':\n",
    "                    ways_writer.writerow(el['way'])\n",
    "                    way_nodes_writer.writerows(el['way_nodes'])\n",
    "                    way_tags_writer.writerows(el['way_tags'])\n",
    "\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    # Note: Validation is ~ 10X slower. For the project consider using a small\n",
    "    # sample of the map when validating.\n",
    "    process_map(OSM_PATH, validate=True)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Importing CSV Files to SQL Database\n",
    "\n",
    "With the CSV files written, I was able to begin the process of importing the data into a SQL database. In my case, I used SQLite 3 and accessed it through notebook to first create database named a Mumbai_india,then add all data from csv files to database.\n",
    "\n",
    "There are two steps I needed to accomplish to complete this task:\n",
    "\n",
    "1. Create five tables on sqlite based on a schema that matches their respective .csv files\n",
    "2. Import each .csv file into the appropriate table"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import csv, sqlite3\n",
    "from pprint import pprint\n",
    "sql_file=\"Mumbai_india.db\"\n",
    "con = sqlite3.connect(sql_file)\n",
    "cur = con.cursor()\n",
    "\n",
    "cur.execute('''DROP TABLE IF EXISTS nodes;''')\n",
    "con.commit()\n",
    "\n",
    "\n",
    "cur.execute(\"CREATE TABLE nodes(id INTEGER, lat REAL, lon REAL, user TEXT, uid INTEGER, version TEXT, changeset INTEGER, timestamp DATE);\") # use your column names here\n",
    "con.commit()\n",
    "with open('nodes.csv','rb') as thr: # `with` statement available in 2.5+\n",
    "    # csv.DictReader uses first line in file for column headings by default\n",
    "    dr = csv.DictReader(thr) # comma is default delimiter\n",
    "    to_db = [(i['id'].decode(\"utf-8\"),i['lat'].decode(\"utf-8\"),i['lon'].decode(\"utf-8\"),i['user'].decode(\"utf-8\"),i['uid'].decode(\"utf-8\"),i['version'].decode(\"utf-8\"),i['changeset'].decode(\"utf-8\"),i['timestamp'].decode(\"utf-8\")) for i in dr]\n",
    "\n",
    "cur.executemany(\"INSERT INTO nodes (id, lat, lon, user, uid, version, changeset, timestamp) VALUES (?,?,?,?,?,?,?,?);\", to_db)\n",
    "con.commit()\n",
    "#con.close()\n",
    "\n",
    "\n",
    "cur.execute('''DROP TABLE IF EXISTS nodes_tags;''')\n",
    "con.commit()\n",
    "\n",
    "cur.execute(\"CREATE TABLE Nodes_tags (id INTEGER, key TEXT, value TEXT, type TEXT);\") # use your column names here\n",
    "con.commit()\n",
    "with open('nodes_tags.csv','rb') as thr: # `with` statement available in 2.5+\n",
    "    # csv.DictReader uses first line in file for column headings by default\n",
    "    dr = csv.DictReader(thr) # comma is default delimiter\n",
    "    to_db = [(i['id'].decode(\"utf-8\"),i['key'].decode(\"utf-8\"),i['value'].decode(\"utf-8\"),i['type'].decode(\"utf-8\")) for i in dr]\n",
    "\n",
    "cur.executemany(\"INSERT INTO Nodes_tags (id, key, value, type) VALUES (?,?,?,?);\", to_db)\n",
    "con.commit()\n",
    "\n",
    "cur.execute('''DROP TABLE IF EXISTS ways;''')\n",
    "con.commit()\n",
    "\n",
    "\n",
    "cur.execute(\"CREATE TABLE ways (id INTEGER, user TEXT, uid INTEGER, version TEXT, changeset INTEGER, timestamp DATE);\") # use your column names here\n",
    "con.commit()\n",
    "with open('ways.csv','rb') as thr: # `with` statement available in 2.5+\n",
    "    # csv.DictReader uses first line in file for column headings by default\n",
    "    dr = csv.DictReader(thr) # comma is default delimiter\n",
    "    to_db = [(i['id'].decode(\"utf-8\"),i['user'].decode(\"utf-8\"),i['uid'].decode(\"utf-8\"),i['version'].decode(\"utf-8\"),i['changeset'].decode(\"utf-8\"),i['timestamp'].decode(\"utf-8\")) for i in dr]\n",
    "\n",
    "cur.executemany(\"INSERT INTO ways (id , user, uid, version, changeset, timestamp) VALUES (?,?,?,?,?,?);\", to_db)\n",
    "con.commit()\n",
    "\n",
    "\n",
    "cur.execute('''DROP TABLE IF EXISTS ways_tags;''')\n",
    "con.commit()\n",
    "\n",
    "\n",
    "cur.execute(\"CREATE TABLE ways_tags (id INTEGER, key TEXT, value TEXT, type TEXT);\") # use your column names here\n",
    "con.commit()\n",
    "with open('ways_tags.csv','rb') as thr: # `with` statement available in 2.5+\n",
    "    # csv.DictReader uses first line in file for column headings by default\n",
    "    dr = csv.DictReader(thr) # comma is default delimiter\n",
    "    to_db = [(i['id'].decode(\"utf-8\"),i['key'].decode(\"utf-8\"),i['value'].decode(\"utf-8\"),i['type'].decode(\"utf-8\")) for i in dr]\n",
    "\n",
    "\n",
    "cur.executemany(\"INSERT INTO ways_tags (id, key, value, type) VALUES (?,?,?,?);\", to_db)\n",
    "con.commit()\n",
    "\n",
    "\n",
    "\n",
    "cur.execute('''DROP TABLE IF EXISTS ways_nodes;''')\n",
    "con.commit()\n",
    "\n",
    "cur.execute(\"CREATE TABLE ways_nodes (id INTEGER, node_id INTEGER, position INTEGER);\") # use your column names here\n",
    "con.commit()\n",
    "with open('ways_nodes.csv','rb') as thr: # `with` statement available in 2.5+\n",
    "    # csv.DictReader uses first line in file for column headings by default\n",
    "    dr = csv.DictReader(thr) # comma is default delimiter\n",
    "    to_db = [(i['id'].decode(\"utf-8\"),i['node_id'].decode(\"utf-8\"),i['position'].decode(\"utf-8\")) for i in dr]\n",
    "    \n",
    "cur.executemany(\"INSERT INTO ways_nodes (id, node_id, position) VALUES (?,?,?);\", to_db)\n",
    "con.commit()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Overview\n",
    "In this section, I'll execute a number of SQL queries in order to analyze the dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Users with most numbers of contributions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[(u'parambyte', 69327),\n",
      " (u'PlaneMad', 68363),\n",
      " (u'anushap', 62553),\n",
      " (u'Ashok09', 62208),\n",
      " (u'Narsimulu', 55611)]\n"
     ]
    }
   ],
   "source": [
    "sql_file=\"mumbai_india.db\"\n",
    "con = sqlite3.connect(sql_file)\n",
    "cur = con.cursor()\n",
    "cur.execute('SELECT user,count(*) FROM nodes GROUP BY user ORDER BY count(*) DESC LIMIT 5')\n",
    "all_rows=cur.fetchall()\n",
    "pprint(all_rows)\n",
    "\n",
    "\n",
    "con.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[(u'building', 223631),\n",
      " (u'highway', 40437),\n",
      " (u'name', 11741),\n",
      " (u'oneway', 4466),\n",
      " (u'source', 4142)]\n"
     ]
    }
   ],
   "source": [
    "sql_file=\"mumbai_india.db\"\n",
    "con = sqlite3.connect(sql_file)\n",
    "cur = con.cursor()\n",
    "cur.execute('SELECT key,count(*) FROM ways_tags GROUP BY key ORDER BY count(*) DESC LIMIT 5')\n",
    "all_rows=cur.fetchall()\n",
    "pprint(all_rows)\n",
    "\n",
    "\n",
    "con.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Total number of unique users."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[(1739,)]\n"
     ]
    }
   ],
   "source": [
    "sql_file=\"mumbai_india.db\"\n",
    "con = sqlite3.connect(sql_file)\n",
    "cur = con.cursor()\n",
    "cur.execute('SELECT COUNT(DISTINCT(e.uid)) FROM (SELECT uid FROM nodes UNION ALL SELECT uid FROM ways) e')\n",
    "all_rows=cur.fetchall()\n",
    "pprint(all_rows)\n",
    "\n",
    "\n",
    "con.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Total number of users who contributed for less than 5 times."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[(910,)]\n"
     ]
    }
   ],
   "source": [
    "sql_file=\"mumbai_india.db\"\n",
    "con = sqlite3.connect(sql_file)\n",
    "cur = con.cursor()\n",
    "cur.execute('SELECT COUNT(*) FROM (SELECT e.user, COUNT(*) as num FROM (SELECT user FROM nodes UNION ALL SELECT user FROM ways) as e GROUP BY e.user HAVING num<=5)  u')\n",
    "all_rows=cur.fetchall()\n",
    "pprint(all_rows)\n",
    "\n",
    "\n",
    "con.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Different types of cuisines available in mumbai and number of restaurants it is available in."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[(u'indian', 66),\n",
      " (u'regional', 21),\n",
      " (u'pizza', 14),\n",
      " (u'vegetarian', 13),\n",
      " (u'chinese', 12),\n",
      " (u'italian', 11),\n",
      " (u'burger', 5),\n",
      " (u'international', 5),\n",
      " (u'seafood', 4),\n",
      " (u'asian', 3),\n",
      " (u'South_Indian', 2),\n",
      " (u'all_types_of_food', 2),\n",
      " (u'lebanese', 2),\n",
      " (u'Indian,_Chinese_etc', 1),\n",
      " (u'Seafood', 1),\n",
      " (u'Vegetarian_Restaurant', 1),\n",
      " (u'american', 1),\n",
      " (u'cafe', 1),\n",
      " (u'chicken;fish;indian', 1),\n",
      " (u'chicken;kebab;indian', 1),\n",
      " (u'chicken_,fish,cafe', 1),\n",
      " (u'fast_food', 1),\n",
      " (u'grill;coffee_shop;asian;noodles;fish_and_chips;diner;chicken;italian_pizza;indian;curry;fish;french;friture;chinese;barbecue',\n",
      "  1),\n",
      " (u'indian;south_indian', 1),\n",
      " (u'indian_aagri', 1),\n",
      " (u'italian_pizza;pizza', 1),\n",
      " (u'lebanese,_chinese,_indian', 1),\n",
      " (u'local', 1),\n",
      " (u'mediterranean', 1),\n",
      " (u'only_vegiterian', 1),\n",
      " (u'oriental', 1),\n",
      " (u'persian', 1),\n",
      " (u'sad_food', 1),\n",
      " (u'south Indian; Punjabi; agari; malwani; Chinese', 1),\n",
      " (u'south_indian', 1),\n",
      " (u'south_indian,_chinese', 1),\n",
      " (u'spanish', 1),\n",
      " (u'sweets_shop', 1),\n",
      " (u'thai', 1),\n",
      " (u'vegan;ice_cream;italian_pizza;indian;vegetarian;local', 1),\n",
      " (u'vegetarian;indian', 1)]\n"
     ]
    }
   ],
   "source": [
    "sql_file=\"mumbai_india.db\"\n",
    "con = sqlite3.connect(sql_file)\n",
    "cur = con.cursor()\n",
    "cur.execute('SELECT nodes_tags.value, COUNT(*) as num FROM nodes_tags JOIN (SELECT DISTINCT(id) FROM nodes_tags WHERE value=\"restaurant\") as i ON nodes_tags.id=i.id WHERE nodes_tags.key=\"cuisine\" GROUP BY nodes_tags.value ORDER BY num DESC')\n",
    "all_rows=cur.fetchall()\n",
    "pprint(all_rows)\n",
    "\n",
    "\n",
    "con.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Different religions and number of places where they are workshipped."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[(u'hindu', 125),\n",
      " (u'muslim', 71),\n",
      " (u'christian', 34),\n",
      " (u'buddhist', 13),\n",
      " (u'jain', 6),\n",
      " (u'sikh', 4),\n",
      " (u'zoroastrian', 2),\n",
      " (u'jewish', 1)]\n"
     ]
    }
   ],
   "source": [
    "sql_file=\"mumbai_india.db\"\n",
    "con = sqlite3.connect(sql_file)\n",
    "cur = con.cursor()\n",
    "cur.execute('SELECT nodes_tags.value, COUNT(*) as num FROM nodes_tags  JOIN (SELECT DISTINCT(id) FROM nodes_tags WHERE value=\"place_of_worship\") as i ON nodes_tags.id=i.id WHERE nodes_tags.key=\"religion\" GROUP BY nodes_tags.value ORDER BY num DESC')\n",
    "all_rows=cur.fetchall()\n",
    "pprint(all_rows)\n",
    "\n",
    "\n",
    "con.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Different leisures"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[(u'park', 63),\n",
      " (u'playground', 20),\n",
      " (u'sports_centre', 15),\n",
      " (u'garden', 10),\n",
      " (u'fitness_centre', 8),\n",
      " (u'pitch', 6),\n",
      " (u'swimming_pool', 4),\n",
      " (u'aquarium', 1),\n",
      " (u'fitness_station', 1),\n",
      " (u'golf_course', 1),\n",
      " (u'stadium', 1)]\n"
     ]
    }
   ],
   "source": [
    "sql_file=\"mumbai_india.db\"\n",
    "con = sqlite3.connect(sql_file)\n",
    "cur = con.cursor()\n",
    "cur.execute('SELECT nodes_tags.value, count(*) as num FROM nodes_tags  WHERE nodes_tags.key=\"leisure\" GROUP BY nodes_tags.value ORDER BY num DESC')\n",
    "all_rows=cur.fetchall()\n",
    "pprint(all_rows)\n",
    "\n",
    "\n",
    "con.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Names of few corrected streets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[(u'Hanuman Road', 78),\n",
      " (u'Yashavant Nagar Road', 29),\n",
      " (u'Hiranandani Estate', 24),\n",
      " (u'P.L. Lokhande Marg', 24),\n",
      " (u'New Link Road, Andheri West', 21),\n",
      " (u'LBS Marg', 18),\n",
      " (u'Road Number 3', 18),\n",
      " (u'Thane Ghodbunder Road', 18),\n",
      " (u'Eastern Express Highway', 14),\n",
      " (u'GD Somani Road', 13)]\n"
     ]
    }
   ],
   "source": [
    "sql_file=\"mumbai_india.db\"\n",
    "con = sqlite3.connect(sql_file)\n",
    "cur = con.cursor()\n",
    "cur.execute('SELECT value, count(*) as num FROM nodes_tags WHERE key=\"street\" GROUP BY value ORDER BY num DESC LIMIT 10')\n",
    "\n",
    "all_rows=cur.fetchall()\n",
    "pprint(all_rows)\n",
    "\n",
    "\n",
    "con.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Cities surrounding mumbai and number of nodes in those cities."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[(u'Mumbai', 607),\n",
      " (u'Bandra, Mumbai', 566),\n",
      " (u'mumbai', 187),\n",
      " (u'Virar West', 91),\n",
      " (u'Mulund (West)', 79),\n",
      " (u'Navi Mumbai', 70),\n",
      " (u'MUMBAI', 68),\n",
      " (u'Mulund (East)', 62),\n",
      " (u'Thane', 49),\n",
      " (u'Kharghar', 43)]\n"
     ]
    }
   ],
   "source": [
    "sql_file=\"mumbai_india.db\"\n",
    "con = sqlite3.connect(sql_file)\n",
    "cur = con.cursor()\n",
    "cur.execute('SELECT tags.value, COUNT(*) as count FROM (SELECT * FROM nodes_tags UNION ALL  SELECT * FROM ways_tags) tags WHERE tags.key LIKE \"%city\" GROUP BY tags.value ORDER BY count DESC LIMIT 10')\n",
    "\n",
    "all_rows=cur.fetchall()\n",
    "pprint(all_rows)\n",
    "\n",
    "\n",
    "con.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Different postcodes in ways_tags after using update postcode function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[(u'400007', 1),\n",
      " (u'400010', 1),\n",
      " (u'400011', 1),\n",
      " (u'400034', 1),\n",
      " (u'400035', 1),\n",
      " (u'400038', 1),\n",
      " (u'400047', 1),\n",
      " (u'400059', 1),\n",
      " (u'400060', 1),\n",
      " (u'400066', 1),\n",
      " (u'400082', 1),\n",
      " (u'400095', 1),\n",
      " (u'400096', 1),\n",
      " (u'400602', 1),\n",
      " (u'400605', 1),\n",
      " (u'400703', 1),\n",
      " (u'400709', 1),\n",
      " (u'400710', 1),\n",
      " (u'400950', 1),\n",
      " (u'401104', 1),\n",
      " (u'401105', 1),\n",
      " (u'401209', 1),\n",
      " (u'402205', 1),\n",
      " (u'400015', 2),\n",
      " (u'400020', 2),\n",
      " (u'400025', 2),\n",
      " (u'400030', 2),\n",
      " (u'400054', 2),\n",
      " (u'400055', 2),\n",
      " (u'400068', 2),\n",
      " (u'400069', 2),\n",
      " (u'400079', 2),\n",
      " (u'400091', 2),\n",
      " (u'400098', 2),\n",
      " (u'400103', 2),\n",
      " (u'400612', 2),\n",
      " (u'400701', 2),\n",
      " (u'401101', 2),\n",
      " (u'400008', 3),\n",
      " (u'400024', 3),\n",
      " (u'400031', 3),\n",
      " (u'400037', 3),\n",
      " (u'400042', 3),\n",
      " (u'400056', 3),\n",
      " (u'400063', 3),\n",
      " (u'400092', 3),\n",
      " (u'400101', 3),\n",
      " (u'400102', 3),\n",
      " (u'400708', 3),\n",
      " (u'401202', 3),\n",
      " (u'400052', 4),\n",
      " (u'400077', 4),\n",
      " (u'400104', 4),\n",
      " (u'400606', 4),\n",
      " (u'400705', 4),\n",
      " (u'401201', 4),\n",
      " (u'400016', 5),\n",
      " (u'400022', 5),\n",
      " (u'400049', 5),\n",
      " (u'400062', 5),\n",
      " (u'400078', 5),\n",
      " (u'400097', 5),\n",
      " (u'400099', 5),\n",
      " (u'401107', 5),\n",
      " (u'400013', 6),\n",
      " (u'400014', 6),\n",
      " (u'400018', 6),\n",
      " (u'400028', 6),\n",
      " (u'400058', 6),\n",
      " (u'400075', 7),\n",
      " (u'400607', 7),\n",
      " (u'400601', 8),\n",
      " (u'400012', 9),\n",
      " (u'400072', 9),\n",
      " (u'400086', 9),\n",
      " (u'400043', 10),\n",
      " (u'400021', 11),\n",
      " (u'400094', 12),\n",
      " (u'400001', 13),\n",
      " (u'400004', 13),\n",
      " (u'400061', 13),\n",
      " (u'400076', 14),\n",
      " (u'400706', 14),\n",
      " (u'400067', 15),\n",
      " (u'400093', 15),\n",
      " (u'400051', 17),\n",
      " (u'400064', 19),\n",
      " (u'400089', 19),\n",
      " (u'400071', 20),\n",
      " (u'400088', 21),\n",
      " (u'400002', 24),\n",
      " (u'400080', 26),\n",
      " (u'400074', 28),\n",
      " (u'400057', 36),\n",
      " (u'400070', 38),\n",
      " (u'400081', 38),\n",
      " (u'400053', 44),\n",
      " (u'400005', 47),\n",
      " (u'401303', 85),\n",
      " (u'000000', 88),\n",
      " (u'400050', 551)]\n"
     ]
    }
   ],
   "source": [
    "sql_file=\"mumbai_india.db\"\n",
    "con = sqlite3.connect(sql_file)\n",
    "cur = con.cursor()\n",
    "cur.execute('SELECT value, COUNT(*) as count FROM ways_tags WHERE key=\"postcode\" GROUP BY value ORDER BY count')\n",
    "\n",
    "all_rows=cur.fetchall()\n",
    "pprint(all_rows)\n",
    "\n",
    "\n",
    "con.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Potential Additional Improvements\n",
    "\n",
    "There are several areas of improvement of the project in the future. The first one is on the completeness of the data. All the above analysis is based on a dataset that reflects a big part of mumbai but not only mumbai. The reason for this is the lack of a way to download a dataset for the entire mumbai without including parts of the neighboring cities. The analyst has to either select a part of the island/city or select a wider area that includes parts of thane and ratnagiri. Also, because of relations between nodes, ways, and relations, the downloaded data expand much further than the actual selection.\n",
    "\n",
    "As a future improvement, I would download a wider selection or the metro extract from MapZen and filter the non-mumbai nodes and their references. The initial filtering could take place by introducing some latitude/longitude limits in the code to sort out most of the \"non-m\" nodes.\n",
    "\n",
    "The second area with room for future improvement is the exploratory analysis of the dataset. Just to mention some of the explorings that could take place:\n",
    "\n",
    "1.Distribution of commits per contributor.\n",
    "\n",
    "2.Plotting of element creation per type, per day.\n",
    "\n",
    "3.Distribution of distance between different types of amenities\n",
    "\n",
    "4.Popular franchises in the country (fast food, conventional stores, etc.)\n",
    "\n",
    "5.Selection of a bank based on the average distance you have to walk for an ATM.\n",
    "\n",
    "6.Which area has the biggest parks and recreation spaces.\n",
    "\n",
    "The scope of the current project was the wrangling of the dataset, so all the above have been left for future improvement.\n",
    "\n",
    "Increasing Submissions\n",
    "\n",
    "Going through this dataset, my concerns were less with the cleanliness of the data - as I found it surprisingly clean - and more with the lack of data. This part of mumbai is too big to have as little information as it does. I think OpenStreetMap can go a long way in developing their map database if they took on certain initiative to increase engagement with their service. One possible initiative would be for OpenStreetMap to form partnerships with educational institutions such as schools, or maybe libraries, to engage students with their service. As a way to develop computer and internet literacy, computer-related courses can teach students how to use OpenStreetMap. It'll expose them to online maps, GPS technology, how to participate in open source projects, and more - all while adding data to a free resource that could benefit the members of the community and the world.\n",
    "\n",
    "Anticipated Problem: However, the concern here is that you might see an influx of dirty, unreliable data, particularly if the people behind them aren't very computer literate or only participating because it's a mandatory portion of a course. Naturally the data that come from volunteers who get involved because of their genuine passion for the project would be of higher quality.\n",
    "\n",
    "Ensuring Data Consistency\n",
    "\n",
    "For data improvement, the biggest problem I came across my data before I cleaned it was the lack of a unified format for street types or phone numbers, or simply incomplete information. If OpenStreetMap had a hard format that street types, phone numbers, zip codes, etc. should follow - and they ensured the format is appropriate for the city/country - there would be much cleaner data for analysis.\n",
    "\n",
    "Anticipated Problem: The issue here is that you could very likely see a decrease in submissions due to the stricter guidelines.\n",
    "\n",
    "Finally, open data are here to make average people's life better. For the non-data analyst, it would be nice if there was an application (mobile or web) that could evaluate the suitability of a potential rental house. The work addresses of all family members, importance weights on several amenities like supermarkets, convenience stores, cafes, public transportation, etc. and the application would calculate the suitability of each potential rental. The user would be able to sort them by score and compare them."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Conclusion\n",
    "\n",
    "It's clear from what we've seen that the mumbai OpenStretMap data is still incomplete and incorrect but there is still much in this city to be found and explored. The upside is that a lot of the data that has been entered is fairly clean, so future OSM users who embark on the task of improving the dataset with new information won't have much to worry about with regards to cleaning prior submissions. I'm looking forward to seeing how the data for the area grows overtime, and I hope to also play a role in that mission myself."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}