re_tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d5c3ad02",
   "metadata": {},
   "source": [
    "## In this repository we will explore the basic functionality of python re library."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8f3a9240",
   "metadata": {},
   "source": [
    "## But first, a quick reminder of regex\n",
    "### These are the most common regular expression operators by category, and most of the time you'll use them to create your regular expressions. However, there are many more specialized operators that can be used for more advanced pattern matching."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5fd7f841",
   "metadata": {},
   "source": [
    "### Anchors"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "09f580cf",
   "metadata": {},
   "source": [
    "^: Matches the start of a line  \n",
    "$: Matches the end of a line"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2fc2599d",
   "metadata": {},
   "source": [
    "### Character classes"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "96e375ed",
   "metadata": {},
   "source": [
    "\\w: Matches any word character (alphanumeric and underscore)  \n",
    "\\d: Matches any digit  \n",
    "\\s: Matches any whitespace  \n",
    "\\b: Matches a word boundary  \n",
    "\\W: Matches any non-word character  \n",
    "\\D: Matches any non-digit character  \n",
    "\\S: Matches any non-whitespace character  \n",
    "\\B: Matches a non-word boundary  \n",
    "  \n",
    "[]: Matches any character inside the brackets  \n",
    "[a-z]: Matches any lowercase letter  \n",
    "[A-Z]: Matches any uppercase letter  \n",
    "[0-9]: Matches any digit  \n",
    "[a-zA-z0-9]: Matches any letter or digit  \n",
    "  \n",
    "\\p{Punct}: Matches any punctuation character  \n",
    "\\p{Lower}: Matches any lowercase letter  \n",
    "\\p{Upper}: Matches any uppercase letter  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "19f3a17b",
   "metadata": {},
   "source": [
    "### Quantifiers"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "796027d0",
   "metadata": {},
   "source": [
    "*: Matches zero or more of the preceding element  \n",
    "+: Matches one or more of the preceding element  \n",
    "?: Matches zero or one of the preceding element  \n",
    "{n}: Matches exactly n of the preceding element  \n",
    "{n,}: Matches n or more of the preceding element  \n",
    "{n,m}: Matches at least n and at most m of the preceding element  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "58debe9e",
   "metadata": {},
   "source": [
    "### Special characters"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "97251202",
   "metadata": {},
   "source": [
    ".: Matches any character except newline  \n",
    "\\\\: Escapes the following character  \n",
    "|: Alternation operator, matches the preceding or the next pattern (Alternation)  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1a81e6e",
   "metadata": {},
   "source": [
    "### Groups"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0990fee7",
   "metadata": {},
   "source": [
    "(...): Creates a capture group  \n",
    "(?:...): Creates a non-capturing group  \n",
    "  \n",
    "Lookaround:  \n",
    "(?=...): Positive lookahead  \n",
    "(?!...): Negative lookahead  \n",
    "(?<=...): Positive lookbehind  \n",
    "(?<!...): Negative lookbehind  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef379ce9",
   "metadata": {},
   "source": [
    "### The main re methods"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79cf9e8c",
   "metadata": {},
   "source": [
    "- compile(pattern, flags=0): This method is used to create a regular expression object \n",
    "\n",
    "- match(string): This method matches the pattern only at the beginning of the string and returns a match object if found.\n",
    "\n",
    "- search(string): This method searches for the first occurrence of a pattern in a string and returns a match object if found.\n",
    "\n",
    "- findall(string): This method returns a list of all non-overlapping matches of the pattern in the string as a list of strings.\n",
    "\n",
    "- finditer(string): This method returns an iterator yielding match objects for all non-overlapping matches of the pattern in the string.\n",
    "\n",
    "- sub(repl, string): This method replaces all occurrences of the pattern in the string with a replacement string.\n",
    "\n",
    "- split(string): This method splits the string by the occurrences of the pattern.\n",
    "\n",
    "\n",
    "### The methods can be either called directly from re library ( re.search(regex_pattern, string) ) or as a pattern object method ( re.compile(regex_pattern).search(string) ).\n",
    "### In this tutorial we will follow the latter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "b90fe338",
   "metadata": {},
   "outputs": [],
   "source": [
    "import re"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "id": "9e7903a4",
   "metadata": {},
   "outputs": [],
   "source": [
    "string = '''\n",
    "The Cat in the hat, sat there with a fat rat. He ate a mat, and a bat came out of nowhere to attack.\n",
    "He had 2 cats, one was black and one was white. He also had 4 dogs, 2 of them were big and 2 of them were small.\n",
    "He wanted to buy a new Dog, but he couldn't decide between a Labrador or a Golden Retriever.\n",
    "In the end, he decided to adopt a stray Cat from tshe street. He painted his room in #ff0000 color.\n",
    "His email address is moew@gmail.com and his phone numbers 123-321-5555, 555.555.5555, 555555-5555, 555555.5555, 1234560000.\n",
    "He was born on 12/05/2018, and he graduated on 20/01/2023. He also has a website https://www.example.com, how cool?\n",
    "He likes to shop online and his favorite store is http://store.com and also likes to shop at www.store2.com.\n",
    "His favorite color is blue and he likes to buy items that cost $93.99, $19.99, $29.99 and also $39.99.\n",
    "He also likes to buy items that have a discount of $5.99 and $9.99. He has a dog named Buddy with a collar number #123456.\n",
    "His best friend's email is bestfriend@example.com and his phone number is 001-100-5556.\n",
    "They usually meet every 15th of the month at 7:00 pm.\n",
    "My current IP address is 192.168.1.1 but I'm planning to switch to a new one, it's 10.0.0.1. \n",
    "I just made a purchase using my credit card number 4111 1111 1111 1111 with expiration date of 12/24 and security code 123.\n",
    "I can't wait to share my new book #datascience on twitter and Instagram.\n",
    "it's ISBN number is 123-4-56789-0123-5 and it will be available on Amazon and Barnes & Noble.\n",
    "My friend @johndoe lives at 123 Main St, Anytown USA 12345 and he told me that his neighbor lives at 124 Main St, Anytown USA 12345.\n",
    "I'm planning to visit my family in Los Angeles, CA on 12/24/2022 and stay there for 2 days.\n",
    "I'll be staying at my cousin's house located at 1234 Elm St, Los Angeles CA 90210!\n",
    "I'm also planning to visit my friend in New York, NY on 24-12-2022 and stay there for 5 days.\n",
    "I'll be staying at my friend's apartment located at 123 Broadway, New York NY 10001!\n",
    "I'm also planning to visit my sister in London, UK on 2022-12-24 and stay there for 6 days.\n",
    "I'll be staying at my friend's flat located at 123 Oxford St, London UK W1D 2LJ!\n",
    "I need to remember to bring my passport with expiration date of 12/24/2022, my driver's license with number\n",
    "12345678 and my insurance card with number 123456789.\n",
    "'''"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "e657c3ba",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"\\nThe Cat in the hat, sat there with a fat rat. He ate a mat, and a bat came out of nowhere to attack.\\nHe had 2 cats, one was black and one was white. He also had 4 dogs, 2 of them were big and 2 of them were small.\\nHe wanted to buy a new Dog, but he couldn't decide between a Labrador or a Golden Retriever.\\nIn the end, he decided to adopt a stray Cat from tshe street. He painted his room in #ff0000 color.\\nHis email address is moew@gmail.com and his phone numbers 123-321-5555, 555.555.5555, 555555-5555, 555555.5555, 1234560000.\\nHe was born on 12/05/2018, and he graduated on 20/01/2023. He also has a website https://www.example.com, how cool?\\nHe likes to shop online and his favorite store is http://store.com and also likes to shop at www.store2.com.\\nHis favorite color is blue and he likes to buy items that cost $93.99, $19.99, $29.99 and also $39.99.\\nHe also likes to buy items that have a discount of $5.99 and $9.99. He has a dog named Buddy with a collar number #123456.\\nHis best friend's email is bestfriend@example.com and his phone number is 001-100-5556.\\nThey usually meet every 15th of the month at 7:00 pm.\\nMy current IP address is 192.168.1.1 but I'm planning to switch to a new one, it's 10.0.0.1. \\nI just made a purchase using my credit card number 4111 1111 1111 1111 with expiration date of 12/24 and security code 123.\\nI can't wait to share my new book #datascience on twitter and Instagram.\\nit's ISBN number is 123-4-56789-0123-5 and it will be available on Amazon and Barnes & Noble.\\nMy friend @johndoe lives at 123 Main St, Anytown USA 12345 and he told me that his neighbor lives at 124 Main St, Anytown USA 12345.\\nI'm planning to visit my family in Los Angeles, CA on 12/24/2022 and stay there for 2 days.\\nI'll be staying at my cousin's house located at 1234 Elm St, Los Angeles CA 90210!\\nI'm also planning to visit my friend in New York, NY on 24-12-2022 and stay there for 5 days.\\nI'll be staying at my friend's apartment located at 123 Broadway, New York NY 10001!\\nI'm also planning to visit my sister in London, UK on 2022-12-24 and stay there for 6 days.\\nI'll be staying at my friend's flat located at 123 Oxford St, London UK W1D 2LJ!\\nI need to remember to bring my passport with expiration date of 12/24/2022, my driver's license with \\nnumber 12345678 and my insurance card with number 123456789.\\n\""
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "string"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "08223799",
   "metadata": {},
   "source": [
    "## Starter Exercises"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "904b1951",
   "metadata": {},
   "source": [
    "1. Check if string starts with blanks (using match, search, findall and finditer)\n",
    "2. Check if the text contains phone numbers in the format XXX-XXX-XXXX\n",
    "3. Check if the text contains the word cat case-insensitive\n",
    "4. Extract all the phone numbers in the format XXX-XXX-XXXX from the text.\n",
    "5. Extract all email addresses from the text.\n",
    "6. Extract all the prices in the format $999.99 from the text.\n",
    "7. Extract and create a list of all the URLs from the text.\n",
    "8. Replace all the collar numbers in the format ddd-ddd-dddd with \"ddd-XX-XXXX\"\n",
    "9. Find the phone numbers in the format ddd-ddd-dddd and swap the first 3 digits with the second 3 digits \n",
    "9. Split the text into a list of sentences (sentence tokenization). "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "763d3ef8",
   "metadata": {},
   "source": [
    "### Exercise 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "5d97a193",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<re.Match object; span=(0, 1), match='\\n'>\n",
      "<re.Match object; span=(0, 1), match='\\n'>\n",
      "['\\n']\n",
      "<re.Match object; span=(0, 1), match='\\n'>\n"
     ]
    }
   ],
   "source": [
    "# create pattern object\n",
    "pattern = re.compile(r'\\s+')\n",
    "\n",
    "# if found, returns match object None otherwise\n",
    "match = pattern.match(string)\n",
    "print(match)\n",
    "\n",
    "# crete pattern object\n",
    "pattern = re.compile(r'^\\s+')\n",
    "\n",
    "match = pattern.search(string)\n",
    "print(match)\n",
    "\n",
    "matches = pattern.findall(string)\n",
    "print(matches)\n",
    "\n",
    "matches = pattern.finditer(string)\n",
    "for match in matches:\n",
    "    print(match)\n",
    "    \n",
    "# As you can already notice, the caret operator is needed in those three methods in comparison with the match method.\n",
    "# As stated above, match method tries to match the regex object starting from the beginning of the string,\n",
    "# while the search, findall, finditer scans the whole string\n",
    "\n",
    "# This probably means that match method can be replaced by the other three, with some additional regex logic\n",
    "# For this reason, the above exercises are focused on those"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c9c0762",
   "metadata": {},
   "source": [
    "### Exercise 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "3fae10e7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Phone nubmer exists\n"
     ]
    }
   ],
   "source": [
    "pattern = re.compile(r'\\d{3}-\\d{3}-\\d{4}')\n",
    "if pattern.search(string): print(\"Phone nubmer exists\")\n",
    "\n",
    "# Equivelant alternative approaches\n",
    "# if pattern.findall(string): print(\"Phone number exists\")\n",
    "# if pattern.finditer(string): print(\"Phone number exists\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "727f94a9",
   "metadata": {},
   "source": [
    "### Exercise 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "265a6f66",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<re.Match object; span=(5, 8), match='Cat'>\n"
     ]
    }
   ],
   "source": [
    "# re.IGNORECASE is a flag. It speciries not to consider case sensitivity\n",
    "pattern = re.compile(r'cat', re.IGNORECASE)\n",
    "match = pattern.search(string)\n",
    "print(match)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14e2bef4",
   "metadata": {},
   "source": [
    "### Exercise 4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "6c78776c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['123-321-5555', '001-100-5556']\n",
      "<re.Match object; span=(466, 478), match='123-321-5555'>\n",
      "<re.Match object; span=(1057, 1069), match='001-100-5556'>\n"
     ]
    }
   ],
   "source": [
    "# since we need to extract all the possible information, we can either use findall or finditer\n",
    "pattern = re.compile(r'\\d{3}-\\d{3}-\\d{4}')\n",
    "matches = pattern.findall(string)\n",
    "print(matches)\n",
    "\n",
    "matches = pattern.finditer(string)\n",
    "for match in matches:\n",
    "    print(match)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "83b8dc91",
   "metadata": {},
   "source": [
    "### Exercise 5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "cf6d0a48",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "- The email is 'moew@gmail.com'\n",
      "More specifically, the recipient name is 'moew', the domain name is 'gmail' with Top-level domain '.com'\n",
      "\n",
      "- The email is 'bestfriend@example.com'\n",
      "More specifically, the recipient name is 'bestfriend', the domain name is 'example' with Top-level domain '.com'\n",
      "\n",
      "[('moew', 'gmail', '.com'), ('bestfriend', 'example', '.com')]\n"
     ]
    }
   ],
   "source": [
    "# regex emal pattern to find emails in text\n",
    "email_regex = r'\\b([a-zA-z0-9]+[!#$%&*+-/=?^_{|.]?[a-zA-z0-9]+)+@([a-z]+)+(\\.[A-Z|a-z]{2,})\\b'\n",
    "\n",
    "# create pattern object\n",
    "pattern = re.compile(email_regex)\n",
    "\n",
    "# use of finditer\n",
    "matches = pattern.finditer(string)\n",
    "\n",
    "# As u've already noticed, email regex consists of 3 groups\n",
    "# Recipient name (group-1), Domain name (group-2), Top-level domain (group-3)\n",
    "# re.Match object has a very powerful mathod named \"group\"\n",
    "# This method returns the text that matched a specific group (or the entire match if no group is specified (group-0) ).\n",
    "for match in matches:\n",
    "    print(f\"- The email is '{match.group(0)}'\")\n",
    "    print(f\"More specifically, the recipient name is '{match.group(1)}', the domain name is '{match.group(2)}'\", end=' ')\n",
    "    print(f\"with Top-level domain '{match.group(3)}'\", end='\\n\\n')\n",
    "    \n",
    "    \n",
    "# When groups are present in a regex, findall returns a list of tuples,\n",
    "# where each tuple contains the text matched by the groups in the order they appear in the pattern.\n",
    "print(pattern.findall(string))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "827d4504",
   "metadata": {},
   "source": [
    "### Exercise 6"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "049d7743",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['$93.99', '$19.99', '$29.99', '$39.99', '$5.99', '$9.99']\n"
     ]
    }
   ],
   "source": [
    "pattern = re.compile(r'\\$\\d+.\\d*')\n",
    "matches = pattern.findall(string)\n",
    "print(matches)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d07e4f1e",
   "metadata": {},
   "source": [
    "### Exercise 7"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "d8ca1263",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['https://www.example.com', 'http://store.com']\n",
      "['https://www.example.com', 'http://store.com']\n"
     ]
    }
   ],
   "source": [
    "# note that this url regex is for static urls\n",
    "# concatenate \\b([-a-zA-Z0-9()@:%_\\+.~#?&//=]*) to match also dynamic\n",
    "url_regex = r'https?:\\/{2}(www\\.)?[-a-zA-Z0-9@:%._\\+~#=]{1,256}\\.[a-zA-Z0-9()]{1,6}'\n",
    "\n",
    "# create pattern object\n",
    "pattern = re.compile(url_regex)\n",
    "\n",
    "# in this case, we'll use find iter since findall will return a list with the sub matches\n",
    "# and we want the full urls\n",
    "matches = pattern.finditer(string)\n",
    "\n",
    "urls = []\n",
    "for match in matches:\n",
    "    urls += [match.group()]\n",
    "print(urls)\n",
    "\n",
    "# In order to use findll method, u can modify the regex,\n",
    "# stating that the group is not a capturing group with \"(?:...)\"\n",
    "url_regex = r'https?:\\/{2}(?:www\\.)?[-a-zA-Z0-9@:%._\\+~#=]{1,256}\\.[a-zA-Z0-9()]{1,6}'\n",
    "\n",
    "# create pattern object\n",
    "pattern = re.compile(url_regex)\n",
    "\n",
    "# retrieve matches\n",
    "matches = pattern.findall(string)\n",
    "\n",
    "print(matches)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d8390a0f",
   "metadata": {},
   "source": [
    "### Exercise 8"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "6052cee5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "The Cat in the hat, sat there with a fat rat. He ate a mat, and a bat came out of nowhere to attack.\n",
      "He had 2 cats, one was black and one was white. He also had 4 dogs, 2 of them were big and 2 of them were small.\n",
      "He wanted to buy a new Dog, but he couldn't decide between a Labrador or a Golden Retriever.\n",
      "In the end, he decided to adopt a stray Cat from tshe street. He painted his room in #ff0000 color.\n",
      "His email address is moew@gmail.com and his phone numbers 123-xxx-xxxx, 555.555.5555, 555555-5555, 555555.5555, 1234560000.\n",
      "He was born on 12/05/2018, and he graduated on 20/01/2023. He also has a website https://www.example.com, how cool?\n",
      "He likes to shop online and his favorite store is http://store.com and also likes to shop at www.store2.com.\n",
      "His favorite color is blue and he likes to buy items that cost $93.99, $19.99, $29.99 and also $39.99.\n",
      "He also likes to buy items that have a discount of $5.99 and $9.99. He has a dog named Buddy with a collar number #123456.\n",
      "His best friend's email is bestfriend@example.com and his phone number is 001\n"
     ]
    }
   ],
   "source": [
    "# create pattern object\n",
    "pattern = re.compile(r'(\\d{3})-(\\d{3})-(\\d{4})')\n",
    "\n",
    "# This can be easily done with back reference\n",
    "# A backreference in regular expressions allows you to reuse the value of a captured group in the same regular\n",
    "# expression pattern. It's represented by a backslash '\\' followed by the number of the group you want to refer to.\n",
    "hidden_collar_sting = pattern.sub(r'\\1-xxx-xxxx', string)\n",
    "print(hidden_collar_sting[:1060])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c5bbd0bb",
   "metadata": {},
   "source": [
    "###  Exercise 9"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "700b81c2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "The Cat in the hat, sat there with a fat rat. He ate a mat, and a bat came out of nowhere to attack.\n",
      "He had 2 cats, one was black and one was white. He also had 4 dogs, 2 of them were big and 2 of them were small.\n",
      "He wanted to buy a new Dog, but he couldn't decide between a Labrador or a Golden Retriever.\n",
      "In the end, he decided to adopt a stray Cat from tshe street. He painted his room in #ff0000 color.\n",
      "His email address is moew@gmail.com and his phone numbers 321-123-5555, 555.555.5555, 555555-5555, 555555.5555, 1234560000.\n",
      "He was born on 12/05/2018, and he graduated on 20/01/2023. He also has a website https://www.example.com, how cool?\n",
      "He likes to shop online and his favorite store is http://store.com and also likes to shop at www.store2.com.\n",
      "His favorite color is blue and he likes to buy items that cost $93.99, $19.99, $29.99 and also $39.99.\n",
      "He also likes to buy items that have a discount of $5.99 and $9.99. He has a dog named Buddy with a collar number #123456.\n",
      "His best friend's email is bestfriend@example.com and his phone number is 100\n"
     ]
    }
   ],
   "source": [
    "# using the same regex above\n",
    "swap_digits_string = pattern.sub(r'\\2-\\1-\\3', string)\n",
    "print(swap_digits_string[:1060])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3cf6f34",
   "metadata": {},
   "source": [
    "### Exercise 10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "id": "2b00d539",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['\\nThe Cat in the hat, sat there with a fat rat', 'He ate a mat, and a bat came out of nowhere to attack', 'He had 2 cats, one was black and one was white', 'He also had 4 dogs, 2 of them were big and 2 of them were small', \"He wanted to buy a new Dog, but he couldn't decide between a Labrador or a Golden Retriever\", 'In the end, he decided to adopt a stray Cat from tshe street', 'He painted his room in #ff0000 color', 'His email address is moew@gmail.com and his phone numbers 123-321-5555, 555.555.5555, 555555-5555, 555555.5555, 1234560000', 'He was born on 12/05/2018, and he graduated on 20/01/2023', 'He also has a website https://www.example.com, how cool', 'He likes to shop online and his favorite store is http://store.com and also likes to shop at www.store2.com', 'His favorite color is blue and he likes to buy items that cost $93.99, $19.99, $29.99 and also $39.99', 'He also likes to buy items that have a discount of $5.99 and $9.99', 'He has a dog named Buddy with a collar number #123456', \"His best friend's email is bestfriend@example.com and his phone number is 001-100-5556\", 'They usually meet every 15th of the month at 7:00 pm', \"My current IP address is 192.168.1.1 but I'm planning to switch to a new one, it's 10.0.0.1. \\nI just made a purchase using my credit card number 4111 1111 1111 1111 with expiration date of 12/24 and security code 123\", \"I can't wait to share my new book #datascience on twitter and Instagram\", \"it's ISBN number is 123-4-56789-0123-5 and it will be available on Amazon and Barnes & Noble\", 'My friend @johndoe lives at 123 Main St, Anytown USA 12345 and he told me that his neighbor lives at 124 Main St, Anytown USA 12345', \"I'm planning to visit my family in Los Angeles, CA on 12/24/2022 and stay there for 2 days\", \"I'll be staying at my cousin's house located at 1234 Elm St, Los Angeles CA 90210\", \"I'm also planning to visit my friend in New York, NY on 24-12-2022 and stay there for 5 days\", \"I'll be staying at my friend's apartment located at 123 Broadway, New York NY 10001\", \"I'm also planning to visit my sister in London, UK on 2022-12-24 and stay there for 6 days\", \"I'll be staying at my friend's flat located at 123 Oxford St, London UK W1D 2LJ\", \"I need to remember to bring my passport with expiration date of 12/24/2022, my driver's license with number\\n12345678 and my insurance card with number 123456789\", '']\n"
     ]
    }
   ],
   "source": [
    "# Keep in mind this approach is not a perfect solution as it may fail to correctly tokenize sentences \n",
    "# that use different forms of punctuation or those that lack punctuation entirely. Sentence tokenization can\n",
    "# be quite complex and as it involves using natural language processing techniques to understand the structure \n",
    "# and context of the text. Other more sophisticated methods such as using natural language processing \n",
    "# libraries like NLTK or SpaCy will provide more accurate results.\n",
    "\n",
    "# assuming that each sentece ends with a dot, question mark or an exclamation\n",
    "# use lookaround to prevent spliting numbers seperated by dots or emails/urls\n",
    "pattern = re.compile(r'(?<!www)[.?!](?!com)(?!\\d)[\\s](?!\\n)')\n",
    "sentences = pattern.split(string)\n",
    "print(sentences)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4b0a886a",
   "metadata": {},
   "source": [
    "### More Advanced Exercises"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4dad0580",
   "metadata": {},
   "source": [
    "1. Extract all the dates in the format DD/MM/YYYY from the text considering if they are valid dates\n",
    "2. Extract all the email addresses from the text considering if they are valid \n",
    "3. Extract all the phone numbers from the text in different formats (xxx-xxx-xxxx, xxx.xxx.xxxx, xxxxxxxxxx, xxx xxx xxxx)\n",
    "4. Extract all the IP addresses from the text considering if they are valid IP addresses\n",
    "5. Extract all the URLs from the text and check if they are valid URLs\n",
    "6. Extract all the credit card numbers from the text considering if they are valid credit card numbers\n",
    "7. Extract all the hashtags from the text (e.g. #example)\n",
    "8. Extract all the mentions from the text (e.g. @example)\n",
    "9. Extract all the dates and times in multiple formats (MM/DD/YYYY, DD-MM-YYYY, YYYY-MM-DD)\n",
    "10. Extract all the ISBN numbers from the text and check if they are valid ISBN numbers\n",
    "11. Extract all the street addresses from the text\n",
    "12. Extract all the zip codes from the text\n",
    "13. Extract all the words that contains both vowels and consonants from the text\n",
    "14. Extract all the words that contains at least 3 vowels from the text\n",
    "15. Extract all the words that are palindrome from the text\n",
    "16. Replace all the street addresses in the text with the string \"ADDRESS REDACTED\" while keeping the zip codes\n",
    "17. Replace all the credit card numbers in the text with the string \"CARD NUMBER REDACTED\" while keeping the expiration dates\n",
    "18. Split the text into a list of sentences, where each sentence starts with a capital letter and ends with a full stop,       exclamation mark or question mark.\n",
    "19. Split the text into a list of paragraphs, where each paragraph starts with a newline and two spaces.\n",
    "20. Split the text into a list of words, where each word starts with a capital letter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1ea8460a",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}