Objective2_b.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Cleaning: Normalize individual functions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Background\n",
    "\n",
    "There have been quite a few notebooks in this competition focusing on the task of data mining using various types of Python string manipulations, most notably regular expression or regex, for short. While these notebooks achieve highly satisfactory outcomes and create directions for future research or inquiries, they quickly encounter their limits due to the poor quality of the given dataset(s). Recall that we are given a collection of .txt files that describe past City of Los Angeles' jobs along with their pdf versions for the actual job postings. While the City has done an excellent jobs of composing these text files, the **source** that generated these .txt files is not revealed, which makes it very hard for participants to anticipate all possible inconsistencies in the data. For example, a diligent search shows that file, FIRE ASSISTANT CHIEF 2166 011218.txt, doesn't have JOB_DUTIES field, which is very questionable since the `kaggle_data_dictionary.csv` file explicitly spells that every job must have this field (see *Accepts Null Values?* column).\n",
    "\n",
    "By now, hopefully we can see that such inconsistencies will certainly fail any attempt to write code for data mining purposes **before** data cleaning. Even if we can agree to hand-wave this problem (e.g., violating the non-null requirement of JOB_DUTIES field), a person with a strong scientific mindset cannot just hand in the solution and walk away without at least noting where/what/why we have missing values. On the other hand, since it is well-known that exploratory data analysis (EDA) and data cleaning typically take up much of the time of a data scientist, it is understandable that very few kernels (if none at all) are dedicated to tackling data cleaning issue. For unstructured data like .txt files, this could mean one has to actively read the descriptions of several jobs before being able to figure out what should be done for data cleaning.\n",
    "\n",
    "<font size=4, color='red'>**Important Details.**</font> This series of notebooks is dedicated to data cleaning as I've realized that this dataset is very rich in information. Thus, a first complete and accurate [csv file](https://www.kaggle.com/c/data-science-for-good-city-of-los-angeles/overview), which advanced analyses can be built upon, will definitely assist the City of Los Angeles in restructuring their job posting to attract more talents in the upcoming years. The approach taken here is quite novel, at least comparing to other kernels. First, I focus on writing code to parse SYSTEMS ANALYST 1596 102717.txt into a csv file that is exactly the same as `sample job class export template.csv`. This is done by writing 25 main functions, each of which is dedicated to only extracting information for 25 field names, besides two helper functions. Then I use these functions to validate the consistency of other .txt files and manually change/modify the files to fit the pattern of SYSTEMS ANALYST 1596 102717.txt. For example, in the requirement section of ACCOUNTANT 1513 062218.txt, there is no itemizer, e.g., 1) or 1., so I'll manually add a 1. before the word, Graduation, to match with the pattern in SYSTEMS ANALYST 1596 102717.txt.\n",
    "\n",
    "Admittedly, the steps taken here are extremely labor-intensive. While I agree that there might be other easier ways, I do this for a purpose as I've learned that it is the best way to get myself familiar with this unstructured data. As mentioned above, since **the source that generated these text file is not known**, we really have no idea on how to retrieve relevant information from pattern matching on raw data. For example, one might attempt to do something such as `job[job.find('REQUIREMENTS/MINIMUM QUALIFICATIONS':job.find('PROCESS NOTES')]` or any similar expressions using regex. Although that this statement may help him retrieve **only** relevant information regarding school type, education majors, etc., this is not guranteed! For instance, I found that in some jobs, the word, PROCESS NOTES, came after the word WHERE TO APPLY, which unarguably causes severe headache later. The only way to avoid this, as far as I realize, is to patiently do some manual data cleaning before any analysis.\n",
    "\n",
    "Mention job bulletins_clean"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import relevant modules"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os                       # module to interface with the underlying OS\n",
    "import numpy as np              # linear algebra\n",
    "import pandas as pd             # dataframe\n",
    "import re                       # regular expression\n",
    "import matplotlib.pyplot as plt # data visualization\n",
    "%matplotlib inline\n",
    "import toolkits as tk           # user-define module for efficiently reading files"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get paths and names of files in each path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Path and list of jobs in Job Bulletins.\n",
    "# NOTE 1: These are raw data\n",
    "(raw_path, raw_jobs) = tk.get_raw_jobs() # tk is a user-define module\n",
    "\n",
    "# Path and list of jobs in JobBulletins_cleaned\n",
    "# NOTE 2: These are cleaned data\n",
    "(cleaned_path, cleaned_jobs) = tk.get_cleaned_jobs()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Normalize JOB_CLASS_TITLES (jct)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This is a helper function\n",
    "def job_class_title(job):\n",
    "    '''Returns the field JOB_CLASS_TITLE (jct)'''\n",
    "    # From the beginning to the word Class Code is where the information located\n",
    "    temp = job[:job.index('Class Code')]\n",
    "    # Split at newline characters and get the first element in the resulting list\n",
    "    jct = temp.split('\\n')[0]\n",
    "    # Remove redundant white spaces.\n",
    "    # Spliting at white space returns empty strings for white spaces,\n",
    "    # so we can take out redundant white spaces by querying on the length of element e\n",
    "    jct = [e for e in jct.split(' ') if len(e) > 0]\n",
    "    \n",
    "    # Returns\n",
    "    jct = ' '.join(jct) # join back words with white spaces\n",
    "    return jct"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "311 DIRECTOR\n",
      "ACCOUNTANT\n",
      "ACCOUNTING CLERK\n",
      "ACCOUNTING RECORDS SUPERVISOR\n",
      "ADMINISTRATIVE ANALYST\n",
      "ADMINISTRATIVE CLERK\n",
      "ADMINISTRATIVE HEARING EXAMINER\n",
      "ADVANCE PRACTICE PROVIDER CORRECTIONAL CARE\n",
      "AIR CONDITIONING MECHANIC\n",
      "AIR CONDITIONING MECHANIC SUPERVISOR\n",
      "AIRPORT AIDE\n",
      "\n",
      "AIRPORT ENGINEER\n",
      "AIRPORT GUIDE\n",
      "AIRPORT INFORMATION SPECIALIST\n",
      "AIRPORT LABOR RELATIONS ADVOCATE\n",
      "AIRPORT MANAGER\n",
      "AIRPORT POLICE CAPTAIN\n",
      "\n",
      "AIRPORT POLICE OFFICER\n",
      "AIRPORT POLICE SPECIALIST\n",
      "AIRPORT SUPERINTENDENT OF OPERATIONS\n",
      "AIRPORTS MAINTENANCE SUPERINTENDENT\n",
      "AIRPORTS MAINTENANCE SUPERVISOR\n",
      "AIRPORTS PUBLIC AND COMMUNITY RELATIONS DIRECTOR\n",
      "ANIMAL CARE ASSISTANT\n",
      "ANIMAL CARE TECHNICIAN\n",
      "WATER TREATMENT OPERATOR\n",
      "ANIMAL CONTROL OFFICER\n",
      "ANIMAL KEEPER\n",
      "APPARATUS OPERATOR\n",
      "APPLICATIONS PROGRAMMER\n",
      "APPRENTICE - METAL TRADES\n",
      "APPRENTICE MACHINIST\n",
      "AQUARIST\n",
      "AQUARIUM EDUCATOR\n",
      "AQUATIC DIRECTOR\n",
      "AQUATIC FACILITY MANAGER\n",
      "AQUEDUCT AND RESERVOIR KEEPER\n",
      "\n",
      "ARCHITECT\n",
      "CAMPUS INTERVIEWS ONLY\n",
      "ARCHITECTURAL DRAFTING TECHNICIAN\n",
      "ARCHIVIST\n",
      "ART CENTER DIRECTOR\n",
      "\n",
      "ART INSTRUCTOR\n",
      "ARTS ASSOCIATE\n",
      "ARTS MANAGER\n",
      "ASBESTOS SUPERVISOR\n",
      "ASBESTOS WORKER\n",
      "ASPHALT PLANT OPERATOR\n",
      "ASPHALT PLANT SUPERVISOR\n",
      "ASSISTANT AIRPORT MANAGER\n",
      "ASSISTANT COMMUNICATIONS CABLE WORKER\n",
      "ASSISTANT COMMUNICATIONS ELECTRICIAN\n",
      "ASSISTANT DEPUTY SUPERINTENDENT OF BUILDING\n",
      "ASSISTANT DIRECTOR INFORMATION SYSTEMS\n",
      "ASSISTANT GARDENER \t\t\t\t \t\n",
      "ASSISTANT INSPECTOR\n",
      "ASSISTANT RETIREMENT PLAN MANAGER\n",
      "ASSISTANT SIGNAL SYSTEMS ELECTRICIAN\n",
      "ASSISTANT STREET LIGHTING ELECTRICIAN\n",
      "ASSISTANT TREE SURGEON\n",
      "ASSISTANT UTILITY BUYER\n",
      "ASSOCIATE ZONING ADMINISTRATOR\n",
      "AUDIO VISUAL TECHNICIAN\n",
      "AUDITOR\n",
      "AUTO BODY BUILDER AND REPAIRER\n",
      "AUTO BODY REPAIR SUPERVISOR\n",
      "\n",
      "AUTO PAINTER\n",
      "AUTOMOTIVE DISPATCHER\n",
      "AUTOMOTIVE SUPERVISOR\n",
      "AVIONICS SPECIALIST\n",
      "BACKGROUND INVESTIGATION MANAGER\n",
      "BACKGROUND INVESTIGATOR\n",
      "BENEFITS SPECIALIST\n",
      "BLACKSMITH\n",
      "BOILERMAKER\n",
      "BOILERMAKER SUPERVISOR\n",
      "BUILDING CIVIL ENGINEER\n",
      "BUILDING CONSTRUCTION AND MAINTENANCE\n",
      "BUILDING ELECTRICAL ENGINEER\n",
      "BUILDING INSPECTOR\n",
      "BUILDING MAINTENANCE DISTRICT SUPERVISOR\n",
      "BUILDING MECHANICAL ENGINEER\n",
      "BUILDING MECHANICAL INSPECTOR\n",
      "BUILDING OPERATING ENGINEER\n",
      "BUILDING REPAIR SUPERVISOR\n",
      "BUILDING REPAIRER\n",
      "BUS OPERATOR\t\n",
      "BUS OPERATOR SUPERVISOR\n",
      "CABLE TELEVISION PRODUCTION MANAGER\n",
      "CARPENTER\n",
      "CARPENTER SUPERVISOR\n",
      "\n",
      "CEMENT FINISHER\n",
      "CEMENT FINISHER SUPERVISOR\n",
      "CEMENT FINISHER WORKER\n",
      "CHEMIST\n",
      "CHIEF ADMINISTRATIVE ANALYST\n",
      "\n",
      "CHIEF BENEFITS ANALYST\n",
      "CHIEF BUILDING OPERATING ENGINEER\n",
      "CHIEF CLERK\n",
      "CHIEF CLERK PERSONNEL\n",
      "CHIEF CLERK POLICE\n",
      "CHIEF CLERK POLICE\n",
      "CHIEF COMMUNICATIONS OPERATOR\n",
      "CHIEF CONSTRUCTION INSPECTOR\n",
      "CHIEF CUSTODIAN SUPERVISOR\n",
      "CHIEF ELECTRIC PLANT OPERATOR\n",
      "CHIEF ENVIRONMENTAL COMPLIANCE INSPECTOR\n",
      "CHIEF FINANCIAL OFFICER\n",
      "CHIEF FORENSIC CHEMIST\n",
      "CHIEF HARBOR ENGINEER\n",
      "CHIEF INSPECTOR\n",
      "\n",
      "CHIEF MANAGEMENT ANALYST\n",
      "CHIEF OF AIRPORT PLANNING\n",
      "CHIEF OF DRAFTING OPERATIONS\n",
      "CHIEF OF OPERATIONS\n",
      "CHIEF OF PARKING ENFORCEMENT OPERATIONS\n",
      "CHIEF PARK RANGER\t\t\t\t\t\n",
      "CHIEF PORT PILOT\n",
      "CHIEF SAFETY ENGINEER PRESSURE VESSELS\n",
      "CHIEF SECURITY OFFICER\n",
      "CHIEF STREET SERVICES INVESTIGATOR\n",
      "\n",
      "CHIEF TRANSPORTATION INVESTIGATOR\n",
      "CITY PLANNER\n",
      "CITY PLANNING ASSOCIATE\n",
      "CIVIL ENGINEER\n",
      "CIVIL ENGINEERING ASSOCIATE\n",
      "CIVIL ENGINEERING DRAFTING TECHNICIAN\n",
      "CLAIMS AGENT\n",
      "COMMERCIAL FIELD REPRESENTATIVE\n",
      "COMMERCIAL FIELD SUPERVISOR\n",
      "COMMERCIAL SERVICE SUPERVISOR\n",
      "COMMISSION EXECUTIVE ASSISTANT\n",
      "COMMUNICATIONS CABLE SUPERVISOR\n",
      "COMMUNICATIONS CABLE WORKER\n",
      "COMMUNICATIONS ELECTRICIAN\n",
      "COMMUNICATIONS ELECTRICIAN SUPERVISOR\n",
      "COMMUNICATIONS ENGINEER\n",
      "COMMUNICATIONS ENGINEERING ASSOCIATE\n",
      "COMMUNICATIONS INFORMATION REPRESENTATIVE\t\n",
      "COMMUNITY AFFAIRS ADVOCATE\n",
      "COMMUNITY HOUSING PROGRAMS MANAGER\n",
      "COMMUNITY PROGRAM ASSISTANT\n",
      "COMPLIANCE PROGRAM MANAGER\n",
      "CONSTRUCTION AND MAINTENANCE SUPERINTENDENT\n",
      "CONSTRUCTION AND MAINTENANCE SUPERVISOR\n",
      "CONSTRUCTION EQUIPMENT SERVICE WORKER\n",
      "CONSTRUCTION ESTIMATOR\n",
      "CONSTRUCTION INSPECTOR\n",
      "CONTRACT ADMINISTRATOR\n",
      "CONTROL SYSTEMS ENGINEERING ASSOCIATE\n",
      "CORRECTIONAL NURSE\t\n",
      "CRIME AND INTELLIGENCE ANALYST\n",
      "CRIMINALIST\n",
      "CUSTODIAL SERVICES ASSISTANT\n",
      "CUSTODIAN\n",
      "CUSTODIAN SUPERVISOR\n",
      "CUSTOMER SERVICE REPRESENTATIVE\n",
      "CUSTOMER SERVICE SPECIALIST\n",
      "DATA PROCESSING TECHNICIAN\n",
      "DATABASE ARCHITECT\n",
      "DECK HAND\n",
      "DELIVERY DRIVER\n",
      "DEPARTMENTAL CHIEF ACCOUNTANT\n",
      "DETENTION OFFICER\n",
      "DIRECTOR OF AIRPORT MARKETING\n",
      "DIRECTOR OF AIRPORT OPERATIONS\n",
      "DIRECTOR OF AIRPORTS ADMINISTRATION\tREVISED\n",
      "DIRECTOR OF COMMUNICATIONS SERVICES\n",
      "DIRECTOR OF ENFORCEMENT OPERATIONS\n",
      "DIRECTOR OF FIELD OPERATIONS\n",
      "\n",
      "DIRECTOR OF MAINTENANCE AIRPORTS\n",
      "DIRECTOR OF POLICE TRANSPORTATION\n",
      "DIRECTOR OF PORT CONSTRUCTION AND MAINTENANCE\n",
      "DIRECTOR OF PRINTING SERVICES\n",
      "DIRECTOR OF SECURITY SERVICES\n",
      "DIRECTOR OF SYSTEMS\n",
      "DISTRICT SUPERVISOR ANIMAL SERVICES\n",
      "DIVISION LIBRARIAN\n",
      "DRILL RIG OPERATOR\n",
      "DUPLICATING MACHINE OPERATOR\n",
      "ELECTRIC DISTRIBUTION MECHANIC\n",
      "ELECTRIC DISTRIBUTION MECHANIC SUPERVISOR\n",
      "\n",
      "ELECTRIC SERVICE REPRESENTATIVE\n",
      "ELECTRIC STATION OPERATOR\n",
      "ELECTRIC TROUBLE DISPATCHER\n",
      "ELECTRICAL CRAFT HELPER\n",
      "ELECTRICAL ENGINEERING ASSOCIATE\n",
      "ELECTRICAL ENGINEERING DRAFTING TECHNICIAN\n",
      "ELECTRICAL INSPECTOR\n",
      "ELECTRICAL MECHANIC\t\t\n",
      "ELECTRICAL MECHANIC SUPERVISOR\n",
      "ELECTRICAL REPAIR SUPERVISOR\n",
      "ELECTRICAL REPAIRER\n",
      "ELECTRICAL SERVICE WORKER\n",
      "ELECTRICAL SERVICES MANAGER\n",
      "ELECTRICAL TESTER \t\n",
      "ELECTRICIAN\n",
      "ELECTRICIAN SUPERVISOR\n",
      "ELEVATOR MECHANIC\n",
      "ELEVATOR MECHANIC HELPER\n",
      "\n",
      "EMERGENCY MANAGEMENT COORDINATOR\n",
      "EMERGENCY MEDICAL SERVICES (EMS) EDUCATOR\n",
      "EMS ADVANCED PROVIDER\n",
      "EMS NURSE PRACTITIONER SUPERVISOR\n",
      "ENGINEER OF FIRE DEPARTMENT\n",
      "ENGINEER OF SURVEYS\t\n",
      "ENGINEERING DESIGNER\n",
      "ENGINEERING GEOLOGIST\n",
      "ENGINEERING GEOLOGIST ASSOCIATE\n",
      "ENVIRONMENTAL AFFAIRS OFFICER\n",
      "ENVIRONMENTAL COMPLIANCE INSPECTOR\n",
      "ENVIRONMENTAL ENGINEER\n",
      "CAMPUS INTERVIEWS ONLY\n",
      "ENVIRONMENTAL SPECIALIST\n",
      "ENVIRONMENTAL SUPERVISOR\n",
      "EQUIPMENT MECHANIC (Automotive Mechanic)\n",
      "EQUIPMENT OPERATOR\n",
      "EQUIPMENT REPAIR SUPERVISOR\n",
      "EQUIPMENT SPECIALIST\n",
      "EQUIPMENT SUPERINTENDENT\n",
      "EQUIPMENT SUPERVISOR\n",
      "\n",
      "EXECUTIVE ADMINISTRATIVE ASSISTANT\n",
      "EXECUTIVE ASSISTANT AIRPORTS\n",
      "EXHIBIT PREPARATOR\n",
      "FIELD ENGINEERING AIDE\n",
      "FINANCIAL ANALYST\n",
      "FINANCIAL DEVELOPMENT OFFICER\n",
      "FINANCIAL MANAGER\n",
      "FINGERPRINT IDENTIFICATION EXPERT\t\n",
      "FIRE ASSISTANT CHIEF\n",
      "FIRE BATTALION CHIEF\n",
      "FIRE CAPTAIN\n",
      "FIRE HELICOPTER PILOT\n",
      "FIRE INSPECTOR\n",
      "FIRE PROTECTION ENGINEERING ASSOCIATE\n",
      "FIRE SPECIAL INVESTIGATOR\n",
      "FIRE SPRINKLER INSPECTOR\n",
      "\n",
      "FIREBOAT MATE\n",
      "FIREBOAT PILOT\n",
      "FIREFIGHTER\n",
      "FISCAL SYSTEMS SPECIALIST\n",
      "FLEET SERVICES MANAGER\n",
      "FORENSIC PRINT SPECIALIST\n",
      "GALLERY ATTENDANT\n",
      "GARAGE ASSISTANT\n",
      "\n",
      "GARDENER CARETAKER\n",
      "GENERAL AUTOMOTIVE SUPERVISOR\n",
      "GENERAL SERVICES MANAGER\n",
      "GEOGRAPHIC INFORMATION SYSTEMS CHIEF\n",
      "GEOGRAPHIC INFORMATION SYSTEMS SPECIALIST\n",
      "GEOGRAPHIC INFORMATION SYSTEMS SUPERVISOR\n",
      "GEOTECHNICAL ENGINEER\n",
      "GOLF STARTER\n",
      "GOLF STARTER SUPERVISOR\n",
      "GRAPHICS DESIGNER\n",
      "GRAPHICS SUPERVISOR\n",
      "HARBOR ENGINEER\n",
      "HARBOR PLANNING AND ECONOMIC ANALYST\n",
      "HARBOR PLANNING AND RESEARCH DIRECTOR\n",
      "HEAD CUSTODIAN SUPERVISOR\n",
      "HEATING AND REFRIGERATION INSPECTOR\n",
      "HEAVY DUTY EQUIPMENT MECHANIC\n",
      "HEAVY DUTY TRUCK OPERATOR\n",
      "HELICOPTER MECHANIC\n",
      "HELICOPTER MECHANIC SUPERVISOR\n",
      "HOUSING INSPECTOR\n",
      "HOUSING INVESTIGATOR\n",
      "HOUSING PLANNING AND ECONOMIC ANALYST\n",
      "HUMAN RELATIONS ADVOCATE\n",
      "HYDROGRAPHER\n",
      "\n",
      "INDUSTRIAL AND COMMERCIAL FINANCE OFFICER\n",
      "INDUSTRIAL CHEMIST\n",
      "INDUSTRIAL GRAPHICS SUPERVISOR\n",
      "INDUSTRIAL HYGIENIST\n",
      "INFORMATION SERVICES SPECIALIST\n",
      "INFORMATION SYSTEMS MANAGER\n",
      "INSTRUMENT MECHANIC\n",
      "INSTRUMENT MECHANIC SUPERVISOR\n",
      "INTERNAL AUDITOR\n",
      "INVESTMENT OFFICER\n",
      "IRRIGATION SPECIALIST\n",
      "LABOR SUPERVISOR\n",
      "LABORATORY TECHNICIAN\n",
      "LAND SURVEYING ASSISTANT\n",
      "LANDSCAPE ARCHITECT\n",
      "LANDSCAPE ARCHITECTURAL ASSOCIATE\n",
      "LEGISLATIVE ASSISTANT\n",
      "LEGISLATIVE REPRESENTATIVE\n",
      "LIBRARIAN\n",
      "LIBRARY ASSISTANT\n",
      "LICENSED VOCATIONAL NURSE\n",
      "LINE MAINTENANCE ASSISTANT\n",
      "LOCKSMITH\n",
      "MACHINIST\n",
      "MACHINIST SUPERVISOR\n",
      "MAINTENANCE AND CONSTRUCTION HELPER\n",
      "MAINTENANCE LABORER\n",
      "MANAGEMENT AIDE\n",
      "MANAGEMENT ANALYST\n",
      "MANAGEMENT ASSISTANT\t\n",
      "MANAGING WATER UTILITY ENGINEER\n",
      "MARINE AQUARIUM CURATOR\n",
      "MARINE AQUARIUM PROGRAM DIRECTOR\n",
      "MARINE ENVIRONMENTAL MANAGER\n",
      "\n",
      "MASONRY WORKER\n",
      "MATERIALS TESTING ENGINEERING ASSOCIATE\n",
      "MATERIALS TESTING TECHNICIAN\n",
      "MECHANICAL ENGINEER\n",
      "MECHANICAL ENGINEERING ASSOCIATE\n",
      "MECHANICAL ENGINEERING DRAFTING TECHNICIAN\n",
      "MECHANICAL HELPER\n",
      "MECHANICAL REPAIR GENERAL SUPERVISOR\n",
      "MECHANICAL REPAIR SUPERVISOR\n",
      "MECHANICAL REPAIRER\n",
      "MEDICAL ASSISTANT\n",
      "METER READER\n",
      "MOTION PICTURE AND TELEVISION MANAGER\n",
      "MOTOR SWEEPER OPERATOR\n",
      "OCCUPATIONAL HEALTH NURSE\n",
      "OFFICE ENGINEERING TECHNICIAN\n",
      "OFFICE SERVICES ASSISTANT\n",
      "OFFICE TRAINEE\n",
      "OPERATIONS AND STATISTICAL RESEARCH ANALYST\n",
      "PAINTER\n",
      "PAINTER SUPERVISOR\n",
      "PARK MAINTENANCE SUPERVISOR\n",
      "PARK RANGER\t\n",
      "PARK SERVICES ATTENDANT\n",
      "PARK SERVICES SUPERVISOR\n",
      "PARKING ATTENDANT\n",
      "\n",
      "PARKING MANAGER\n",
      "PARKING METER TECHNICIAN\n",
      "PARKING METER TECHNICIAN SUPERVISOR\n",
      "PAYROLL ANALYST\n",
      "PAYROLL SUPERVISOR\n",
      "PERFORMING ARTS DIRECTOR\n",
      "PERSONNEL ANALYST\n",
      "PERSONNEL DIRECTOR\n",
      "PERSONNEL RECORDS SUPERVISOR\n",
      "PERSONNEL RESEARCH ANALYST\n",
      "PHOTOGRAPHER\n",
      "PILE DRIVER WORKER\n",
      "PIPEFITTER\t\n",
      "\n",
      "PLANNING ASSISTANT\n",
      "PLUMBER\n",
      "PLUMBER SUPERVISOR\n",
      "PLUMBING INSPECTOR\n",
      "POLICE ADMINISTRATOR\n",
      "POLICE CAPTAIN\n",
      "POLICE COMMANDER\t\n",
      "POLICE DETECTIVE\n",
      "POLICE LIEUTENANT\n",
      "POLICE OFFICER\n",
      "POLICE PERFORMANCE AUDITOR\n",
      "\n",
      "POLICE SERVICE REPRESENTATIVE\n",
      "POLICE SPECIAL INVESTIGATOR\n",
      "POLICE SPECIALIST\n",
      "\n",
      "POLYGRAPH EXAMINER\n",
      "\n",
      "PORT ELECTRICAL MECHANIC SUPERVISOR\n",
      "PORT MAINTENANCE SUPERVISOR\n",
      "PORT PILOT\t\t\n",
      "PORT POLICE CAPTAIN\n",
      "PORT POLICE LIEUTENANT\n",
      "PORT POLICE OFFICER\n",
      "PORT POLICE SERGEANT\n",
      "PORTFOLIO MANAGER\n",
      "POWER ENGINEERING MANAGER\n",
      "POWER SHOVEL OPERATOR\n",
      "PRE-PRESS OPERATOR \t\n",
      "PRINCIPAL ACCOUNTANT\n",
      "PRINCIPAL ANIMAL KEEPER\n",
      "PRINCIPAL CITY PLANNER\n",
      "\n",
      "PRINCIPAL CIVIL ENGINEERING DRAFTING TECHNICIAN\n",
      "PRINCIPAL CLERK\n",
      "PRINCIPAL CLERK POLICE\n",
      "PRINCIPAL CLERK UTILITY\n",
      "PRINCIPAL COMMUNICATIONS OPERATOR\n",
      "PRINCIPAL CONSTRUCTION INSPECTOR\n",
      "\n",
      "PRINCIPAL DETENTION OFFICER\n",
      "PRINCIPAL ELECTRIC TROUBLE DISPATCHER\n",
      "PRINCIPAL ELECTRICAL ENGINEERING DRAFTING TECHNICIAN\n",
      "PRINCIPAL ENVIRONMENTAL ENGINEER\n",
      "PRINCIPAL GROUNDS MAINTENANCE SUPERVISOR\n",
      "PRINCIPAL INSPECTOR\n",
      "PRINCIPAL LIBRARIAN\n",
      "PRINCIPAL MECHANICAL ENGINEERING DRAFTING TECHNICIAN\n",
      "PRINCIPAL PHOTOGRAPHER\n",
      "PRINCIPAL PROPERTY OFFICER\n",
      "PRINCIPAL PUBLIC RELATIONS REPRESENTATIVE\n",
      "PRINCIPAL RECREATION SUPERVISOR\n",
      "PRINCIPAL SECURITY OFFICER\n",
      "PRINCIPAL STOREKEEPER\n",
      "PRINCIPAL TAX AUDITOR\n",
      "PRINCIPAL TAX COMPLIANCE OFFICER\t\t\n",
      "PRINCIPAL TRANSPORTATION ENGINEER\n",
      "PRINCIPAL UTILITY ACCOUNTANT\n",
      "PRINCIPAL WORKERS' COMPENSATION ANALYST\n",
      "PRINTING PRESS OPERATOR\n",
      "PRINTING SERVICES SUPERINTENDENT\n",
      "PROCUREMENT ANALYST\n",
      "PROCUREMENT SUPERVISOR\n",
      "PROGRAMMER ANALYST\n",
      "PROPERTY MANAGER\n",
      "PROPERTY OFFICER\n",
      "PROTECTIVE COATING WORKER\n",
      "PUBLIC INFORMATION DIRECTOR\n",
      "PUBLIC RELATIONS SPECIALIST\n",
      "PUBLIC SAFETY RISK MANAGER\n",
      "RATES MANAGER\t\n",
      "REAL ESTATE ASSOCIATE\n",
      "REAL ESTATE OFFICER\n",
      "REAL ESTATE TRAINEE\n",
      "RECREATION COORDINATOR\n",
      "RECREATION FACILITY DIRECTOR\n",
      "RECREATION SUPERVISOR\n",
      "REFUSE COLLECTION SUPERVISOR\t\n",
      "REFUSE COLLECTION TRUCK OPERATOR\n",
      "REFUSE CREW FIELD INSTRUCTOR\n",
      "REHABILITATION CONSTRUCTION SPECIALIST\n",
      "\n",
      "REINFORCING STEEL WORKER\n",
      "REPROGRAPHICS OPERATOR\n",
      "REPROGRAPHICS SUPERVISOR\n",
      "\n",
      "RIDESHARE PROGRAM ADMINISTRATOR\n",
      "RISK AND INSURANCE ASSISTANT\n",
      "RISK MANAGEMENT AND PREVENTION PROGRAM SPECIALIST\n",
      "RISK MANAGER\n",
      "ROOFER\n",
      "ROOFER SUPERVISOR\n",
      "SAFETY ADMINISTRATOR\n",
      "SAFETY ENGINEER\n",
      "SAFETY ENGINEER ELEVATORS\n",
      "SAFETY ENGINEER PRESSURE VESSELS\n",
      "SAFETY ENGINEERING ASSOCIATE\n",
      "SANITATION SOLID RESOURCES MANAGER\n",
      "SANITATION WASTEWATER MANAGER\n",
      "SECRETARY\n",
      "SECRETARY LEGAL\n",
      "SECURITY AIDE\n",
      "SECURITY OFFICER\n",
      "SENIOR ACCOUNTANT\n",
      "SENIOR ADMINISTRATIVE ANALYST\n",
      "SENIOR ADMINISTRATIVE CLERK\n",
      "SENIOR ANIMAL CONTROL OFFICER\n",
      "SENIOR ANIMAL KEEPER\n",
      "SENIOR ARCHITECT\n",
      "SENIOR ARCHITECTURAL DRAFTING TECHNICIAN\n",
      "SENIOR AUDITOR\n",
      "SENIOR AUTOMOTIVE SUPERVISOR\n",
      "SENIOR BUILDING INSPECTOR\n",
      "SENIOR BUILDING MECHANICAL INSPECTOR\n",
      "\n",
      "SENIOR CARPENTER\n",
      "SENIOR CHEMIST\n",
      "SENIOR CITY PLANNER\n",
      "SENIOR CIVIL ENGINEER\n",
      "SENIOR CIVIL ENGINEERING DRAFTING TECHNICIAN\n",
      "SENIOR CLAIMS REPRESENTATIVE\n",
      "SENIOR COMMERCIAL FIELD REPRESENTATIVE\n",
      "SENIOR COMMUNICATIONS CABLE WORKER\n",
      "SENIOR COMMUNICATIONS ELECTRICIAN\n",
      "SENIOR COMMUNICATIONS ELECTRICIAN SUPERVISOR\n",
      "SENIOR COMMUNICATIONS ENGINEER\n",
      "SENIOR COMMUNICATIONS OPERATOR\n",
      "SENIOR COMPUTER OPERATOR\n",
      "SENIOR CONSTRUCTION ENGINEER\n",
      "SENIOR CONSTRUCTION ESTIMATOR\n",
      "SENIOR CONSTRUCTION INSPECTOR\n",
      "SENIOR CUSTODIAN\n",
      "SENIOR DATA PROCESSING TECHNICIAN\n",
      "SENIOR DETENTION OFFICER\n",
      "SENIOR ELECTRIC SERVICE REPRESENTATIVE\n",
      "SENIOR ELECTRIC TROUBLE DISPATCHER\n",
      "SENIOR ELECTRICAL ENGINEERING DRAFTING TECHNICIAN\n",
      "SENIOR ELECTRICAL INSPECTOR\n",
      "SENIOR ELECTRICAL MECHANIC\n",
      "SENIOR ELECTRICAL MECHANIC SUPERVISOR\n",
      "SENIOR ELECTRICAL REPAIR SUPERVISOR\n",
      "SENIOR ELECTRICAL TEST TECHNICIAN\n",
      "SENIOR ELECTRICIAN\n",
      "SENIOR ENVIRONMENTAL COMPLIANCE INSPECTOR\n",
      "SENIOR ENVIRONMENTAL ENGINEER\n",
      "SENIOR EQUIPMENT MECHANIC\n",
      "WATER UTILITY SUPERINTENDENT\n",
      "SENIOR FIRE PROTECTION ENGINEER\n",
      "SENIOR FORENSIC PRINT SPECIALIST\n",
      "SENIOR GARDENER\n",
      "SENIOR HEATING AND REFRIGERATION INSPECTOR\n",
      "SENIOR HEAVY DUTY EQUIPMENT MECHANIC\n",
      "SENIOR HOUSING INSPECTOR\n",
      "SENIOR HOUSING INVESTIGATOR\n",
      "SENIOR HYDROGRAPHER\n",
      "SENIOR INDUSTRIAL HYGIENIST\n",
      "SENIOR LABOR RELATIONS SPECIALIST\n",
      "SENIOR LEGISLATIVE ASSISTANT\n",
      "SENIOR LIBRARIAN\n",
      "SENIOR LOAD DISPATCHER\n",
      "SENIOR MACHINIST SUPERVISOR\n",
      "SENIOR MANAGEMENT ANALYST\n",
      "SENIOR MECHANICAL ENGINEERING DRAFTING TECHNICIAN\n",
      "SENIOR MECHANICAL REPAIRER\n",
      "SENIOR PAINTER\n",
      "SENIOR PARK MAINTENANCE SUPERVISOR\n",
      "SENIOR PARK RANGER\n",
      "SENIOR PARK SERVICES ATTENDANT\n",
      "SENIOR PARKING ATTENDANT\n",
      "SENIOR PERSONNEL ANALYST\n",
      "SENIOR PHOTOGRAPHER\t\n",
      "SENIOR PLUMBER\n",
      "SENIOR PLUMBING INSPECTOR\n",
      "SENIOR POLICE SERVICE REPRESENTATIVE\n",
      "SENIOR PORT ELECTRICAL MECHANIC\n",
      "SENIOR PROPERTY OFFICER\n",
      "SENIOR REAL ESTATE OFFICER\n",
      "SENIOR RECREATION DIRECTOR\t\n",
      "SENIOR ROOFER\n",
      "SENIOR SAFETY ENGINEER ELEVATORS\n",
      "SENIOR SAFETY ENGINEER PRESSURE VESSELS\n",
      "SENIOR SECURITY OFFICER\n",
      "SENIOR STOREKEEPER\n",
      "SENIOR STREET LIGHTING ENGINEER\n",
      "SENIOR STRUCTURAL ENGINEER\n",
      "SENIOR SURVEY SUPERVISOR\n",
      "SENIOR SYSTEMS ANALYST\n",
      "SENIOR TAX AUDITOR\n",
      "SENIOR TITLE EXAMINER\n",
      "SENIOR TRAFFIC SUPERVISOR\n",
      "SENIOR TRANSPORTATION ENGINEER\n",
      "SENIOR TRANSPORTATION INVESTIGATOR\n",
      "SENIOR UNDERGROUND DISTRIBUTION CONSTRUCTION SUPERVISOR\n",
      "SENIOR UTILITY ACCOUNTANT\n",
      "SENIOR UTILITY BUYER\n",
      "SENIOR UTILITY SERVICES SPECIALIST\n",
      "\n",
      "\n",
      "SENIOR WINDOW CLEANER\n",
      "SENIOR WORKERS' COMPENSATION ANALYST\n",
      "SHEET METAL SUPERVISOR\n",
      "SHEET METAL WORKER\n",
      "SHIFT SUPERINTENDENT WASTEWATER TREATMENT\t\n",
      "SHIP CARPENTER\n",
      "SHOPS SUPERINTENDENT\n",
      "\n",
      "\n",
      "SIGNAL SYSTEMS ELECTRICIAN\t\t\n",
      "SIGNAL SYSTEMS SUPERINTENDENT\n",
      "SIGNAL SYSTEMS SUPERVISOR\n",
      "SOCIAL WORKER\n",
      "SOLID RESOURCES SUPERINTENDENT\n",
      "SOLID WASTE DISPOSAL SUPERINTENDENT\n",
      "SPECIAL INVESTIGATOR\n",
      "\n",
      "STAFF ASSISTANT TO GENERAL MANAGER WATER AND POWER\n",
      "STEAM PLANT ASSISTANT\n",
      "STEAM PLANT MAINTENANCE MECHANIC\n",
      "STEAM PLANT MAINTENANCE SUPERVISOR\n",
      "STEAM PLANT OPERATOR\n",
      "STOREKEEPER\n",
      "STORES SUPERVISOR\n",
      "STREET LIGHTING CONSTRUCTION AND MAINTENANCE SUPERINTENDENT\n",
      "STREET LIGHTING ELECTRICIAN\n",
      "STREET LIGHTING ELECTRICIAN SUPERVISOR\n",
      "\n",
      "CAMPUS INTERVIEWS ONLY\n",
      "STREET SERVICES GENERAL SUPERINTENDENT\n",
      "STREET SERVICES INVESTIGATOR\n",
      "STREET SERVICES SUPERINTENDENT\n",
      "STREET SERVICES SUPERVISOR\n",
      "STREET SERVICES WORKER\n",
      "STREET TREE SUPERINTENDENT\n",
      "STRUCTURAL ENGINEER\n",
      "STRUCTURAL ENGINEERING ASSOCIATE\n",
      "STRUCTURAL STEEL FABRICATOR\n",
      "STRUCTURAL STEEL FABRICATOR SUPERVISOR\n",
      "SUPERINTENDENT OF RECREATION AND PARKS OPERATIONS\n",
      "SUPERVISING CRIMINALIST\n",
      "SUPERVISING OCCUPATIONAL HEALTH NURSE\n",
      "\n",
      "SUPERVISING WATER SERVICE REPRESENTATIVE\n",
      "SUPPLY SERVICES MANAGER\n",
      "\n",
      "SURVEY PARTY CHIEF\n",
      "SURVEY SUPERVISOR\n",
      "SYSTEMS AIDE\n",
      "SYSTEMS ANALYST\n",
      "\n",
      "TAX AUDITOR\n",
      "TAX COMPLIANCE AIDE\t\n",
      "TAX COMPLIANCE OFFICER\n",
      "TELECOMMUNICATIONS PLANNING AND UTILIZATION OFFICER\n",
      "\n",
      "TIRE REPAIRER\n",
      "TITLE EXAMINER\n",
      "\n",
      "TRAFFIC OFFICER\n",
      "TRAFFIC PAINTER AND SIGN POSTER\n",
      "TRANSMISSION AND DISTRIBUTION DISTRICT SUPERVISOR\n",
      "TRANSPORTATION ENGINEER\n",
      "TRANSPORTATION ENGINEERING AIDE\n",
      "TRANSPORTATION ENGINEERING ASSOCIATE\n",
      "TRANSPORTATION INVESTIGATOR\n",
      "TRANSPORTATION PLANNING ASSOCIATE\n",
      "TREASURY ACCOUNTANT\n",
      "TREE SURGEON\n",
      "TREE SURGEON ASSISTANT\n",
      "TREE SURGEON SUPERVISOR\n",
      "TRUCK AND EQUIPMENT DISPATCHER\n",
      "TRUCK OPERATOR\n",
      "UNDERGROUND DISTRIBUTION CONSTRUCTION MECHANIC\n",
      "UNDERGROUND DISTRIBUTION CONSTRUCTION SUPERVISOR\n",
      "UPHOLSTERER\n",
      "UTILITIES SERVICE INVESTIGATOR\n",
      "UTILITY ACCOUNTANT\n",
      "UTILITY ADMINISTRATOR\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "UTILITY BUYER\n",
      "UTILITY EXECUTIVE SECRETARY\n",
      "UTILITY SERVICES MANAGER\n",
      "UTILITY SERVICES SPECIALIST\n",
      "VETERINARY TECHNICIAN \t\t\t\t \t\n",
      "VIDEO PRODUCTION COORDINATOR\n",
      "VIDEO TECHNICIAN\n",
      "VOLUNTEER COORDINATOR\n",
      "WAREHOUSE AND TOOLROOM WORKER\n",
      "WASTEWATER TREATMENT OPERATOR\n",
      "WASTEWATER COLLECTION WORKER\n",
      "WASTEWATER TREATMENT ELECTRICIAN\n",
      "WASTEWATER TREATMENT ELECTRICIAN SUPERVISOR\n",
      "WASTEWATER TREATMENT LABORATORY MANAGER\n",
      "WASTEWATER TREATMENT MECHANIC\n",
      "WASTEWATER TREATMENT MECHANIC SUPERVISOR\n",
      "WASTEWATER TREATMENT OPERATOR\n",
      "WATER BIOLOGIST\n",
      "\n",
      "WATER SERVICE REPRESENTATIVE\n",
      "WATER SERVICE SUPERVISOR\n",
      "WATER SERVICE WORKER\n",
      "WATER SERVICES MANAGER\n",
      "WATER TREATMENT OPERATOR\n",
      "WATER TREATMENT SUPERVISOR\n",
      "WATER UTILITY OPERATOR\n",
      "WATER UTILITY OPERATOR SUPERVISOR\n",
      "WATER UTILITY SUPERINTENDENT\n",
      "WATER UTILITY SUPERVISOR\n",
      "WATER UTILITY WORKER\n",
      "WATERSHED RESOURCES SPECIALIST\n",
      "WATERWORKS ENGINEER\n",
      "WATERWORKS MECHANIC SUPERVISOR\n",
      "WELDER\n",
      "WELDER SUPERVISOR\n",
      "WHARFINGER\n",
      "WINDOW CLEANER\n",
      "WORKERS' COMPENSATION ANALYST\n",
      "WORKERS' COMPENSATION CLAIMS ASSISTANT\n",
      "X-Ray and Laboratory Technician\n",
      "\n",
      "ZOO CURATOR OF EDUCATION\n",
      "ZOO REGISTRAR\n"
     ]
    }
   ],
   "source": [
    "# Normalization Strategy: \n",
    "# print jct in the try clause and print job_path in the except clause\n",
    "# look at printouts and detect unusual jct's.\n",
    "for file_name in raw_jobs:\n",
    "    job_path = raw_path + file_name        # define path to file_name\n",
    "    raw_job  = open(job_path, 'rt').read() # read in job as a string\n",
    "    try:\n",
    "        print(job_class_title(job=raw_job)) \n",
    "    except:                                # do some pretty printings here to help our eyes from pain\n",
    "        ## define some useful variables\n",
    "        border_line = '##############################################################################################'\n",
    "        how_many    = int((len(border_line) - len(job_path))/2)\n",
    "        print(border_line)\n",
    "        ## do pretty printings\n",
    "        print('#'*how_many + job_path + '#'*how_many)\n",
    "        print(border_line)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A few things to notice here:\n",
    "1. The function `job_class_title` was able to read all raw jobs (no errors were catched).\n",
    "2. There are some suspicious blank lines, e.g. AIRPORT AIDE blank-line AIRPORT ENGINEER. \n",
    "    * By actively looking at the Job Bulletins folder, we see that the function returns an empty string for some jobs, e.g., AIRPORT CHIEF INFORMATION SECURITY OFFICER 1404 120415_Modified.txt. Of course, this is due to the bug introduced in our code. This job doesn't fall into the pattern specified in SYSTEMS ANALYST 1596 102717.txt, because they start with newline characters. \n",
    "    * One may come back to the function `job_class_title` and modify it by using regex, for example, to capture all titles. However, we will **not** do that since the approach we are taking here is **data normalization**, that is, we make sure every job follows the same pattern as SYSTEMS ANALYST 1596 102717.txt by modifying its content.\n",
    "    * This may sound unexciting; however, the real benefit of this approach won't come in until later when the need to parse information regarding jobs' requirements arises. Once getting there, we'll see that the data normalization approach outperforms most of the traditional ones, which focus on writing functions that fit all jobs. On top of that, by patiently modifying these jobs manually, we've already familiarized ourselves with this type of unstructured data as well as raised our awarenesses when missing values occur.\n",
    "    * Here are the jobs that were modified by removing initial newline characters:\n",
    "        * CityofLA/Job Bulletins/AIRPORT CHIEF INFORMATION SECURITY OFFICER 1404 120415_Modified.txt\n",
    "        * CityofLA/Job Bulletins/AIRPORT POLICE LIEUTENANT 3227 091616.txt\n",
    "        * CityofLA/Job Bulletins/AQUEDUCT AND RESERVOIR SUPERVISOR 5816 091115.txt\n",
    "        * CityofLA/Job Bulletins/ART CURATOR 2448 071516 REV 072816.txt\n",
    "        * CityofLA/Job Bulletins/AUTO ELECTRICIAN 3707 052215.txt\n",
    "        * CityofLA/Job Bulletins/CARPET LAYER 3418 061915.txt\n",
    "        * CityofLA/Job Bulletins/CHIEF AIRPORTS ENGINEER 7274 051515 (1).txt\n",
    "        * CityofLA/Job Bulletins/CHIEF INTERNAL AUDITOR 1619 090916 (5).txt\n",
    "        * CityofLA/Job Bulletins/CHIEF TAX COMPLIANCE OFFICER 1211 041814.txt\n",
    "        * CityofLA/Job Bulletins/DIRECTOR OF HOUSING 1568 062317.txt\n",
    "        * CityofLA/Job Bulletins/ELECTRIC METER SETTER 3822 012017.txt\n",
    "        * CityofLA/Job Bulletins/ELEVATOR REPAIR SUPERVISOR 032516 REVISED 040516.txt\n",
    "        * CityofLA/Job Bulletins/EXAMINER OF QUESTIONED DOCUMENTS 3229 120415.txt\n",
    "        * CityofLA/Job Bulletins/FIREARMS EXAMINER 2233 062416.txt\n",
    "        * CityofLA/Job Bulletins/GARAGE ATTENDANT 3531 013015.txt\n",
    "        * CityofLA/Job Bulletins/IMPROVEMENT ASSESSOR SUPERVISOR 1564 100215.txt\n",
    "        * CityofLA/Job Bulletins/MARINE ENVIRONMENTAL SUPERVISOR 9433 071114 (1).txt\n",
    "        * CityofLA/Job Bulletins/PARKING ENFORCEMENT MANAGER 9025 021916 rev022516.txt\n",
    "        * CityofLA/Job Bulletins/PIPEFITTER SUPERVISOR 3438 081216.txt\n",
    "        * CityofLA/Job Bulletins/POLICE SERGEANT 2227 102116.txt\n",
    "        * CityofLA/Job Bulletins/POLICE SURVEILLANCE SPECIALIST 3687 052215.txt\n",
    "        * CityofLA/Job Bulletins/PORT ELECTRICAL MECHANIC 3758 022616.txt\n",
    "        * CityofLA/Job Bulletins/PRINCIPAL CIVIL ENGINEER 9489 022318.txt\n",
    "        * CityofLA/Job Bulletins/PRINCIPAL DEPUTY CONTROLLER 7260 032814.txt\n",
    "        * CityofLA/Job Bulletins/REHABILITATION PROJECT COORDINATOR 8502 032715.txt\n",
    "        * CityofLA/Job Bulletins/RETIREMENT PLAN MANAGER 9149 052314 (1).txt\n",
    "        * CityofLA/Job Bulletins/SENIOR BUILDING OPERATING ENGINEER 5925 011615 (1).txt\n",
    "        * CityofLA/Job Bulletins/SENIOR UTILITY SERVICES SPECIALIST 3753 121815 (1).txt\n",
    "        * CityofLA/Job Bulletins/SENIOR WASTEWATER TREATMENT OPERATOR 4124 041417.txt\n",
    "        * CityofLA/Job Bulletins/SIGN PAINTER 3428 121214.txt\n",
    "        * CityofLA/Job Bulletins/SIGN SHOP SUPERVISOR 3419 030615.txt\n",
    "        * CityofLA/Job Bulletins/SR CRIME _ INTELLIGENCE ANALYST 2241 011516.txt\n",
    "        * CityofLA/Job Bulletins/STREET LIGHTING ENGINEER 7537 052617 (4).txt\n",
    "        * CityofLA/Job Bulletins/SUPERVISING TRANSPORTATION PLANNER 2481 072216.txt\n",
    "        * CityofLA/Job Bulletins/SUPPLY SERVICES PAYMENT CLERK 1214 031017-.txt\n",
    "        * CityofLA/Job Bulletins/SYSTEMS PROGRAMMER 1455 091616 REV 100416.txt\n",
    "        * CityofLA/Job Bulletins/TILE SETTER 3493 090415.txt\n",
    "        * CityofLA/Job Bulletins/TRAFFIC MARKING AND SIGN SUPERINTENDENT 3430 032219.txt\n",
    "        * CityofLA/Job Bulletins/WATER MICROBIOLOGIST  7857 072514 rev073114.txt\n",
    "        * CityofLA/Job Bulletins/ZOO CURATOR 4297 040816.txt\n",
    "3. Some jobs have suspiciously weird titles, such as CAMPUS INTERVIEW ONLY. \n",
    "    * The following list names the jobs that were modified. The phrase CAMPUS INTERVIEW ONLY that used to appear before the job title was moved to the section `NOTES:` and put inside forward and backward Python prompt symbol, i.e. >>>CAMPUS INTERVIEW ONLY<<<\n",
    "        * CityofLA/Job Bulletins/ARCHITECTURAL ASSOCIATE 7926 013114 REV 032916.txt\n",
    "        * CityofLA/Job Bulletins/ENVIRONMENTAL ENGINEERING ASSOCIATE  7871 020113 REV 032916.txt\n",
    "        * CityofLA/Job Bulletins/STREET LIGHTING ENGINEERING ASSOCIATE 7527 101102 REV 032916.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Let's rerun the function and observe the changes. Note we use .txt files in the JobBulletins_cleaned folder.**\n",
    "\n",
    "Observing the printouts carefully this time, we see that all of the nuances above have been resolved."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "311 DIRECTOR\n",
      "ACCOUNTANT\n",
      "ACCOUNTING CLERK\n",
      "ACCOUNTING RECORDS SUPERVISOR\n",
      "ADMINISTRATIVE ANALYST\n",
      "ADMINISTRATIVE CLERK\n",
      "ADMINISTRATIVE HEARING EXAMINER\n",
      "ADVANCE PRACTICE PROVIDER CORRECTIONAL CARE\n",
      "AIR CONDITIONING MECHANIC\n",
      "AIR CONDITIONING MECHANIC SUPERVISOR\n",
      "AIRPORT AIDE\n",
      "AIRPORT CHIEF INFORMATION SECURITY OFFICER\n",
      "AIRPORT ENGINEER\n",
      "AIRPORT GUIDE\n",
      "AIRPORT INFORMATION SPECIALIST\n",
      "AIRPORT LABOR RELATIONS ADVOCATE\n",
      "AIRPORT MANAGER\n",
      "AIRPORT POLICE CAPTAIN\n",
      "AIRPORT POLICE LIEUTENANT\n",
      "AIRPORT POLICE OFFICER\n",
      "AIRPORT POLICE SPECIALIST\n",
      "AIRPORT SUPERINTENDENT OF OPERATIONS\n",
      "AIRPORTS MAINTENANCE SUPERINTENDENT\n",
      "AIRPORTS MAINTENANCE SUPERVISOR\n",
      "AIRPORTS PUBLIC AND COMMUNITY RELATIONS DIRECTOR\n",
      "ANIMAL CARE ASSISTANT\n",
      "ANIMAL CARE TECHNICIAN\n",
      "WATER TREATMENT OPERATOR\n",
      "ANIMAL CONTROL OFFICER\n",
      "ANIMAL KEEPER\n",
      "APPARATUS OPERATOR\n",
      "APPLICATIONS PROGRAMMER\n",
      "APPRENTICE - METAL TRADES\n",
      "APPRENTICE MACHINIST\n",
      "AQUARIST\n",
      "AQUARIUM EDUCATOR\n",
      "AQUATIC DIRECTOR\n",
      "AQUATIC FACILITY MANAGER\n",
      "AQUEDUCT AND RESERVOIR KEEPER\n",
      "AQUEDUCT AND RESERVOIR SUPERVISOR\n",
      "ARCHITECT\n",
      "ARCHITECTURAL ASSOCIATE\n",
      "ARCHITECTURAL DRAFTING TECHNICIAN\n",
      "ARCHIVIST\n",
      "ART CENTER DIRECTOR\n",
      "ART CURATOR\n",
      "ART INSTRUCTOR\n",
      "ARTS ASSOCIATE\n",
      "ARTS MANAGER\n",
      "ASBESTOS SUPERVISOR\n",
      "ASBESTOS WORKER\n",
      "ASPHALT PLANT OPERATOR\n",
      "ASPHALT PLANT SUPERVISOR\n",
      "ASSISTANT AIRPORT MANAGER\n",
      "ASSISTANT COMMUNICATIONS CABLE WORKER\n",
      "ASSISTANT COMMUNICATIONS ELECTRICIAN\n",
      "ASSISTANT DEPUTY SUPERINTENDENT OF BUILDING\n",
      "ASSISTANT DIRECTOR INFORMATION SYSTEMS\n",
      "ASSISTANT GARDENER \t\t\t\t \t\n",
      "ASSISTANT INSPECTOR\n",
      "ASSISTANT RETIREMENT PLAN MANAGER\n",
      "ASSISTANT SIGNAL SYSTEMS ELECTRICIAN\n",
      "ASSISTANT STREET LIGHTING ELECTRICIAN\n",
      "ASSISTANT TREE SURGEON\n",
      "ASSISTANT UTILITY BUYER\n",
      "ASSOCIATE ZONING ADMINISTRATOR\n",
      "AUDIO VISUAL TECHNICIAN\n",
      "AUDITOR\n",
      "AUTO BODY BUILDER AND REPAIRER\n",
      "AUTO BODY REPAIR SUPERVISOR\n",
      "AUTO ELECTRICIAN\n",
      "AUTO PAINTER\n",
      "AUTOMOTIVE DISPATCHER\n",
      "AUTOMOTIVE SUPERVISOR\n",
      "AVIONICS SPECIALIST\n",
      "BACKGROUND INVESTIGATION MANAGER\n",
      "BACKGROUND INVESTIGATOR\n",
      "BENEFITS SPECIALIST\n",
      "BLACKSMITH\n",
      "BOILERMAKER\n",
      "BOILERMAKER SUPERVISOR\n",
      "BUILDING CIVIL ENGINEER\n",
      "BUILDING CONSTRUCTION AND MAINTENANCE\n",
      "BUILDING ELECTRICAL ENGINEER\n",
      "BUILDING INSPECTOR\n",
      "BUILDING MAINTENANCE DISTRICT SUPERVISOR\n",
      "BUILDING MECHANICAL ENGINEER\n",
      "BUILDING MECHANICAL INSPECTOR\n",
      "BUILDING OPERATING ENGINEER\n",
      "BUILDING REPAIR SUPERVISOR\n",
      "BUILDING REPAIRER\n",
      "BUS OPERATOR\t\n",
      "BUS OPERATOR SUPERVISOR\n",
      "CABLE TELEVISION PRODUCTION MANAGER\n",
      "CARPENTER\n",
      "CARPENTER SUPERVISOR\n",
      "CARPET LAYER\n",
      "CEMENT FINISHER\n",
      "CEMENT FINISHER SUPERVISOR\n",
      "CEMENT FINISHER WORKER\n",
      "CHEMIST\n",
      "CHIEF ADMINISTRATIVE ANALYST\n",
      "CHIEF AIRPORTS ENGINEER\n",
      "CHIEF BENEFITS ANALYST\n",
      "CHIEF BUILDING OPERATING ENGINEER\n",
      "CHIEF CLERK\n",
      "CHIEF CLERK PERSONNEL\n",
      "CHIEF CLERK POLICE\n",
      "CHIEF CLERK POLICE\n",
      "CHIEF COMMUNICATIONS OPERATOR\n",
      "CHIEF CONSTRUCTION INSPECTOR\n",
      "CHIEF CUSTODIAN SUPERVISOR\n",
      "CHIEF ELECTRIC PLANT OPERATOR\n",
      "CHIEF ENVIRONMENTAL COMPLIANCE INSPECTOR\n",
      "CHIEF FINANCIAL OFFICER\n",
      "CHIEF FORENSIC CHEMIST\n",
      "CHIEF HARBOR ENGINEER\n",
      "CHIEF INSPECTOR\n",
      "CHIEF INTERNAL AUDITOR\n",
      "CHIEF MANAGEMENT ANALYST\n",
      "CHIEF OF AIRPORT PLANNING\n",
      "CHIEF OF DRAFTING OPERATIONS\n",
      "CHIEF OF OPERATIONS\n",
      "CHIEF OF PARKING ENFORCEMENT OPERATIONS\n",
      "CHIEF PARK RANGER\t\t\t\t\t\n",
      "CHIEF PORT PILOT\n",
      "CHIEF SAFETY ENGINEER PRESSURE VESSELS\n",
      "CHIEF SECURITY OFFICER\n",
      "CHIEF STREET SERVICES INVESTIGATOR\n",
      "CHIEF TAX COMPLIANCE OFFICER\n",
      "CHIEF TRANSPORTATION INVESTIGATOR\n",
      "CITY PLANNER\n",
      "CITY PLANNING ASSOCIATE\n",
      "CIVIL ENGINEER\n",
      "CIVIL ENGINEERING ASSOCIATE\n",
      "CIVIL ENGINEERING DRAFTING TECHNICIAN\n",
      "CLAIMS AGENT\n",
      "COMMERCIAL FIELD REPRESENTATIVE\n",
      "COMMERCIAL FIELD SUPERVISOR\n",
      "COMMERCIAL SERVICE SUPERVISOR\n",
      "COMMISSION EXECUTIVE ASSISTANT\n",
      "COMMUNICATIONS CABLE SUPERVISOR\n",
      "COMMUNICATIONS CABLE WORKER\n",
      "COMMUNICATIONS ELECTRICIAN\n",
      "COMMUNICATIONS ELECTRICIAN SUPERVISOR\n",
      "COMMUNICATIONS ENGINEER\n",
      "COMMUNICATIONS ENGINEERING ASSOCIATE\n",
      "COMMUNICATIONS INFORMATION REPRESENTATIVE\t\n",
      "COMMUNITY AFFAIRS ADVOCATE\n",
      "COMMUNITY HOUSING PROGRAMS MANAGER\n",
      "COMMUNITY PROGRAM ASSISTANT\n",
      "COMPLIANCE PROGRAM MANAGER\n",
      "CONSTRUCTION AND MAINTENANCE SUPERINTENDENT\n",
      "CONSTRUCTION AND MAINTENANCE SUPERVISOR\n",
      "CONSTRUCTION EQUIPMENT SERVICE WORKER\n",
      "CONSTRUCTION ESTIMATOR\n",
      "CONSTRUCTION INSPECTOR\n",
      "CONTRACT ADMINISTRATOR\n",
      "CONTROL SYSTEMS ENGINEERING ASSOCIATE\n",
      "CORRECTIONAL NURSE\t\n",
      "CRIME AND INTELLIGENCE ANALYST\n",
      "CRIMINALIST\n",
      "CUSTODIAL SERVICES ASSISTANT\n",
      "CUSTODIAN\n",
      "CUSTODIAN SUPERVISOR\n",
      "CUSTOMER SERVICE REPRESENTATIVE\n",
      "CUSTOMER SERVICE SPECIALIST\n",
      "DATA PROCESSING TECHNICIAN\n",
      "DATABASE ARCHITECT\n",
      "DECK HAND\n",
      "DELIVERY DRIVER\n",
      "DEPARTMENTAL CHIEF ACCOUNTANT\n",
      "DETENTION OFFICER\n",
      "DIRECTOR OF AIRPORT MARKETING\n",
      "DIRECTOR OF AIRPORT OPERATIONS\n",
      "DIRECTOR OF AIRPORTS ADMINISTRATION\tREVISED\n",
      "DIRECTOR OF COMMUNICATIONS SERVICES\n",
      "DIRECTOR OF ENFORCEMENT OPERATIONS\n",
      "DIRECTOR OF FIELD OPERATIONS\n",
      "DIRECTOR OF HOUSING\n",
      "DIRECTOR OF MAINTENANCE AIRPORTS\n",
      "DIRECTOR OF POLICE TRANSPORTATION\n",
      "DIRECTOR OF PORT CONSTRUCTION AND MAINTENANCE\n",
      "DIRECTOR OF PRINTING SERVICES\n",
      "DIRECTOR OF SECURITY SERVICES\n",
      "DIRECTOR OF SYSTEMS\n",
      "DISTRICT SUPERVISOR ANIMAL SERVICES\n",
      "DIVISION LIBRARIAN\n",
      "DRILL RIG OPERATOR\n",
      "DUPLICATING MACHINE OPERATOR\n",
      "ELECTRIC DISTRIBUTION MECHANIC\n",
      "ELECTRIC DISTRIBUTION MECHANIC SUPERVISOR\n",
      "ELECTRIC METER SETTER\n",
      "ELECTRIC SERVICE REPRESENTATIVE\n",
      "ELECTRIC STATION OPERATOR\n",
      "ELECTRIC TROUBLE DISPATCHER\n",
      "ELECTRICAL CRAFT HELPER\n",
      "ELECTRICAL ENGINEERING ASSOCIATE\n",
      "ELECTRICAL ENGINEERING DRAFTING TECHNICIAN\n",
      "ELECTRICAL INSPECTOR\n",
      "ELECTRICAL MECHANIC\t\t\n",
      "ELECTRICAL MECHANIC SUPERVISOR\n",
      "ELECTRICAL REPAIR SUPERVISOR\n",
      "ELECTRICAL REPAIRER\n",
      "ELECTRICAL SERVICE WORKER\n",
      "ELECTRICAL SERVICES MANAGER\n",
      "ELECTRICAL TESTER \t\n",
      "ELECTRICIAN\n",
      "ELECTRICIAN SUPERVISOR\n",
      "ELEVATOR MECHANIC\n",
      "ELEVATOR MECHANIC HELPER\n",
      "ELEVATOR REPAIR SUPERVISOR\n",
      "EMERGENCY MANAGEMENT COORDINATOR\n",
      "EMERGENCY MEDICAL SERVICES (EMS) EDUCATOR\n",
      "EMS ADVANCED PROVIDER\n",
      "EMS NURSE PRACTITIONER SUPERVISOR\n",
      "ENGINEER OF FIRE DEPARTMENT\n",
      "ENGINEER OF SURVEYS\t\n",
      "ENGINEERING DESIGNER\n",
      "ENGINEERING GEOLOGIST\n",
      "ENGINEERING GEOLOGIST ASSOCIATE\n",
      "ENVIRONMENTAL AFFAIRS OFFICER\n",
      "ENVIRONMENTAL COMPLIANCE INSPECTOR\n",
      "ENVIRONMENTAL ENGINEER\n",
      "ENVIRONMENTAL ENGINEERING ASSOCIATE\n",
      "ENVIRONMENTAL SPECIALIST\n",
      "ENVIRONMENTAL SUPERVISOR\n",
      "EQUIPMENT MECHANIC (Automotive Mechanic)\n",
      "EQUIPMENT OPERATOR\n",
      "EQUIPMENT REPAIR SUPERVISOR\n",
      "EQUIPMENT SPECIALIST\n",
      "EQUIPMENT SUPERINTENDENT\n",
      "EQUIPMENT SUPERVISOR\n",
      "EXAMINER OF QUESTIONED DOCUMENTS\n",
      "EXECUTIVE ADMINISTRATIVE ASSISTANT\n",
      "EXECUTIVE ASSISTANT AIRPORTS\n",
      "EXHIBIT PREPARATOR\n",
      "FIELD ENGINEERING AIDE\n",
      "FINANCIAL ANALYST\n",
      "FINANCIAL DEVELOPMENT OFFICER\n",
      "FINANCIAL MANAGER\n",
      "FINGERPRINT IDENTIFICATION EXPERT\t\n",
      "FIRE ASSISTANT CHIEF\n",
      "FIRE BATTALION CHIEF\n",
      "FIRE CAPTAIN\n",
      "FIRE HELICOPTER PILOT\n",
      "FIRE INSPECTOR\n",
      "FIRE PROTECTION ENGINEERING ASSOCIATE\n",
      "FIRE SPECIAL INVESTIGATOR\n",
      "FIRE SPRINKLER INSPECTOR\n",
      "FIREARMS EXAMINER\n",
      "FIREBOAT MATE\n",
      "FIREBOAT PILOT\n",
      "FIREFIGHTER\n",
      "FISCAL SYSTEMS SPECIALIST\n",
      "FLEET SERVICES MANAGER\n",
      "FORENSIC PRINT SPECIALIST\n",
      "GALLERY ATTENDANT\n",
      "GARAGE ASSISTANT\n",
      "GARAGE ATTENDANT\n",
      "GARDENER CARETAKER\n",
      "GENERAL AUTOMOTIVE SUPERVISOR\n",
      "GENERAL SERVICES MANAGER\n",
      "GEOGRAPHIC INFORMATION SYSTEMS CHIEF\n",
      "GEOGRAPHIC INFORMATION SYSTEMS SPECIALIST\n",
      "GEOGRAPHIC INFORMATION SYSTEMS SUPERVISOR\n",
      "GEOTECHNICAL ENGINEER\n",
      "GOLF STARTER\n",
      "GOLF STARTER SUPERVISOR\n",
      "GRAPHICS DESIGNER\n",
      "GRAPHICS SUPERVISOR\n",
      "HARBOR ENGINEER\n",
      "HARBOR PLANNING AND ECONOMIC ANALYST\n",
      "HARBOR PLANNING AND RESEARCH DIRECTOR\n",
      "HEAD CUSTODIAN SUPERVISOR\n",
      "HEATING AND REFRIGERATION INSPECTOR\n",
      "HEAVY DUTY EQUIPMENT MECHANIC\n",
      "HEAVY DUTY TRUCK OPERATOR\n",
      "HELICOPTER MECHANIC\n",
      "HELICOPTER MECHANIC SUPERVISOR\n",
      "HOUSING INSPECTOR\n",
      "HOUSING INVESTIGATOR\n",
      "HOUSING PLANNING AND ECONOMIC ANALYST\n",
      "HUMAN RELATIONS ADVOCATE\n",
      "HYDROGRAPHER\n",
      "IMPROVEMENT ASSESSOR SUPERVISOR\n",
      "INDUSTRIAL AND COMMERCIAL FINANCE OFFICER\n",
      "INDUSTRIAL CHEMIST\n",
      "INDUSTRIAL GRAPHICS SUPERVISOR\n",
      "INDUSTRIAL HYGIENIST\n",
      "INFORMATION SERVICES SPECIALIST\n",
      "INFORMATION SYSTEMS MANAGER\n",
      "INSTRUMENT MECHANIC\n",
      "INSTRUMENT MECHANIC SUPERVISOR\n",
      "INTERNAL AUDITOR\n",
      "INVESTMENT OFFICER\n",
      "IRRIGATION SPECIALIST\n",
      "LABOR SUPERVISOR\n",
      "LABORATORY TECHNICIAN\n",
      "LAND SURVEYING ASSISTANT\n",
      "LANDSCAPE ARCHITECT\n",
      "LANDSCAPE ARCHITECTURAL ASSOCIATE\n",
      "LEGISLATIVE ASSISTANT\n",
      "LEGISLATIVE REPRESENTATIVE\n",
      "LIBRARIAN\n",
      "LIBRARY ASSISTANT\n",
      "LICENSED VOCATIONAL NURSE\n",
      "LINE MAINTENANCE ASSISTANT\n",
      "LOCKSMITH\n",
      "MACHINIST\n",
      "MACHINIST SUPERVISOR\n",
      "MAINTENANCE AND CONSTRUCTION HELPER\n",
      "MAINTENANCE LABORER\n",
      "MANAGEMENT AIDE\n",
      "MANAGEMENT ANALYST\n",
      "MANAGEMENT ASSISTANT\t\n",
      "MANAGING WATER UTILITY ENGINEER\n",
      "MARINE AQUARIUM CURATOR\n",
      "MARINE AQUARIUM PROGRAM DIRECTOR\n",
      "MARINE ENVIRONMENTAL MANAGER\n",
      "MARINE ENVIRONMENTAL SUPERVISOR\n",
      "MASONRY WORKER\n",
      "MATERIALS TESTING ENGINEERING ASSOCIATE\n",
      "MATERIALS TESTING TECHNICIAN\n",
      "MECHANICAL ENGINEER\n",
      "MECHANICAL ENGINEERING ASSOCIATE\n",
      "MECHANICAL ENGINEERING DRAFTING TECHNICIAN\n",
      "MECHANICAL HELPER\n",
      "MECHANICAL REPAIR GENERAL SUPERVISOR\n",
      "MECHANICAL REPAIR SUPERVISOR\n",
      "MECHANICAL REPAIRER\n",
      "MEDICAL ASSISTANT\n",
      "METER READER\n",
      "MOTION PICTURE AND TELEVISION MANAGER\n",
      "MOTOR SWEEPER OPERATOR\n",
      "OCCUPATIONAL HEALTH NURSE\n",
      "OFFICE ENGINEERING TECHNICIAN\n",
      "OFFICE SERVICES ASSISTANT\n",
      "OFFICE TRAINEE\n",
      "OPERATIONS AND STATISTICAL RESEARCH ANALYST\n",
      "PAINTER\n",
      "PAINTER SUPERVISOR\n",
      "PARK MAINTENANCE SUPERVISOR\n",
      "PARK RANGER\t\n",
      "PARK SERVICES ATTENDANT\n",
      "PARK SERVICES SUPERVISOR\n",
      "PARKING ATTENDANT\n",
      "PARKING ENFORCEMENT MANAGER\n",
      "PARKING MANAGER\n",
      "PARKING METER TECHNICIAN\n",
      "PARKING METER TECHNICIAN SUPERVISOR\n",
      "PAYROLL ANALYST\n",
      "PAYROLL SUPERVISOR\n",
      "PERFORMING ARTS DIRECTOR\n",
      "PERSONNEL ANALYST\n",
      "PERSONNEL DIRECTOR\n",
      "PERSONNEL RECORDS SUPERVISOR\n",
      "PERSONNEL RESEARCH ANALYST\n",
      "PHOTOGRAPHER\n",
      "PILE DRIVER WORKER\n",
      "PIPEFITTER\t\n",
      "PIPEFITTER SUPERVISOR\n",
      "PLANNING ASSISTANT\n",
      "PLUMBER\n",
      "PLUMBER SUPERVISOR\n",
      "PLUMBING INSPECTOR\n",
      "POLICE ADMINISTRATOR\n",
      "POLICE CAPTAIN\n",
      "POLICE COMMANDER\t\n",
      "POLICE DETECTIVE\n",
      "POLICE LIEUTENANT\n",
      "POLICE OFFICER\n",
      "POLICE PERFORMANCE AUDITOR\n",
      "POLICE SERGEANT\t\t\n",
      "POLICE SERVICE REPRESENTATIVE\n",
      "POLICE SPECIAL INVESTIGATOR\n",
      "POLICE SPECIALIST\n",
      "POLICE SURVEILLANCE SPECIALIST\n",
      "POLYGRAPH EXAMINER\n",
      "PORT ELECTRICAL MECHANIC\n",
      "PORT ELECTRICAL MECHANIC SUPERVISOR\n",
      "PORT MAINTENANCE SUPERVISOR\n",
      "PORT PILOT\t\t\n",
      "PORT POLICE CAPTAIN\n",
      "PORT POLICE LIEUTENANT\n",
      "PORT POLICE OFFICER\n",
      "PORT POLICE SERGEANT\n",
      "PORTFOLIO MANAGER\n",
      "POWER ENGINEERING MANAGER\n",
      "POWER SHOVEL OPERATOR\n",
      "PRE-PRESS OPERATOR \t\n",
      "PRINCIPAL ACCOUNTANT\n",
      "PRINCIPAL ANIMAL KEEPER\n",
      "PRINCIPAL CITY PLANNER\n",
      "PRINCIPAL CIVIL ENGINEER\n",
      "PRINCIPAL CIVIL ENGINEERING DRAFTING TECHNICIAN\n",
      "PRINCIPAL CLERK\n",
      "PRINCIPAL CLERK POLICE\n",
      "PRINCIPAL CLERK UTILITY\n",
      "PRINCIPAL COMMUNICATIONS OPERATOR\n",
      "PRINCIPAL CONSTRUCTION INSPECTOR\n",
      "PRINCIPAL DEPUTY CONTROLLER\n",
      "PRINCIPAL DETENTION OFFICER\n",
      "PRINCIPAL ELECTRIC TROUBLE DISPATCHER\n",
      "PRINCIPAL ELECTRICAL ENGINEERING DRAFTING TECHNICIAN\n",
      "PRINCIPAL ENVIRONMENTAL ENGINEER\n",
      "PRINCIPAL GROUNDS MAINTENANCE SUPERVISOR\n",
      "PRINCIPAL INSPECTOR\n",
      "PRINCIPAL LIBRARIAN\n",
      "PRINCIPAL MECHANICAL ENGINEERING DRAFTING TECHNICIAN\n",
      "PRINCIPAL PHOTOGRAPHER\n",
      "PRINCIPAL PROPERTY OFFICER\n",
      "PRINCIPAL PUBLIC RELATIONS REPRESENTATIVE\n",
      "PRINCIPAL RECREATION SUPERVISOR\n",
      "PRINCIPAL SECURITY OFFICER\n",
      "PRINCIPAL STOREKEEPER\n",
      "PRINCIPAL TAX AUDITOR\n",
      "PRINCIPAL TAX COMPLIANCE OFFICER\t\t\n",
      "PRINCIPAL TRANSPORTATION ENGINEER\n",
      "PRINCIPAL UTILITY ACCOUNTANT\n",
      "PRINCIPAL WORKERS' COMPENSATION ANALYST\n",
      "PRINTING PRESS OPERATOR\n",
      "PRINTING SERVICES SUPERINTENDENT\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PROCUREMENT ANALYST\n",
      "PROCUREMENT SUPERVISOR\n",
      "PROGRAMMER ANALYST\n",
      "PROPERTY MANAGER\n",
      "PROPERTY OFFICER\n",
      "PROTECTIVE COATING WORKER\n",
      "PUBLIC INFORMATION DIRECTOR\n",
      "PUBLIC RELATIONS SPECIALIST\n",
      "PUBLIC SAFETY RISK MANAGER\n",
      "RATES MANAGER\t\n",
      "REAL ESTATE ASSOCIATE\n",
      "REAL ESTATE OFFICER\n",
      "REAL ESTATE TRAINEE\n",
      "RECREATION COORDINATOR\n",
      "RECREATION FACILITY DIRECTOR\n",
      "RECREATION SUPERVISOR\n",
      "REFUSE COLLECTION SUPERVISOR\t\n",
      "REFUSE COLLECTION TRUCK OPERATOR\n",
      "REFUSE CREW FIELD INSTRUCTOR\n",
      "REHABILITATION CONSTRUCTION SPECIALIST\n",
      "REHABILITATION PROJECT COORDINATOR\n",
      "REINFORCING STEEL WORKER\n",
      "REPROGRAPHICS OPERATOR\n",
      "REPROGRAPHICS SUPERVISOR\n",
      "RETIREMENT PLAN MANAGER\n",
      "RIDESHARE PROGRAM ADMINISTRATOR\n",
      "RISK AND INSURANCE ASSISTANT\n",
      "RISK MANAGEMENT AND PREVENTION PROGRAM SPECIALIST\n",
      "RISK MANAGER\n",
      "ROOFER\n",
      "ROOFER SUPERVISOR\n",
      "SAFETY ADMINISTRATOR\n",
      "SAFETY ENGINEER\n",
      "SAFETY ENGINEER ELEVATORS\n",
      "SAFETY ENGINEER PRESSURE VESSELS\n",
      "SAFETY ENGINEERING ASSOCIATE\n",
      "SANITATION SOLID RESOURCES MANAGER\n",
      "SANITATION WASTEWATER MANAGER\n",
      "SECRETARY\n",
      "SECRETARY LEGAL\n",
      "SECURITY AIDE\n",
      "SECURITY OFFICER\n",
      "SENIOR ACCOUNTANT\n",
      "SENIOR ADMINISTRATIVE ANALYST\n",
      "SENIOR ADMINISTRATIVE CLERK\n",
      "SENIOR ANIMAL CONTROL OFFICER\n",
      "SENIOR ANIMAL KEEPER\n",
      "SENIOR ARCHITECT\n",
      "SENIOR ARCHITECTURAL DRAFTING TECHNICIAN\n",
      "SENIOR AUDITOR\n",
      "SENIOR AUTOMOTIVE SUPERVISOR\n",
      "SENIOR BUILDING INSPECTOR\n",
      "SENIOR BUILDING MECHANICAL INSPECTOR\n",
      "SENIOR BUILDING OPERATING ENGINEER\n",
      "SENIOR CARPENTER\n",
      "SENIOR CHEMIST\n",
      "SENIOR CITY PLANNER\n",
      "SENIOR CIVIL ENGINEER\n",
      "SENIOR CIVIL ENGINEERING DRAFTING TECHNICIAN\n",
      "SENIOR CLAIMS REPRESENTATIVE\n",
      "SENIOR COMMERCIAL FIELD REPRESENTATIVE\n",
      "SENIOR COMMUNICATIONS CABLE WORKER\n",
      "SENIOR COMMUNICATIONS ELECTRICIAN\n",
      "SENIOR COMMUNICATIONS ELECTRICIAN SUPERVISOR\n",
      "SENIOR COMMUNICATIONS ENGINEER\n",
      "SENIOR COMMUNICATIONS OPERATOR\n",
      "SENIOR COMPUTER OPERATOR\n",
      "SENIOR CONSTRUCTION ENGINEER\n",
      "SENIOR CONSTRUCTION ESTIMATOR\n",
      "SENIOR CONSTRUCTION INSPECTOR\n",
      "SENIOR CUSTODIAN\n",
      "SENIOR DATA PROCESSING TECHNICIAN\n",
      "SENIOR DETENTION OFFICER\n",
      "SENIOR ELECTRIC SERVICE REPRESENTATIVE\n",
      "SENIOR ELECTRIC TROUBLE DISPATCHER\n",
      "SENIOR ELECTRICAL ENGINEERING DRAFTING TECHNICIAN\n",
      "SENIOR ELECTRICAL INSPECTOR\n",
      "SENIOR ELECTRICAL MECHANIC\n",
      "SENIOR ELECTRICAL MECHANIC SUPERVISOR\n",
      "SENIOR ELECTRICAL REPAIR SUPERVISOR\n",
      "SENIOR ELECTRICAL TEST TECHNICIAN\n",
      "SENIOR ELECTRICIAN\n",
      "SENIOR ENVIRONMENTAL COMPLIANCE INSPECTOR\n",
      "SENIOR ENVIRONMENTAL ENGINEER\n",
      "SENIOR EQUIPMENT MECHANIC\n",
      "WATER UTILITY SUPERINTENDENT\n",
      "SENIOR FIRE PROTECTION ENGINEER\n",
      "SENIOR FORENSIC PRINT SPECIALIST\n",
      "SENIOR GARDENER\n",
      "SENIOR HEATING AND REFRIGERATION INSPECTOR\n",
      "SENIOR HEAVY DUTY EQUIPMENT MECHANIC\n",
      "SENIOR HOUSING INSPECTOR\n",
      "SENIOR HOUSING INVESTIGATOR\n",
      "SENIOR HYDROGRAPHER\n",
      "SENIOR INDUSTRIAL HYGIENIST\n",
      "SENIOR LABOR RELATIONS SPECIALIST\n",
      "SENIOR LEGISLATIVE ASSISTANT\n",
      "SENIOR LIBRARIAN\n",
      "SENIOR LOAD DISPATCHER\n",
      "SENIOR MACHINIST SUPERVISOR\n",
      "SENIOR MANAGEMENT ANALYST\n",
      "SENIOR MECHANICAL ENGINEERING DRAFTING TECHNICIAN\n",
      "SENIOR MECHANICAL REPAIRER\n",
      "SENIOR PAINTER\n",
      "SENIOR PARK MAINTENANCE SUPERVISOR\n",
      "SENIOR PARK RANGER\n",
      "SENIOR PARK SERVICES ATTENDANT\n",
      "SENIOR PARKING ATTENDANT\n",
      "SENIOR PERSONNEL ANALYST\n",
      "SENIOR PHOTOGRAPHER\t\n",
      "SENIOR PLUMBER\n",
      "SENIOR PLUMBING INSPECTOR\n",
      "SENIOR POLICE SERVICE REPRESENTATIVE\n",
      "SENIOR PORT ELECTRICAL MECHANIC\n",
      "SENIOR PROPERTY OFFICER\n",
      "SENIOR REAL ESTATE OFFICER\n",
      "SENIOR RECREATION DIRECTOR\t\n",
      "SENIOR ROOFER\n",
      "SENIOR SAFETY ENGINEER ELEVATORS\n",
      "SENIOR SAFETY ENGINEER PRESSURE VESSELS\n",
      "SENIOR SECURITY OFFICER\n",
      "SENIOR STOREKEEPER\n",
      "SENIOR STREET LIGHTING ENGINEER\n",
      "SENIOR STRUCTURAL ENGINEER\n",
      "SENIOR SURVEY SUPERVISOR\n",
      "SENIOR SYSTEMS ANALYST\n",
      "SENIOR TAX AUDITOR\n",
      "SENIOR TITLE EXAMINER\n",
      "SENIOR TRAFFIC SUPERVISOR\n",
      "SENIOR TRANSPORTATION ENGINEER\n",
      "SENIOR TRANSPORTATION INVESTIGATOR\n",
      "SENIOR UNDERGROUND DISTRIBUTION CONSTRUCTION SUPERVISOR\n",
      "SENIOR UTILITY ACCOUNTANT\n",
      "SENIOR UTILITY BUYER\n",
      "SENIOR UTILITY SERVICES SPECIALIST\n",
      "SENIOR UTILITY SERVICES SPECIALIST\n",
      "SENIOR WASTEWATER TREATMENT OPERATOR\n",
      "SENIOR WINDOW CLEANER\n",
      "SENIOR WORKERS' COMPENSATION ANALYST\n",
      "SHEET METAL SUPERVISOR\n",
      "SHEET METAL WORKER\n",
      "SHIFT SUPERINTENDENT WASTEWATER TREATMENT\t\n",
      "SHIP CARPENTER\n",
      "SHOPS SUPERINTENDENT\n",
      "SIGN PAINTER\n",
      "SIGN SHOP SUPERVISOR\n",
      "SIGNAL SYSTEMS ELECTRICIAN\t\t\n",
      "SIGNAL SYSTEMS SUPERINTENDENT\n",
      "SIGNAL SYSTEMS SUPERVISOR\n",
      "SOCIAL WORKER\n",
      "SOLID RESOURCES SUPERINTENDENT\n",
      "SOLID WASTE DISPOSAL SUPERINTENDENT\n",
      "SPECIAL INVESTIGATOR\n",
      "SENIOR CRIME AND INTELLIGENCE ANALYST\n",
      "STAFF ASSISTANT TO GENERAL MANAGER WATER AND POWER\n",
      "STEAM PLANT ASSISTANT\n",
      "STEAM PLANT MAINTENANCE MECHANIC\n",
      "STEAM PLANT MAINTENANCE SUPERVISOR\n",
      "STEAM PLANT OPERATOR\n",
      "STOREKEEPER\n",
      "STORES SUPERVISOR\n",
      "STREET LIGHTING CONSTRUCTION AND MAINTENANCE SUPERINTENDENT\n",
      "STREET LIGHTING ELECTRICIAN\n",
      "STREET LIGHTING ELECTRICIAN SUPERVISOR\n",
      "STREET LIGHTING ENGINEER\n",
      "STREET LIGHTING ENGINEERING ASSOCIATE\n",
      "STREET SERVICES GENERAL SUPERINTENDENT\n",
      "STREET SERVICES INVESTIGATOR\n",
      "STREET SERVICES SUPERINTENDENT\n",
      "STREET SERVICES SUPERVISOR\n",
      "STREET SERVICES WORKER\n",
      "STREET TREE SUPERINTENDENT\n",
      "STRUCTURAL ENGINEER\n",
      "STRUCTURAL ENGINEERING ASSOCIATE\n",
      "STRUCTURAL STEEL FABRICATOR\n",
      "STRUCTURAL STEEL FABRICATOR SUPERVISOR\n",
      "SUPERINTENDENT OF RECREATION AND PARKS OPERATIONS\n",
      "SUPERVISING CRIMINALIST\n",
      "SUPERVISING OCCUPATIONAL HEALTH NURSE\n",
      "SUPERVISING TRANSPORTATION PLANNER\n",
      "SUPERVISING WATER SERVICE REPRESENTATIVE\n",
      "SUPPLY SERVICES MANAGER\n",
      "SUPPLY SERVICES PAYMENT CLERK\n",
      "SURVEY PARTY CHIEF\n",
      "SURVEY SUPERVISOR\n",
      "SYSTEMS AIDE\n",
      "SYSTEMS ANALYST\n",
      "SYSTEMS PROGRAMMER\n",
      "TAX AUDITOR\n",
      "TAX COMPLIANCE AIDE\t\n",
      "TAX COMPLIANCE OFFICER\n",
      "TELECOMMUNICATIONS PLANNING AND UTILIZATION OFFICER\n",
      "TILE SETTER\n",
      "TIRE REPAIRER\n",
      "TITLE EXAMINER\n",
      "TRAFFIC MARKING AND SIGN SUPERINTENDENT\n",
      "TRAFFIC OFFICER\n",
      "TRAFFIC PAINTER AND SIGN POSTER\n",
      "TRANSMISSION AND DISTRIBUTION DISTRICT SUPERVISOR\n",
      "TRANSPORTATION ENGINEER\n",
      "TRANSPORTATION ENGINEERING AIDE\n",
      "TRANSPORTATION ENGINEERING ASSOCIATE\n",
      "TRANSPORTATION INVESTIGATOR\n",
      "TRANSPORTATION PLANNING ASSOCIATE\n",
      "TREASURY ACCOUNTANT\n",
      "TREE SURGEON\n",
      "TREE SURGEON ASSISTANT\n",
      "TREE SURGEON SUPERVISOR\n",
      "TRUCK AND EQUIPMENT DISPATCHER\n",
      "TRUCK OPERATOR\n",
      "UNDERGROUND DISTRIBUTION CONSTRUCTION MECHANIC\n",
      "UNDERGROUND DISTRIBUTION CONSTRUCTION SUPERVISOR\n",
      "UPHOLSTERER\n",
      "UTILITIES SERVICE INVESTIGATOR\n",
      "UTILITY ACCOUNTANT\n",
      "UTILITY ADMINISTRATOR\n",
      "UTILITY BUYER\n",
      "UTILITY EXECUTIVE SECRETARY\n",
      "UTILITY SERVICES MANAGER\n",
      "UTILITY SERVICES SPECIALIST\n",
      "VETERINARY TECHNICIAN \t\t\t\t \t\n",
      "VIDEO PRODUCTION COORDINATOR\n",
      "VIDEO TECHNICIAN\n",
      "VOLUNTEER COORDINATOR\n",
      "WAREHOUSE AND TOOLROOM WORKER\n",
      "WASTEWATER TREATMENT OPERATOR\n",
      "WASTEWATER COLLECTION WORKER\n",
      "WASTEWATER TREATMENT ELECTRICIAN\n",
      "WASTEWATER TREATMENT ELECTRICIAN SUPERVISOR\n",
      "WASTEWATER TREATMENT LABORATORY MANAGER\n",
      "WASTEWATER TREATMENT MECHANIC\n",
      "WASTEWATER TREATMENT MECHANIC SUPERVISOR\n",
      "WASTEWATER TREATMENT OPERATOR\n",
      "WATER BIOLOGIST\n",
      "WATER MICROBIOLOGIST\n",
      "WATER SERVICE REPRESENTATIVE\n",
      "WATER SERVICE SUPERVISOR\n",
      "WATER SERVICE WORKER\n",
      "WATER SERVICES MANAGER\n",
      "WATER TREATMENT OPERATOR\n",
      "WATER TREATMENT SUPERVISOR\n",
      "WATER UTILITY OPERATOR\n",
      "WATER UTILITY OPERATOR SUPERVISOR\n",
      "WATER UTILITY SUPERINTENDENT\n",
      "WATER UTILITY SUPERVISOR\n",
      "WATER UTILITY WORKER\n",
      "WATERSHED RESOURCES SPECIALIST\n",
      "WATERWORKS ENGINEER\n",
      "WATERWORKS MECHANIC SUPERVISOR\n",
      "WELDER\n",
      "WELDER SUPERVISOR\n",
      "WHARFINGER\n",
      "WINDOW CLEANER\n",
      "WORKERS' COMPENSATION ANALYST\n",
      "WORKERS' COMPENSATION CLAIMS ASSISTANT\n",
      "X-Ray and Laboratory Technician\n",
      "ZOO CURATOR\n",
      "ZOO CURATOR OF EDUCATION\n",
      "ZOO REGISTRAR\n"
     ]
    }
   ],
   "source": [
    "# Rerun the function job_class_title using cleaned data\n",
    "for file_name in cleaned_jobs:\n",
    "    job_path     = cleaned_path + file_name     # define path to file_name\n",
    "    cleaned_job  = open(job_path, 'rt').read()  # read in job as a string\n",
    "    try:\n",
    "        print(job_class_title(job=cleaned_job)) \n",
    "    except:                                     # do some pretty printings here to help our eyes from pain\n",
    "        ## define some useful variables\n",
    "        border_line = '##############################################################################################'\n",
    "        how_many    = int((len(border_line) - len(job_path))/2)\n",
    "        print(border_line)\n",
    "        ## do pretty printings\n",
    "        print('#'*how_many + job_path + '#'*how_many)\n",
    "        print(border_line)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Normalize JOB_CLASS_NO (jcn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This is one of the helper function\n",
    "def job_class_no(job):\n",
    "    '''Returns the field JOB_CLASS_NO (jcn)'''\n",
    "    # From the word Class Code to the word Open Date is where the information located.\n",
    "    temp = job[job.index('Class Code'):job.index('Open Date')]\n",
    "    # Check if anything in temp is a digit via isdigit(). If it is, get it\n",
    "    jcn  = [string_num for string_num in temp.split() if string_num.isdigit()][0] # first element is what we want\n",
    "    \n",
    "    return jcn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9206\n",
      "1513\n",
      "1223\n",
      "1119\n",
      "1590\n",
      "1358\n",
      "9135\n",
      "2325\n",
      "3774\n",
      "3781\n",
      "1540\n",
      "1404\n",
      "7256\n",
      "0845\n",
      "1783\n",
      "9210\n",
      "7260\n",
      "3228\n",
      "3227\n",
      "3225\n",
      "3236\n",
      "7268\n",
      "3331\n",
      "3336\n",
      "1788\n",
      "4323\n",
      "4310\n",
      "5885\n",
      "4311\n",
      "4304\n",
      "2121\n",
      "1429\n",
      "3789\n",
      "3764\n",
      "2400\n",
      "2493\n",
      "2419\n",
      "2423\n",
      "5813\n",
      "5816\n",
      "7925\n",
      "7926\n",
      "7922\n",
      "1191\n",
      "2478\n",
      "2448\n",
      "2447\n",
      "2454\n",
      "2455\n",
      "3440\n",
      "3435\n",
      "4143\n",
      "4145\n",
      "7259\n",
      "3808\n",
      "3684\n",
      "4219\n",
      "9377\n",
      "3142\n",
      "4208\n",
      "9415\n",
      "3818\n",
      "3809\n",
      "3150\n",
      "1860\n",
      "7998\n",
      "6147\n",
      "1517\n",
      "3704\n",
      "3706\n",
      "3707\n",
      "3721\n",
      "3595\n",
      "3714\n",
      "3565\n",
      "1759\n",
      "1764\n",
      "1203\n",
      "3733\n",
      "3735\n",
      "3737\n",
      "7244\n",
      "3124\n",
      "7543\n",
      "4211\n",
      "3190\n",
      "7561\n",
      "4251\n",
      "5923\n",
      "3338\n",
      "3333\n",
      "3588\n",
      "3589\n",
      "1801\n",
      "3344\n",
      "3346\n",
      "3418\n",
      "3353\n",
      "3354\n",
      "3351\n",
      "7833\n",
      "1554\n",
      "7274\n",
      "9151\n",
      "5927\n",
      "1253\n",
      "1260\n",
      "1249\n",
      "1249\n",
      "1466\n",
      "7296\n",
      "3182\n",
      "5237\n",
      "4289\n",
      "9230\n",
      "2237\n",
      "9286\n",
      "4254\n",
      "1619\n",
      "9182\n",
      "7945\n",
      "7271\n",
      "7258\n",
      "9180\n",
      "1968\n",
      "5154\n",
      "4260\n",
      "3187\n",
      "4286\n",
      "1211\n",
      "4275\n",
      "7944\n",
      "7941\n",
      "7237\n",
      "7246\n",
      "7232\n",
      "1767\n",
      "1600\n",
      "1603\n",
      "1213\n",
      "9734\n",
      "3800\n",
      "3802\n",
      "3686\n",
      "3689\n",
      "7610\n",
      "7607\n",
      "1461\n",
      "2496\n",
      "8500\n",
      "2501\n",
      "9165\n",
      "3129\n",
      "3127\n",
      "3541\n",
      "3341\n",
      "7291\n",
      "9168\n",
      "7230\n",
      "2317\n",
      "2236\n",
      "2234\n",
      "3149\n",
      "3156\n",
      "3176\n",
      "1230\n",
      "1229\n",
      "1136\n",
      "1470\n",
      "5131\n",
      "1121\n",
      "1593\n",
      "3211\n",
      "1768\n",
      "9304\n",
      "9302\n",
      "7625\n",
      "4266\n",
      "4321\n",
      "1568\n",
      "7270\n",
      "3722\n",
      "3123\n",
      "1488\n",
      "3208\n",
      "9375\n",
      "4320\n",
      "6157\n",
      "3521\n",
      "1493\n",
      "3879\n",
      "3873\n",
      "3822\n",
      "7520\n",
      "5224\n",
      "3828\n",
      "3799\n",
      "7525\n",
      "7532\n",
      "4221\n",
      "3841\n",
      "3835\n",
      "3855\n",
      "3853\n",
      "3825\n",
      "5265\n",
      "7512\n",
      "3863\n",
      "3865\n",
      "3866\n",
      "3860\n",
      "3869\n",
      "1702\n",
      "2322\n",
      "2341\n",
      "2340\n",
      "2131\n",
      "9486\n",
      "7217\n",
      "7255\n",
      "7253\n",
      "7320\n",
      "4292\n",
      "7872\n",
      "7871\n",
      "7310\n",
      "7304\n",
      "3711\n",
      "3525\n",
      "3746\n",
      "3734\n",
      "3750\n",
      "3527\n",
      "3229\n",
      "1117\n",
      "9186\n",
      "2444\n",
      "7228\n",
      "1549\n",
      "1571\n",
      "1557\n",
      "1157\n",
      "2166\n",
      "2152\n",
      "2142\n",
      "3563\n",
      "2128\n",
      "7978\n",
      "1632\n",
      "4240\n",
      "2233\n",
      "5125\n",
      "5127\n",
      "2112\n",
      "1555\n",
      "9103\n",
      "2200\n",
      "2442\n",
      "3538\n",
      "3531\n",
      "3141\n",
      "3718\n",
      "9601\n",
      "7211\n",
      "7213\n",
      "7214\n",
      "7239\n",
      "2453\n",
      "2479\n",
      "1670\n",
      "7935\n",
      "9279\n",
      "9224\n",
      "9234\n",
      "3178\n",
      "4245\n",
      "3743\n",
      "3584\n",
      "3742\n",
      "3749\n",
      "4243\n",
      "8516\n",
      "8504\n",
      "9207\n",
      "7263\n",
      "1564\n",
      "9191\n",
      "7834\n",
      "1490\n",
      "2330\n",
      "1456\n",
      "1409\n",
      "3843\n",
      "3844\n",
      "1625\n",
      "9146\n",
      "3913\n",
      "3126\n",
      "7854\n",
      "7283\n",
      "7929\n",
      "7933\n",
      "1182\n",
      "9482\n",
      "6152\n",
      "1172\n",
      "2332\n",
      "3882\n",
      "3393\n",
      "3763\n",
      "3766\n",
      "3115\n",
      "3112\n",
      "1508\n",
      "9184\n",
      "1539\n",
      "9406\n",
      "2425\n",
      "2403\n",
      "9437\n",
      "##############################################################################################\n",
      "##########CityofLA/Job Bulletins/MARINE ENVIRONMENTAL SUPERVISOR 9433 071114 (1).txt##########\n",
      "##############################################################################################\n",
      "3451\n",
      "7967\n",
      "7968\n",
      "7558\n",
      "7554\n",
      "7551\n",
      "3771\n",
      "3731\n",
      "3795\n",
      "3773\n",
      "2310\n",
      "1611\n",
      "1789\n",
      "3585\n",
      "2314\n",
      "7212\n",
      "1360\n",
      "1101\n",
      "1779\n",
      "3423\n",
      "3426\n",
      "3145\n",
      "1966\n",
      "2412\n",
      "2426\n",
      "3530\n",
      "9025\n",
      "9170\n",
      "3738\n",
      "3757\n",
      "1630\n",
      "1170\n",
      "2449\n",
      "1731\n",
      "1714\n",
      "1129\n",
      "1739\n",
      "1793\n",
      "3553\n",
      "3433\n",
      "3438\n",
      "7939\n",
      "3443\n",
      "3446\n",
      "4231\n",
      "9196\n",
      "2244\n",
      "2251\n",
      "2223\n",
      "2232\n",
      "2214\n",
      "1627\n",
      "2227\n",
      "2207\n",
      "1640\n",
      "2217\n",
      "3687\n",
      "2240\n",
      "3758\n",
      "3759\n",
      "3128\n",
      "5151\n",
      "3224\n",
      "3223\n",
      "3221\n",
      "3222\n",
      "9143\n",
      "9453\n",
      "3558\n",
      "1481\n",
      "1525\n",
      "4312\n",
      "7946\n",
      "9489\n",
      "7219\n",
      "1201\n",
      "1152\n",
      "1202\n",
      "1458\n",
      "7297\n",
      "9653\n",
      "3215\n",
      "3830\n",
      "7531\n",
      "7875\n",
      "3147\n",
      "4226\n",
      "6155\n",
      "7550\n",
      "1794\n",
      "3210\n",
      "1786\n",
      "2464\n",
      "3200\n",
      "1839\n",
      "1524\n",
      "1195\n",
      "9266\n",
      "1589\n",
      "1777\n",
      "1494\n",
      "1496\n",
      "1859\n",
      "1852\n",
      "1431\n",
      "1964\n",
      "3207\n",
      "3463\n",
      "##############################################################################################\n",
      "##############CityofLA/Job Bulletins/PUBLIC INFORMATION DIRECTOR 1800 030317.txt##############\n",
      "##############################################################################################\n",
      "1785\n",
      "7976\n",
      "5601\n",
      "1941\n",
      "1960\n",
      "1937\n",
      "2469\n",
      "2434\n",
      "2460\n",
      "4101\n",
      "3580\n",
      "4100\n",
      "1569\n",
      "8502\n",
      "3483\n",
      "3162\n",
      "3163\n",
      "9149\n",
      "2485\n",
      "1645\n",
      "7980\n",
      "1530\n",
      "3476\n",
      "3478\n",
      "1728\n",
      "1727\n",
      "4263\n",
      "4261\n",
      "1726\n",
      "4126\n",
      "4128\n",
      "1116\n",
      "1924\n",
      "3199\n",
      "3181\n",
      "1523\n",
      "1541\n",
      "1368\n",
      "4316\n",
      "4305\n",
      "7927\n",
      "7208\n",
      "1518\n",
      "3716\n",
      "4213\n",
      "4253\n",
      "5925\n",
      "3345\n",
      "7830\n",
      "7947\n",
      "9485\n",
      "7207\n",
      "1770\n",
      "1602\n",
      "3801\n",
      "3638\n",
      "3691\n",
      "7614\n",
      "1467\n",
      "1428\n",
      "7289\n",
      "3347\n",
      "7294\n",
      "3157\n",
      "1139\n",
      "3212\n",
      "7521\n",
      "3829\n",
      "7209\n",
      "4223\n",
      "3834\n",
      "3836\n",
      "3856\n",
      "7515\n",
      "3864\n",
      "4293\n",
      "7874\n",
      "3712\n",
      "3980\n",
      "7981\n",
      "2201\n",
      "3143\n",
      "4247\n",
      "3745\n",
      "4244\n",
      "8517\n",
      "7264\n",
      "2331\n",
      "9202\n",
      "1187\n",
      "6153\n",
      "5235\n",
      "3768\n",
      "9171\n",
      "7210\n",
      "3772\n",
      "3424\n",
      "3146\n",
      "1967\n",
      "2422\n",
      "3529\n",
      "9167\n",
      "1795\n",
      "3444\n",
      "4233\n",
      "2209\n",
      "3847\n",
      "3209\n",
      "1961\n",
      "2446\n",
      "3477\n",
      "4264\n",
      "4262\n",
      "3184\n",
      "1837\n",
      "9536\n",
      "9425\n",
      "7288\n",
      "1597\n",
      "1519\n",
      "1947\n",
      "3218\n",
      "9262\n",
      "4273\n",
      "3815\n",
      "1521\n",
      "1862\n",
      "3753\n",
      "3753\n",
      "4124\n",
      "3174\n",
      "1769\n",
      "3777\n",
      "3775\n",
      "7242\n",
      "3348\n",
      "3780\n",
      "3428\n",
      "3419\n",
      "3819\n",
      "3832\n",
      "3839\n",
      "2385\n",
      "4102\n",
      "4108\n",
      "0602\n",
      "2241\n",
      "9185\n",
      "5622\n",
      "5630\n",
      "3786\n",
      "5624\n",
      "1835\n",
      "1866\n",
      "3820\n",
      "3811\n",
      "3840\n",
      "7537\n",
      "7527\n",
      "4160\n",
      "4283\n",
      "4158\n",
      "4152\n",
      "4150\n",
      "3160\n",
      "7956\n",
      "7957\n",
      "3793\n",
      "3794\n",
      "2472\n",
      "2235\n",
      "2315\n",
      "2481\n",
      "1697\n",
      "1865\n",
      "1214\n",
      "7286\n",
      "7287\n",
      "1599\n",
      "1596\n",
      "1455\n",
      "1514\n",
      "1173\n",
      "1179\n",
      "7640\n",
      "3493\n",
      "3727\n",
      "1943\n",
      "3430\n",
      "3214\n",
      "3421\n",
      "3875\n",
      "7278\n",
      "7285\n",
      "7280\n",
      "4271\n",
      "2480\n",
      "1609\n",
      "3114\n",
      "3151\n",
      "3117\n",
      "3586\n",
      "3583\n",
      "3812\n",
      "3814\n",
      "3723\n",
      "1631\n",
      "1511\n",
      "9105\n",
      "1861\n",
      "1336\n",
      "9106\n",
      "3755\n",
      "2369\n",
      "1802\n",
      "6145\n",
      "2495\n",
      "1832\n",
      "4123\n",
      "4110\n",
      "5615\n",
      "5613\n",
      "7840\n",
      "5614\n",
      "5617\n",
      "4123\n",
      "7856\n",
      "7857\n",
      "1693\n",
      "3930\n",
      "3931\n",
      "9602\n",
      "5885\n",
      "5887\n",
      "5854\n",
      "5857\n",
      "3980\n",
      "3976\n",
      "3912\n",
      "7862\n",
      "7248\n",
      "3987\n",
      "3796\n",
      "3798\n",
      "1190\n",
      "3173\n",
      "1774\n",
      "1775\n",
      "2358\n",
      "4297\n",
      "4300\n",
      "4290\n"
     ]
    }
   ],
   "source": [
    "# Normalization Strategy: \n",
    "# print jct in the try clause and print job_path in the except clause\n",
    "# look at printouts and detect unusual jct's.\n",
    "for file_name in raw_jobs:\n",
    "    job_path = raw_path + file_name        # define path to file_name\n",
    "    raw_job  = open(job_path, 'rt').read() # read in job as a string\n",
    "    try:\n",
    "        print(job_class_no(job=raw_job))\n",
    "    except:                                # do some pretty printings here to help our eyes from pain\n",
    "        ## define some useful variables\n",
    "        border_line = '##############################################################################################'\n",
    "        how_many    = int((len(border_line) - len(job_path))/2)\n",
    "        print(border_line)\n",
    "        ## do pretty printings\n",
    "        print('#'*how_many + job_path + '#'*how_many)\n",
    "        print(border_line)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}