Verify_analysed_data_retrieval.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set up the dependencies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# for reading and validating data\n",
    "import emeval.input.spec_details as eisd\n",
    "import emeval.input.phone_view as eipv\n",
    "import emeval.input.eval_view as eiev"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Visualization helpers\n",
    "import emeval.viz.phone_view as ezpv\n",
    "import emeval.viz.eval_view as ezev"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Analytics results\n",
    "import emeval.metrics.baseline_segmentation as embs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# For plots\n",
    "import matplotlib.pyplot as plt\n",
    "from matplotlib.collections import PatchCollection\n",
    "from matplotlib.patches import Rectangle\n",
    "%matplotlib inline\n",
    "\n",
    "import IPython.display as ipyd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# For maps\n",
    "import folium\n",
    "import branca.element as bre"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# For easier debugging while working on modules\n",
    "import importlib"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "pd.options.display.float_format = '{:.6f}'.format\n",
    "import arrow\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "THIRTY_MINUTES = 30 * 60\n",
    "TEN_MINUTES = 10 * 60"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The spec\n",
    "\n",
    "The spec defines what experiments were done, and over which time ranges. Once the experiment is complete, most of the structure is read back from the data, but we use the spec to validate that it all worked correctly. The spec also contains the ground truth for the legs. Here, we read the spec for the trip to UC Berkeley."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "DATASTORE_LOC = \"bin/data\"\n",
    "AUTHOR_EMAIL = \"shankari@eecs.berkeley.edu\"\n",
    "sd_la = eisd.FileSpecDetails(DATASTORE_LOC, AUTHOR_EMAIL, \"unimodal_trip_car_bike_mtv_la\")\n",
    "sd_sj = eisd.FileSpecDetails(DATASTORE_LOC, AUTHOR_EMAIL, \"car_scooter_brex_san_jose\")\n",
    "sd_ucb = eisd.FileSpecDetails(DATASTORE_LOC, AUTHOR_EMAIL, \"train_bus_ebike_mtv_ucb\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The views\n",
    "\n",
    "There are two main views for the data - the phone view and the evaluation view. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Phone view\n",
    "\n",
    "In the phone view, the phone is primary, and then there is a tree that you can traverse to get the data that you want. Traversing that tree typically involves nested for loops; here's an example of loading the phone view and traversing it. You can replace the print statements with real code. When you are ready to check this in, please move the function to one of the python modules so that we can invoke it more generally"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "importlib.reload(eipv)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "pv_la = eipv.PhoneView(sd_la)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pv_sj = eipv.PhoneView(sd_sj)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pv_ucb = eipv.PhoneView(sd_ucb)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import emeval.analysed.phone_view as eapv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "importlib.reload(eapv)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "MASTER_DATA_LOC = \"bin/data/master_9b70c97\"\n",
    "master_spec = eisd.FileSpecDetails(MASTER_DATA_LOC, AUTHOR_EMAIL)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "av_la_master = eapv.create_analysed_view(pv_la, master_spec, \"analysis/recreated_location\", \"analysis/cleaned_trip\", \"analysis/inferred_section\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "GIS_DATA_LOC = \"bin/data/gis_9b679e3/\"\n",
    "gis_spec = eisd.FileSpecDetails(GIS_DATA_LOC, AUTHOR_EMAIL)\n",
    "av_la_gis = eapv.create_analysed_view(pv_la, gis_spec, \"analysis/recreated_location\", \"analysis/cleaned_trip\", \"analysis/inferred_section\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "range_0 = av_la_master.map()[\"ios\"][\"ucb-sdb-ios-3\"][\"evaluation_ranges\"][0]\n",
    "for t in range_0[\"sensed_trip_ranges\"]:\n",
    "    print(arrow.get(t[\"start_ts\"]).to(\"America/Los_angeles\"), arrow.get(t[\"end_ts\"]).to(\"America/Los_angeles\"))\n",
    "print(\"=======\")\n",
    "for s in range_0[\"sensed_section_ranges\"]:\n",
    "    print(arrow.get(s[\"start_ts\"]).to(\"America/Los_angeles\"), arrow.get(s[\"end_ts\"]).to(\"America/Los_angeles\"))\n",
    "print(\"=======\")\n",
    "\n",
    "for t in range_0[\"evaluation_trip_ranges\"]:\n",
    "    print(arrow.get(t[\"start_ts\"]).to(\"America/Los_angeles\"), arrow.get(t[\"end_ts\"]).to(\"America/Los_angeles\"))\n",
    "    for s in t[\"sensed_section_ranges\"]:\n",
    "        print(\"-------\", arrow.get(s[\"start_ts\"]).to(\"America/Los_angeles\"), arrow.get(s[\"end_ts\"]).to(\"America/Los_angeles\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# av_la = eapv.create_analysed_view(pv_la, \"http://localhost:8080\", \"analysis/recreated_location\", \"analysis/cleaned_trip\", \"analysis/inferred_section\")\n",
    "# av_sj = eapv.create_analysed_view(pv_sj, eisd.FileSpecDetails(\"bin/data/master_9b70c97\", AUTHOR_EMAIL), \"analysis/recreated_location\", \"analysis/cleaned_trip\", \"analysis/inferred_section\")\n",
    "# av_ucb = eapv.create_analysed_view(pv_ucb, eisd.FileSpecDetails(\"bin/data/master_9b70c97\", AUTHOR_EMAIL), \"analysis/recreated_location\", \"analysis/cleaned_trip\", \"analysis/inferred_section\")\n",
    "# av_ucb_reroute = eapv.create_analysed_view(pv_ucb_reroute, \"http://localhost:8080\", \"analysis/recreated_location\", \"analysis/cleaned_trip\", \"analysis/inferred_section\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def verify_range(eval_range):\n",
    "    if len(eval_range[\"evaluation_trip_ranges\"]) > len(eval_range[\"sensed_trip_ranges\"]):\n",
    "        print(f\"MISSING TRIP: found {len(eval_range['sensed_trip_ranges'])}, expected {len(eval_range['evaluation_trip_ranges'])}\")\n",
    "    if len(eval_range[\"evaluation_trip_ranges\"]) < len(eval_range[\"sensed_trip_ranges\"]):\n",
    "        print(f\"EXTRA TRIP: found {len(eval_range['sensed_trip_ranges'])}, expected {len(eval_range['evaluation_trip_ranges'])}\")\n",
    "    for i, t in enumerate(eval_range[\"evaluation_trip_ranges\"]):\n",
    "        if len(t[\"evaluation_section_ranges\"]) > len(t[\"sensed_section_ranges\"]):\n",
    "            print(f\"MISSING SECTION: In trip {i}, found {len(t['sensed_section_ranges'])}, expected {len(t['evaluation_section_ranges'])}\")\n",
    "        if len(t[\"evaluation_section_ranges\"]) < len(t[\"sensed_section_ranges\"]):\n",
    "            print(f\"EXTRA SECTION: In trip {i}, found {len(t['sensed_section_ranges'])}, expected {len(t['evaluation_section_ranges'])}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def verify_analysed_view(av):\n",
    "    for phone_os, phone_map in av.map().items():\n",
    "        print(15 * \"=*\")\n",
    "        for phone_label, phone_detail_map in phone_map.items():\n",
    "            for r in phone_detail_map[\"evaluation_ranges\"]:\n",
    "                print(8 * ' ', 30 * \"=\")\n",
    "                print(8 * ' ',phone_os, phone_label, r[\"trip_id\"], r[\"eval_common_trip_id\"], r[\"eval_role\"])\n",
    "                if \"control\" not in r[\"eval_role\"]:\n",
    "                    verify_range(r)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "verify_analysed_view(av_la_master)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def compare_ranges(eval_range_a, eval_range_b):\n",
    "    if len(eval_range_a[\"sensed_trip_ranges\"]) > len(eval_range_b[\"sensed_trip_ranges\"]):\n",
    "        print(f\"MISMATCH: extra trip found in range_a\")\n",
    "    if len(eval_range_a[\"sensed_trip_ranges\"]) < len(eval_range_b[\"sensed_trip_ranges\"]):\n",
    "        print(f\"MISMATCH: extra trip found in range_b\")\n",
    "    for i, (t_a, t_b) in enumerate(zip(eval_range_a[\"evaluation_trip_ranges\"], eval_range_b[\"evaluation_trip_ranges\"])):\n",
    "        if len(t_a[\"sensed_section_ranges\"]) > len(t_b[\"sensed_section_ranges\"]):\n",
    "            print(f\"MISMATCH: extra section found in range_a {len(t_a['sensed_section_ranges'])} > {len(t_b['sensed_section_ranges'])}\")\n",
    "        if len(t_a[\"sensed_section_ranges\"]) < len(t_b[\"sensed_section_ranges\"]):\n",
    "            print(f\"MISMATCH: extra section found in range_b {len(t_a['sensed_section_ranges'])} < {len(t_b['sensed_section_ranges'])}\")\n",
    "        for i, (s_a, s_b) in enumerate(zip(t_a[\"sensed_section_ranges\"], t_b[\"sensed_section_ranges\"])):\n",
    "            if s_a[\"sensed_mode\"] != s_b[\"sensed_mode\"]:\n",
    "                print(f\"MISMATCH: sensed_mode for section {i} {s_a['sensed_mode']} != {s_b['sensed_mode']}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def compare_analysed_views(av_a, av_b):\n",
    "    for phone_os, phone_map in av_a.map().items():\n",
    "        print(15 * \"=*\")\n",
    "        for phone_label, phone_detail_map in phone_map.items():\n",
    "            for i, r_a in enumerate(phone_detail_map[\"evaluation_ranges\"]):\n",
    "                print(8 * ' ', 30 * \"=\")\n",
    "                print(8 * ' ',phone_os, phone_label, r_a[\"trip_id\"], r_a[\"eval_common_trip_id\"], r_a[\"eval_role\"])\n",
    "                if \"control\" not in r_a[\"eval_role\"]:\n",
    "                    r_b = av_b.map()[phone_os][phone_label][\"evaluation_ranges\"][i]\n",
    "                    compare_ranges(r_a, r_b)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "compare_analysed_views(av_la_master, av_la_gis)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "av_ucb_master = eapv.create_analysed_view(pv_ucb, master_spec, \"analysis/recreated_location\", \"analysis/cleaned_trip\", \"analysis/inferred_section\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "av_ucb_gis = eapv.create_analysed_view(pv_ucb, gis_spec, \"analysis/recreated_location\", \"analysis/cleaned_trip\", \"analysis/inferred_section\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "compare_analysed_views(av_ucb_master, av_ucb_gis)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## There is only one missing trip; let's plot it to be sure"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "first_repetition = pv_la.map()[\"ios\"][\"ucb-sdb-ios-3\"][\"evaluation_ranges\"][0]; first_repetition[\"trip_id\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "missing_trip = first_repetition[\"evaluation_trip_ranges\"][0]; print(missing_trip[\"trip_id\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "missing_trip.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "missing_trip[\"location_df\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "missing_trip[\"battery_df\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "missing_trip[\"motion_activity_df\"].ts.apply(lambda ts: arrow.get(ts).to(\"America/Los_angeles\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Checking the other specs\n",
    "\n",
    "- Both of them have extra trips, no missing trips\n",
    "- They have both missing and extra sections, but nothing where the sensed sections are zero"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "verify_analysed_view(av_sj)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "verify_analysed_view(av_ucb)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}