2. Tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook will first explain how does `dataprep.connector` get the data using configs.\n",
    "\n",
    "After that, we will demonstrate how to create a new config to get data from the `earnings_calendar` API endpoint using `dataprep.connector` and do visualization on the data using `dataprep.eda`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Config"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`dataprep.connector` is a component that simplifies the API call. It first reads a config, which describes how to call a specific API, and then lets you make API calls using a simple function.\n",
    "\n",
    "An example config for downloading the [earnings_calendar](https://finnhub.io/docs/api/earnings-calendar) information from the [finnhub](https://finnhub.io/) website is already provided in the `finnhub` folder."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A config is a folder whose name is usually the website name. In our case, it's `finnhub`.\n",
    "\n",
    "The folder consists of at least two files, a `_meta.json` which describes the names of the endpoints under this website,\n",
    "and an API description file to describe **one** API endpoint information (if you want to describe different API endpoints under the same website, that will be multiple API description files).\n",
    "\n",
    "Currently, the API description file for the `countries` API endpoint is provided in the `finnhub/earnings_calendar.json` file."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### _meta.json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The following code outputs the content of the _meta.json\n",
    "!cat finnhub/_meta.json"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-03-05T02:25:45.771126Z",
     "iopub.status.busy": "2021-03-05T02:25:45.770477Z",
     "iopub.status.idle": "2021-03-05T02:25:45.782548Z",
     "shell.execute_reply": "2021-03-05T02:25:45.780641Z",
     "shell.execute_reply.started": "2021-03-05T02:25:45.771048Z"
    }
   },
   "source": [
    "First, take a look at the content of \"_meta.json\", it is a JSON object, which has only one field called \"tables\". Under \"tables\" is an array of strings, where each of them is the name of an API endpoint. Later in the hackathon, if you'd like to create a config for a website, don't forget to create the `_meta.json` file and add the endpoint names into the file.\n",
    "\n",
    "`dataprep.connector` will first read this file when you call `connect(\"finnhub\")` to know what are the names of all the API endpoints under this website, as well as reading all the API descriptions for each `name` from the `{name}.json` file. For example, here under the `tables` field, there's one element called `earnings_calendar`. As a result, `connect` will read a file called `earnings_calendar.json` in the same folder as `_meta.json`.\n",
    "\n",
    "You may notice here we name the the API endpoints as tables. This is because, for these endpoints, they return a bunch of records that resembles a database table."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-03-05T02:35:16.463846Z",
     "iopub.status.busy": "2021-03-05T02:35:16.462100Z",
     "iopub.status.idle": "2021-03-05T02:35:16.472395Z",
     "shell.execute_reply": "2021-03-05T02:35:16.470982Z",
     "shell.execute_reply.started": "2021-03-05T02:35:16.463720Z"
    }
   },
   "source": [
    "### API Description"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# The following code outputs the content of the countries.json\n",
    "!cat finnhub/earnings_calendar.json"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-03-05T02:36:06.670320Z",
     "iopub.status.busy": "2021-03-05T02:36:06.668724Z",
     "iopub.status.idle": "2021-03-05T02:36:06.686633Z",
     "shell.execute_reply": "2021-03-05T02:36:06.683547Z",
     "shell.execute_reply.started": "2021-03-05T02:36:06.670231Z"
    }
   },
   "source": [
    "As mentioned in the video, the API description file consists of three main fields:\n",
    "```json\n",
    "{\n",
    "    \"version\": 1,\n",
    "    \"request\": {},\n",
    "    \"response\": {}\n",
    "}\n",
    "\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-03-05T02:38:03.482910Z",
     "iopub.status.busy": "2021-03-05T02:38:03.482076Z",
     "iopub.status.idle": "2021-03-05T02:38:03.497050Z",
     "shell.execute_reply": "2021-03-05T02:38:03.495023Z",
     "shell.execute_reply.started": "2021-03-05T02:38:03.482832Z"
    }
   },
   "source": [
    "The `version` field indicates the version of this config file format, for now, it will always be 1.\n",
    "\n",
    "The `request` field is an object, which describes how to make a request to an endpoint. Specifically, it describes the endpoint URL, the HTTP method, and the parameters required.\n",
    "\n",
    "The `response` field is also an object, which describes how will connector parse the response from the API endpoint. It describes the fields the response contains, as well as the format of the response."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### API Description Explained"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-03-05T05:05:54.916550Z",
     "iopub.status.busy": "2021-03-05T05:05:54.915760Z",
     "iopub.status.idle": "2021-03-05T05:05:54.929543Z",
     "shell.execute_reply": "2021-03-05T05:05:54.926952Z",
     "shell.execute_reply.started": "2021-03-05T05:05:54.916471Z"
    }
   },
   "source": [
    "There are two fields in the API description requires some elaboration, the `request` and `response`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-03-05T20:49:59.179554Z",
     "iopub.status.busy": "2021-03-05T20:49:59.178544Z",
     "iopub.status.idle": "2021-03-05T20:49:59.190326Z",
     "shell.execute_reply": "2021-03-05T20:49:59.188098Z",
     "shell.execute_reply.started": "2021-03-05T20:49:59.179430Z"
    }
   },
   "source": [
    "#### Request"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```json\n",
    "\"request\": {\n",
    "    \"url\": \"https://finnhub.io/api/v1/calendar/earnings\",\n",
    "    \"method\": \"GET\",\n",
    "    \"authorization\": {\n",
    "        \"type\": \"QueryParam\",\n",
    "        \"keyParam\": \"token\"\n",
    "    },\n",
    "    \"params\": {\n",
    "        \"symbol\": false,\n",
    "        \"from_\": false,\n",
    "        \"from\": {\n",
    "            \"required\": false,\n",
    "            \"removeIfEmpty\": true,\n",
    "            \"template\": \"{{from_}}\",\n",
    "            \"fromKey\": [\n",
    "                \"from_\"\n",
    "            ]\n",
    "        },\n",
    "        \"to\": false,\n",
    "        \"international\": false\n",
    "    }\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The request field describes how to issue a request to the API endpoint. \n",
    "\n",
    "Usually you can get these information from the API documentation. For example, you can get the information for `earnings_calendar` [here](https://finnhub.io/docs/api/earnings-calendar).\n",
    "\n",
    "The request field at least should contain the following fields:\n",
    "    \n",
    "1. `url` field, which indicates the location of the endpoint.\n",
    "2. `method` field, which indicates the HTTP method to issue the request. Usually, this will be `GET`.\n",
    "3. `authorization` field. This field is optional and is only required if the website needs some form of authorization. For example, \n",
    "   finnhub requires the `QueryParam` authorization type. Essentially this means the API key should be put in the URL as a query parameter. The name of the parameter is\n",
    "   indicated using `keyParam` field. In our case, it is called `token.\n",
    "4. `params` defines all the parameters the API request required. They can be mandatory or optional. The simplest form of `params` is `\"parameter name\": required or not`. \n",
    "   For example, `\"symbol\": false` means the endpoint takes a parameter called `symbol`, and it is optional.\n",
    "   A more complex form is like this\n",
    "   ```json\n",
    "    \"from\": {\n",
    "        \"required\": false,\n",
    "        \"removeIfEmpty\": true,\n",
    "        \"template\": \"{{from_}}\",\n",
    "        \"fromKey\": [\n",
    "            \"from_\"\n",
    "        ]\n",
    "    },\n",
    "    ```\n",
    "    which basically says that there's a parameter called from, it is not required, and it should be removed if the value of it is empty. It is a template parameter, where it takes the value of another parameter called `from_` (remind that `from` is the Python keyword so we cannot use it as the variable name. That's why we define the query parameter `from` gets the value from `from_`). `fromKey` indicates `from` is dependent from another parameter called `from_`.\n",
    "  \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Response"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following is the `response` field (some fields are elided for easy reading)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```json\n",
    "\"response\": {\n",
    "    \"ctype\": \"application/json\",\n",
    "    \"tablePath\": \"$.earningsCalendar[*]\",\n",
    "    \"schema\": {\n",
    "        \"date\": {\n",
    "            \"target\": \"$.date\",\n",
    "            \"type\": \"string\"\n",
    "        },\n",
    "        \"epsActual\": {\n",
    "            \"target\": \"$.epsActual\",\n",
    "            \"type\": \"float\"\n",
    "        },\n",
    "        ...\n",
    "    },\n",
    "    \"orient\": \"records\"\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Response field describes how to parse data from the response of the endpoint."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are four subfields required:\n",
    "\n",
    "1. `ctype` indicates the API response format. Keep this as `application/json` since usually, a website returns JSON as the response.\n",
    "2. `orient`, `tablePath` and `schema` require some understanding of the response. For example, from the finnhub [documentation](https://finnhub.io/docs/api/earnings-calendar), the response of \n",
    "   earnings calendar looks like this:\n",
    "   ```json\n",
    "   {\n",
    "      \"earningsCalendar\": [\n",
    "        {\n",
    "          \"date\": \"2020-01-28\",\n",
    "          \"epsActual\": 4.99,\n",
    "          \"epsEstimate\": 4.5474,\n",
    "          \"hour\": \"amc\",\n",
    "          \"quarter\": 1,\n",
    "          \"revenueActual\": 91819000000,\n",
    "          \"revenueEstimate\": 88496400810,\n",
    "          \"symbol\": \"AAPL\",\n",
    "          \"year\": 2020\n",
    "        },\n",
    "        {\n",
    "          \"date\": \"2019-10-30\",\n",
    "          \"epsActual\": 3.03,\n",
    "          \"epsEstimate\": 2.8393,\n",
    "          \"hour\": \"amc\",\n",
    "          \"quarter\": 4,\n",
    "          \"revenueActual\": 64040000000,\n",
    "          \"revenueEstimate\": 62985161760,\n",
    "          \"symbol\": \"AAPL\",\n",
    "          \"year\": 2019\n",
    "        }\n",
    "       ]\n",
    "    }\n",
    "   ```\n",
    "   We see that the response is a JSON object, which contains a single field `earningsCalendar`. Under this field, it is an array of objects, and each object is a record.\n",
    "   The way that uses an array of objects to represent the content is called `records`. As a result, the `orient` field should have the value `records`.\n",
    "   \n",
    "3. Next, since the array of objects is under the key `earningsCalendar`, we use `\"tablePath\": \"$.earningsCalendar[*]\"` to indicate the location of the data records. \n",
    "   The syntax of `$.earningsCalendar[*]` is called JSONPath. You can find the detail at https://github.com/h2non/jsonpath-ng.\n",
    " \n",
    "4. Finally, we write down the description of each field under the `schema` field, with their name and the JSONPath with respect to the `tablePath`. \n",
    "   For example, for the `date` field in the response we create an object under the `schema` field in the endpoint description as\n",
    "   ```json\n",
    "   \"date\": {\n",
    "        \"target\": \"$.date\",\n",
    "        \"type\": \"string\"\n",
    "    },\n",
    "   ```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now you must have a good understanding on the config file format. \n",
    "\n",
    "Next we are going to show an example workflow for EZHacks based on the `earnings_calendar` API.\n",
    "\n",
    "Specifically, we will do a step by step demostration on how to create a config and get data from the `earnings_calendar` API, and then do a visualization."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Creating a new config file"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Suppose during the hackathon, you want to do some investigation on investing US stock market.\n",
    "As the first step, you'd like to see the revenue trend of some tech companies, for example, TESLA."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Plan"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Following is the plan to finish the task:\n",
    "\n",
    "1. Find a website that provides financial data API.\n",
    "2. Use `dataprep.connector` to download the data.\n",
    "3. Use `dataprep.eda` to visualize the data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Finnhub"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-03-05T01:55:05.149360Z",
     "iopub.status.busy": "2021-03-05T01:55:05.148501Z",
     "iopub.status.idle": "2021-03-05T01:55:05.163071Z",
     "shell.execute_reply": "2021-03-05T01:55:05.160509Z",
     "shell.execute_reply.started": "2021-03-05T01:55:05.149254Z"
    }
   },
   "source": [
    "After some investigation, you decide to use the [Finnhub](https://finnhub.io/) website, where they provide\n",
    "API access to fetch the financial related data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The most related one is the [earnings calendar](https://finnhub.io/docs/api/earnings-calendar) API, where it provides the quarterly revenue data of a company."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-03-05T01:57:53.716518Z",
     "iopub.status.busy": "2021-03-05T01:57:53.715749Z",
     "iopub.status.idle": "2021-03-05T01:57:53.729763Z",
     "shell.execute_reply": "2021-03-05T01:57:53.726681Z",
     "shell.execute_reply.started": "2021-03-05T01:57:53.716441Z"
    }
   },
   "source": [
    "Reading through their [documentation](https://finnhub.io/docs/api/authentication), you notice the first thing you need to do is creating an account\n",
    "because all the API calls to finnhub require an API key."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once registered, go to the [dashboard](https://finnhub.io/dashboard) and write down the API key."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-03-05T04:14:58.488509Z",
     "iopub.status.busy": "2021-03-05T04:14:58.485861Z",
     "iopub.status.idle": "2021-03-05T04:14:58.498851Z",
     "shell.execute_reply": "2021-03-05T04:14:58.496538Z",
     "shell.execute_reply.started": "2021-03-05T04:14:58.488359Z"
    }
   },
   "source": [
    "### Create a config for the earnings calendar endpoint"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first thing to do is adding the endpoint name \"earnings_calendar\" to the `_meta.json`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The content of `_meta.json` will become this:\n",
    "```json\n",
    "{\n",
    "    \"tables\": [\n",
    "        \"earnings_calendar\"\n",
    "    ]\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "\n",
    "# The following code adds the content above to finnhub/_meta.json\n",
    "\n",
    "cat > finnhub/_meta.json << EOF\n",
    "{\n",
    "    \"tables\": [\n",
    "        \"earnings_calendar\"\n",
    "    ]\n",
    "}\n",
    "EOF"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, create a new file called `earnings_calendar.json`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-03-05T04:22:08.314372Z",
     "iopub.status.busy": "2021-03-05T04:22:08.313513Z",
     "iopub.status.idle": "2021-03-05T04:22:08.329357Z",
     "shell.execute_reply": "2021-03-05T04:22:08.326332Z",
     "shell.execute_reply.started": "2021-03-05T04:22:08.314294Z"
    }
   },
   "source": [
    "Put the following content into `earnings_calendar.json`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-03-05T04:49:18.338876Z",
     "iopub.status.busy": "2021-03-05T04:49:18.337233Z",
     "iopub.status.idle": "2021-03-05T04:49:18.361178Z",
     "shell.execute_reply": "2021-03-05T04:49:18.359004Z",
     "shell.execute_reply.started": "2021-03-05T04:49:18.338795Z"
    }
   },
   "source": [
    "```json\n",
    "{\n",
    "    \"version\": 1,\n",
    "    \"request\": {\n",
    "        \"url\": \"https://finnhub.io/api/v1/calendar/earnings\",\n",
    "        \"method\": \"GET\",\n",
    "        \"authorization\": {\n",
    "            \"type\": \"QueryParam\",\n",
    "            \"keyParam\": \"token\"\n",
    "        },\n",
    "        \"params\": {\n",
    "            \"symbol\": false,\n",
    "            \"from_\": false,\n",
    "            \"from\": {\n",
    "                \"required\": false,\n",
    "                \"removeIfEmpty\": true,\n",
    "                \"template\": \"{{from_}}\",\n",
    "                \"fromKey\": [\n",
    "                    \"from_\"\n",
    "                ]\n",
    "            },\n",
    "            \"to\": false,\n",
    "            \"international\": false\n",
    "        }\n",
    "    },\n",
    "    \"response\": {\n",
    "        \"ctype\": \"application/json\",\n",
    "        \"tablePath\": \"$.earningsCalendar[*]\",\n",
    "        \"schema\": {\n",
    "            \"date\": {\n",
    "                \"target\": \"$.date\",\n",
    "                \"type\": \"string\"\n",
    "            },\n",
    "            \"epsActual\": {\n",
    "                \"target\": \"$.epsActual\",\n",
    "                \"type\": \"float\"\n",
    "            },\n",
    "            \"epsEstimate\": {\n",
    "                \"target\": \"$.epsEstimate\",\n",
    "                \"type\": \"float\"\n",
    "            },\n",
    "            \"hour\": {\n",
    "                \"target\": \"$.hour\",\n",
    "                \"type\": \"string\"\n",
    "            },\n",
    "            \"quarter\": {\n",
    "                \"target\": \"$.quarter\",\n",
    "                \"type\": \"int\"\n",
    "            },\n",
    "            \"revenueActual\": {\n",
    "                \"target\": \"$.revenueActual\",\n",
    "                \"type\": \"float\"\n",
    "            },\n",
    "            \"revenueEstimate\": {\n",
    "                \"target\": \"$.revenueEstimate\",\n",
    "                \"type\": \"float\"\n",
    "            },\n",
    "            \"symbol\": {\n",
    "                \"target\": \"$.symbol\",\n",
    "                \"type\": \"string\"\n",
    "            },\n",
    "            \"year\": {\n",
    "                \"target\": \"$.year\",\n",
    "                \"type\": \"int\"\n",
    "            }\n",
    "        },\n",
    "        \"orient\": \"records\"\n",
    "    }\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "\n",
    "# The following code adds the content above to finnhub/earnings_calendar.json\n",
    "\n",
    "cat > finnhub/earnings_calendar.json << EOF\n",
    "{\n",
    "    \"version\": 1,\n",
    "    \"request\": {\n",
    "        \"url\": \"https://finnhub.io/api/v1/calendar/earnings\",\n",
    "        \"method\": \"GET\",\n",
    "        \"authorization\": {\n",
    "            \"type\": \"QueryParam\",\n",
    "            \"keyParam\": \"token\"\n",
    "        },\n",
    "        \"params\": {\n",
    "            \"symbol\": false,\n",
    "            \"from_\": false,\n",
    "            \"from\": {\n",
    "                \"required\": false,\n",
    "                \"removeIfEmpty\": true,\n",
    "                \"template\": \"{{from_}}\",\n",
    "                \"fromKey\": [\n",
    "                    \"from_\"\n",
    "                ]\n",
    "            },\n",
    "            \"to\": false,\n",
    "            \"international\": false\n",
    "        }\n",
    "    },\n",
    "    \"response\": {\n",
    "        \"ctype\": \"application/json\",\n",
    "        \"tablePath\": \"$.earningsCalendar[*]\",\n",
    "        \"schema\": {\n",
    "            \"date\": {\n",
    "                \"target\": \"$.date\",\n",
    "                \"type\": \"string\"\n",
    "            },\n",
    "            \"epsActual\": {\n",
    "                \"target\": \"$.epsActual\",\n",
    "                \"type\": \"float\"\n",
    "            },\n",
    "            \"epsEstimate\": {\n",
    "                \"target\": \"$.epsEstimate\",\n",
    "                \"type\": \"float\"\n",
    "            },\n",
    "            \"hour\": {\n",
    "                \"target\": \"$.hour\",\n",
    "                \"type\": \"string\"\n",
    "            },\n",
    "            \"quarter\": {\n",
    "                \"target\": \"$.quarter\",\n",
    "                \"type\": \"int\"\n",
    "            },\n",
    "            \"revenueActual\": {\n",
    "                \"target\": \"$.revenueActual\",\n",
    "                \"type\": \"float\"\n",
    "            },\n",
    "            \"revenueEstimate\": {\n",
    "                \"target\": \"$.revenueEstimate\",\n",
    "                \"type\": \"float\"\n",
    "            },\n",
    "            \"symbol\": {\n",
    "                \"target\": \"$.symbol\",\n",
    "                \"type\": \"string\"\n",
    "            },\n",
    "            \"year\": {\n",
    "                \"target\": \"$.year\",\n",
    "                \"type\": \"int\"\n",
    "            }\n",
    "        },\n",
    "        \"orient\": \"records\"\n",
    "    }\n",
    "}\n",
    "EOF"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now run the following code"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# Import related functions\n",
    "from dataprep.connector import connect\n",
    "from datetime import datetime, timedelta, timezone\n",
    "\n",
    "# connect to finnhub, the connector will automatically read the config from the finnhub folder. The access token is the API key you just acquired from the finnhub dashboard.\n",
    "conn_finnhub = connect(\"./finnhub\", _auth={\"access_token\":\"put your finnhub API token here\"})\n",
    "\n",
    "# issue a query. The query consists of two things, the API endpoint name (earnings_calendar) and the parameters. From the finnhub documentation https://finnhub.io/docs/api/earnings-calendar\n",
    "# we know that earnings_calendar takes four parameters, but now we only care about three of them (symbol, from, to). Since from is a python keyword, we rename it to from_.\n",
    "# the await keyword is required because we are using the python async/await feature to issue HTTP requests.\n",
    "tesla = await conn_finnhub.query('earnings_calendar', symbol='TSLA', from_=datetime(2011,1,1), to=datetime.now())\n",
    "\n",
    "# Voila! the API response is parsed into a dataframe\n",
    "tesla"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plot and Inspect the Result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since we are interested in the trend of the revenue of Tesla, we can use the `dataprep.eda` module to visualize the `year` and the `revenueActual` column. (Want to know where do these two column names come from? Check the DataFrame header of the previous cell!)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# this will import the plot function from the eda module.\n",
    "from dataprep.eda import plot\n",
    "\n",
    "# The plot function takes at least one argument, which is a DataFrme, i.e. `plot(tesla)`. This will give you an overview of each column of the DataFrame.\n",
    "# It can also take two or three arguments. For the following three-argument call, `plot` will give you a plot showing the relationship of two columns, which are passed as\n",
    "# the second and the third parameter.\n",
    "\n",
    "plot(tesla, \"year\", \"revenueActual\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion from the plot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By inspecting the plot, you found that the revenue of Tesla is increasing year by year, which indicates a potentially good investment target. \n",
    "However, if you take a look at the box plot, the variance of the recent year becomes bigger and bigger. \n",
    "You may think the variance of the company is too much, so you decide to do other investigations before making a final decision."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "dataprep",
   "language": "python",
   "name": "dataprep"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}