A JavaScript (ES6 ECMAScript) module for forecast visualization.
To use the component in your app, you'll need to add the following to your HTML:
- add
<script>and stylesheet<link>tags for dependencies to the<head> - add a
<div>to the<body>for the component to fill - add a
<script>in the<body>that initializes the component via the (default)Appobject that's exported
See the "HTML file example" section below for an example, and see the "JavaScript API" section for how to initialize the component.
In your HTML file, load the required CSS and JavaScript files:
- In the
<head>, load Bootstrap 4 and other required library files:
<!-- Bootstrap 4 -->
<script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.11.0/umd/popper.js"></script>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.12.4/jquery.min.js"></script>
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/[email protected]/dist/css/bootstrap.min.css">
<script src="https://stackpath.bootstrapcdn.com/bootstrap/4.4.1/js/bootstrap.min.js"></script>
<!-- https://www.daterangepicker.com/ -->
<script type="text/javascript" src="https://cdn.jsdelivr.net/momentjs/latest/moment.min.js"></script>
<script type="text/javascript" src="https://cdn.jsdelivr.net/npm/daterangepicker/daterangepicker.min.js"></script>
<link rel="stylesheet" type="text/css" href="https://cdn.jsdelivr.net/npm/daterangepicker/daterangepicker.css"/>
<!-- predtimechart -->
<script src="https://cdn.plot.ly/plotly-2.12.1.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/moment.js/2.29.3/moment.min.js"></script>- In the
<body>, add the row<div>that will hold the component:
<div id="forecastViz_row" class="row"></div>- In the
<body>, load and use the predtimechart module:
<script type="module">
// import the module's entry point (the `App` object)
import App from 'https://cdn.jsdelivr.net/gh/reichlab/[email protected]/dist/predtimechart.js';
// set up _fetchData, _calcUemForecasts (optional), and options
function _fetchData(isForecast, targetKey, taskIDs, referenceDate) { ... }
function _calcUemForecasts(componentModels, targetKey, referenceDate, userModelName) { ... }
const options = {...};
// initialize the component: componentDiv, _fetchData, isIndicateRedraw, options, _calcUemForecasts
App.initialize('forecastViz_row', _fetchData, false, options, _calcUemForecasts);
</script>The component is accessed via the App object, and is initialized via the App.initialize() function. After that, everything else is taken care of by the app.
Predtimechart has adopted a simplified version of the modeling tasks concept defined in Modeling Hub task ID variables, and specifically the idea of task ID variables. These variables are used in these ways (details follow):
- Initializing the application's UI: The
task_idsobject passed to App.initialize() via the containing options object (see the "Options object" section below) configures the dropdown menus in the options section of the UI. Specifically, each task ID gets its own dropdown, positioned between the fixed Outcome and Interval dropdowns. For example, if the options object'stask_idsspecifies two task IDs ("scenario_id" and "location") then there will be two dropdowns between Outcome and Interval: "scenario_id" and "location". Those dropdowns' values will contain the values passed in thetask_idsobject. - Initializing the dropdown default values: Predtimechart uses the options object's
initial_task_idsto select the initial values that the user sees. - Args to the
_fetchDatafunction: When the user takes an action that requires obtaining data, predtimechart will pass the currently-selected values in the above dropdowns to the_fetchDatafunction.
App.initialize(componentDiv, _fetchData, isIndicateRedraw, options, _calcUemForecasts) takes these args:
componentDiv:idof the empty<div>to place the component into._fetchData: function to retrieve truth and forecast data. It is called whenever the plot needs updating. See the "fetchData data format" section for details. It takes these args:isForecast:booleanindicating type of data to retrieve.truemeans retrieve forecast data andfalsemeans get truth data.targetKey:stringnaming the target of interest. Must be one of the values in the options object'starget_variablesvalue.taskIDs:objectspecifying which modeling task to retrieve data for (see task ID variables above). Must be consistent with theoptionsobject'stask_idsvalue (see the "Options object" section for details).referenceDate: "" reference date "". Must be one of the values in the options object'savailable_as_ofsvalue.
isIndicateRedraw:booleanthat controls whether the plot area should be grayed out while waiting for data requests. Useful for applications that have a noticeable delay when fetching data.options:objectthat contains initialization data. See the "Options object" section for details.calcUemForecasts: optional human judgement ensemble model function. pass null to disable the feature. It takes these args:componentModels: an array of model names, a subset of those in theoptionsobject (see below).targetKey: same as passed to_fetchDataabove.referenceDate: ""userModelName: The model's displayed name. Must be a valid name, e.g., no spaces, commas, etc.
The component is initialized by a JavaScript object with the following keys and values. See the "Example options object" for a detailed example. See src/schema.json for the options object's JSON Schema.
available_as_ofs:objectthat mapstarget_variablesvalueto anarrayof dates in 'YYYY-MM-DD' format that have truth and/or forecasts availablecurrent_date:available_as_ofsvaluekey to use for the initial plotdisclaimer:stringproviding any important information users should knowinitial_as_of:stringspecifying the initial date from 'available_as_ofs' (in 'YYYY-MM-DD' format) to use for the initially-selected as_of dateinitial_checked_models:modelsvalue(s) to use for the initial plotinitial_interval:intervalsvalue to use for the initial plotinitial_target_var:target_variablesvaluekey to use for the initial plotinitial_task_ids: anobjectto use for the initial plot. Its format is identical to_fetchData()'staskIDsarg above.initial_xaxis_range:arrayof two dates in 'YYYY-MM-DD' format that specify the initial xaxis range to use. To not initialize the range, passnullfor its valueinitial_yaxis_range:arrayof two values (format depends on outcome variable) that specify the initial yaxis range to use. To not initialize the range, passnullfor its valueintervals:arrayof one or more integers between 0 and 100 inclusive, representing percentages (purpose: TBD)models:arrayof model names (strings) that provide datatarget_variables:arrayofobjects defining the target variables in the data. Each object contains three keys:- 'value': used as the main value that's passed around for the target
- 'text': human-readable text
- 'plot_text': plot text (purpose: TBD)
task_ids:objectdefining the tasks in the data as described in_fetchData'staskIDsarg above. The object contains astringkey for each task ID variable, the value of which is anarrayofobjects defining possible values. The keys should consist of only ASCII letters, digits,_, and-, and should start with a letter. (Note that, for display, the IDs are split on_and-and the title-cased.) The objects have two keys:value: used as the main value that's passed around for the task IDtext: human-readable text
Here's a real-world example from the COVID-19 Forecast Hub project. Only the first two items in the lists are shown.
{
"available_as_ofs": {
"day_ahead_cumulative_deaths": [
"2020-03-15",
"2020-03-22",
"..."
],
"day_ahead_incident_deaths": [
"2020-03-15",
"2020-03-22",
"..."
],
"...": "..."
},
"current_date": "2022-10-22",
"disclaimer": "Most forecasts have failed to reliably predict rapid changes in the trends of reported cases and hospitalizations...",
"initial_as_of": "2022-10-12",
"initial_checked_models": [
"COVIDhub-baseline",
"COVIDhub-ensemble"
],
"initial_interval": "95%",
"initial_target_var": "week_ahead_incident_deaths",
"initial_task_ids": {"unit": "48"},
"initial_xaxis_range": null,
"initial_yaxis_range": null,
"intervals": ["0%", "50%", "95%"],
"models": [
"COVIDhub-baseline",
"COVIDhub-ensemble",
"..."
],
"target_variables": [
{
"value": "day_ahead_cumulative_deaths",
"text": "day ahead cumulative deaths",
"plot_text": "day ahead cumulative deaths"
},
{
"value": "day_ahead_incident_deaths",
"text": "day ahead incident deaths",
"plot_text": "day ahead incident deaths"
},
"..."
],
"task_ids": {
"unit": [
{"value": "48", "text": "Texas"},
{"value": "US", "text": "US"},
"..."
]
}
}As described above, the fetchData(isForecast, targetKey, unitAbbrev, referenceDate) function passed to App.initialize() is responsible for returning truth and forecast data as directed by the isForecast arg. It uses the other three args to retrieve and return the requested data. The data is in the following formats.
Truth data is represented as an object with x/y pairs represented as columns, where x=date and y=truth_value. The dates must correspond to those in the options object's available_as_ofs. For example:
{
"date": ["2020-03-15", "2020-03-22", "..."],
"y": [0, 15, "..."]
}Forecast data is an object with one entry for each model in the options object's models, each of which is in turn an object with entries for target end date of the forecast and the quantiles required to use to display point predictions and 50% or 95% prediction intervals. For example:
{
"UChicagoCHATTOPADHYAY-UnIT": {
"target_end_date": ["2021-09-11", "2021-09-18"],
"q0.025": [1150165.71, 1176055.78],
"q0.25": [1151044.42, 1178626.67],
"q0.5": [1151438.21, 1179605.9],
"q0.75": [1152121.55, 1180758.16],
"q0.975": [1152907.55, 1182505.14]
},
"USC-SI_kJalpha": {
"target_end_date": ["2021-09-11", "2021-09-18"],
"q0.025": [941239.7761, 775112.557],
"q0.25": [1010616.1863, 896160.705],
"q0.5": [1149400.162, 1137280.4614],
"q0.75": [1313447.0159, 1461013.716],
"q0.975": [1456851.692, 1771312.0932]
},
"...": "..."
}Following is how to do development-related activities. You must first install the required Node.js packages via npm install --save-dev.
We use QUnit for our unit tests. To run the tests, execute the package.json test script: npm run test. You should see output at the bottom like this:
...
# pass 22
# skip 0
# todo 0
# fail 0We've included src/index.html as a simple example of the app in action. The file hard-codes truth and forecast data, and has ata for only one referenceDate, but can be useful during development. To use it, follow the below packaging step, serve dist/index.html from your development environment (trying to open it directly from the filesystem will cause a CORS error), and then view it in your browser.
We use Ajv to validate App.initialize()'s options object. More specifically we use a standalone validation function generated from the schema, which is saved to schema-validator.cjs. This file must be regenerated whenever the schema file /src/schema.json changes. To do so, execute the package.json ajv_compile script via npm run ajv_compile.
We use webpack to package up all dependencies into a single dist/predtimechart.bundle.js file for end users. To do so, execute the package.json build script via npm run build, which will update all files in dist/.
TBC
To jump to a specific "as of" date (i.e., reference date), use the calendar icon in the plot's modebar, which is located in the top right of the plot. This strip of gray icons is shown by hovering over the chart. (See the below screenshot for an example.) Clicking the icon will pop up a date picker where you can navigate to a date and then click "Apply" to set it as the current "as of" date. Note that the year select box above calendar is constrained to the current target's data date range. Because not every calendar data has a corresponding "as of" date, picking a date that corresponds to the currently-set "as of" date will not update the display.
Predtimechart includes an optionally-enabled beta feature that supports creating an ensemble forecast file based on the existing models initialized in the app. These input "component models" are used by the human judgement ensemble model code to calculate forecasts based on those models' data. For now the arithmetic mean is used, but future versions will support other calculations. Note that the feature must be enabled by the predtimechart developer to be available. Otherwise, the "Actions" dropdown menu documented below will not be present.
All interactions with the feature take place using the "Actions" dropdown menu next to the "Shuffle Colors" button in the left sidebar. (Note that you will not see that dropdown if the feature is disabled.) The actions are:
- Add User Ensemble (always enabled): Creates a new model or replaces the existing one using the currently-selected models as components. Visually, the model works like any other model (shows in the model list and the plot, can be checked/unchecked, re-calculated when changing outcome/unit/reference date, etc.)
- Remove User Ensemble (enabled if model exists): Removes the model from model list.
- Download User Ensemble CSV (enabled ""): Calls the _
calcUemForecastsfunction passed toApp.initialize()(see above) compute and download a cross-unit forecast. The CSV file format is documented here. - User Ensemble Info... (enabled if model exists): Shows details about the model: The model's name, its component models, and the last error encountered trying to compute the forecasts, if any.
- Edit User Ensemble Model Name... (enabled if model does not exist): Shows a dialog box for renaming the user model from the default. Note: You can only edit the model name when there's no current user ensemble model added. To rename you must first do "Remove User Ensemble" if you've previously added one.
- Help...: Takes you to this page.
A typical Human judgement ensemble model workflow is:
- Explore the existing models' forecasts to decide which might make good component models.
- Select those component models in the models list in the left sidebar under "Select Models".
- Select the "Add User Ensemble" item in the "Actions" dropdown menu to add the "User Ensemble" model to the app.
- Explore the new model's forecasts and either move to the next step if you're happy with them, or repeat the above steps to try other component models.
- Select "Download User Ensemble CSV" in the "Actions" dropdown menu. This will contact the server, which will compute a forecasts for all units in the system, create a CSV file, and send it back to the app. When done, you will see a notification that the file was downloaded and saved.