desc-tevelop.tex

%!TEX root = paper.tex
\subsection{Time Series Evolution Operator ({\tt tevelop})}

Time series capture the variation of values against time, and common to a a number of scientific problems. Time series can be multiple dimensions. For example geospatial datasets are two-dimensional series, based both on time and spatial position. One of the common tasks when dealing with time series is the ability to predict or forecast them in advance.  Such a task is considerably easier if the underlying time series has a clear evolution structure across dimensions. For example, if the evolution structure can be established on the spatial aspects (i.e.  there is a  strong correlation between nearby spatial points), estimating the evolution becomes relatively easier. The problem chosen is termed as a spatial bag where there is spatial variation, but it is not clearly linked to the geometric distance
between spatial regions. In contrast, traffic-related time series have a
strong spatial structure. As such, identifying the evolution in time series is a common problem across a number of domains. This particular benchmark focuses on extracting the evolution, using earthquake as the driving example. We summarize the key features of the benchmark  in Table~\ref{tab:eq-summary}.

\begin{small}
\begin{table}
\caption{Summary of the {\tt tevelop} Benchmark}\label{tab:eq-summary}
%\resizebox{1.0\textwidth}{!}{
\begin{center}
\begin{tabular}{p{0.4\columnwidth}p{0.6\columnwidth}}
\hline
\hline
{\bf Description} & Earthquake Forecasting~\cite{fox2022-jm,TFT-21,eq-code,eq-data}.\\
\hline
{\bf Objectives} &  Improve the quality of Earthquake
forecasting in a region of Southern California.\\
\hline
{\bf Metrics} & Normalized Nash-Sutcliffe model efficiency coefficient (NNSE)with $0.8\leq NNSE\leq 0.99$\\
\hline
{\bf Data}  & Type:  Richter Measurements with spatial and temporal information (Events). \\
  &  Input:  Earthquakes since 1950.\\
  &  Size:  11.3GB (Uncompressed), 21.3MB (Compressed)\\
  & Training samples: 2,400 spatial bins\\
  & Validation samples:   100 spatial bins\\
  & Source:  USGS Servers~\cite{eq-data}\\
\hline
{\bf Reference Implementation} & \cite{eq-code}\\
% \hline
\hline
\end{tabular}
\end{center}
%}
\end{table}
\end{small}

\noindent {\bf Benchmarking Objectives and Metrics:} The scientific objective is to extract the evolution of a time series, exemplified using earthquake forecasting. To make the benchmarking exercise more focused, this forecasting is done on a subset of the overall earthquake dataset for the region of  Southern California. Conventional methods for forecasting relies on statistical techniques. Here, the aim is to use ML for not only extracting the evolution, but also to test the effectiveness using forecasting. The exact scientific metric for quantifying the benefit of the forecasting is the Nash Sutcliffe Efficiency (NSE) \cite{nash-79}. It is also possible to qualitatively asses prediction by comparing the observed earthquake, if one desires, but the benchmarks relies on the former~\cite{fox2022-jm}.

\smallskip 

\noindent {\bf Data:} The benchmark relies on a very small subset of the earthquake data from United States Geological Survey (USGS) focused between the regions of Southern California (latitude: $32^\circ$N to $36^\circ$N, longitude: $-120^\circ$S to $-114^\circ$S). The subset of the data for this region covers all earthquakes in that region since 1950. There are four measurements per record, namely,  magnitude, spatial location, depth from the crust, and time. The curated dataset is organized to cover this in different temporal and spatial bins. Although the actual time lapse between measurements is one day, we accumulate this into a fortnightly data. The region is then divided into a grid of  $40\times 60$ with each each pixel covering actual zone of $0.1\deg\times 0.1$ or $11km\times 11km$ grid. The dataset also includes an assignment of pixels to known faults, and a list of the largest earthquakes in that region from 1950. We have chosen various samplings of the dataset to provide both
input and predicted values. These include time ranges from a fortnight
up to four years. Furthermore, we calculate summed magnitudes and depths and
counts of significant quakes (magnitude $< 3.29$). 

\smallskip 

\noindent {\bf Reference Implementation:} The benchmark includes three distinct deep learning-based reference implementations. These are Long short-term memory (LSTM)-based model, Google Temporal Fusion Transformer (TFT)~\cite{TFT-21}-based model, and a custom hybrid transformer model. The TFT-based model uses  two distinct LSTMs, covering a an encoder a decoder with a temporal attention-based transformer. The custom model includes a space-time transformer for the Decoder and a two-layer LSTM for the encoder. Each model predicts NSE and generates visualizations illustrating the TFT for interpretable multi-horizon time series forecasting~\cite{TFT-21}. Details of the current reference models can be found in~\cite{fox2022-jm}.