deploy: b77513c

leap-stc · Jan 30, 2024 · 2188f76 · 2188f76
1 parent b8616b1
commit 2188f76
Show file tree

Hide file tree

Showing 16 changed files with 522 additions and 31 deletions.
diff --git a/README.html b/README.html
@@ -65,7 +65,7 @@
     <script>DOCUMENTATION_OPTIONS.pagename = 'README';</script>
     <link rel="index" title="Index" href="genindex.html" />
     <link rel="search" title="Search" href="search.html" />
-    <link rel="next" title="Quickstart" href="quickstart.html" />
+    <link rel="next" title="Motivation" href="motivation.html" />
   <meta name="viewport" content="width=device-width, initial-scale=1"/>
   <meta name="docsearch:language" content="en"/>
   </head>
@@ -166,6 +166,7 @@
             </li>
         </ul>
         <ul class="nav bd-sidenav">
+<li class="toctree-l1"><a class="reference internal" href="motivation.html">Motivation</a></li>
 <li class="toctree-l1"><a class="reference internal" href="quickstart.html">Quickstart</a></li>
 <li class="toctree-l1"><a class="reference internal" href="dataset.html">Dataset Information</a></li>
 <li class="toctree-l1"><a class="reference internal" href="task.html">Task Overview</a></li>
@@ -373,7 +374,6 @@ <h2> Contents </h2>
 <li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#getting-started">Getting Started</a></li>
 <li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#build-your-own-model">Build Your Own Model</a></li>
 <li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#benchmarking">Benchmarking</a></li>
-<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#motivation">Motivation</a></li>
 </ul>
             </nav>
         </div>
@@ -402,6 +402,7 @@ <h2>Features<a class="headerlink" href="#features" title="Permalink to this head
 <section id="getting-started">
 <h2>Getting Started<a class="headerlink" href="#getting-started" title="Permalink to this heading">#</a></h2>
 <ul class="simple">
+<li><p><a class="reference external" href="https://leap-stc.github.io/ChaosBench/motivation.html">Motivation</a></p></li>
 <li><p><a class="reference external" href="https://leap-stc.github.io/ChaosBench/quickstart.html">Quickstart</a></p></li>
 <li><p><a class="reference external" href="https://leap-stc.github.io/ChaosBench/dataset.html">Dataset Overview</a></p></li>
 <li><p><a class="reference external" href="https://leap-stc.github.io/ChaosBench/task.html">Task Overview</a></p></li>
@@ -421,13 +422,6 @@ <h2>Benchmarking<a class="headerlink" href="#benchmarking" title="Permalink to t
 <li><p><a class="reference external" href="https://leap-stc.github.io/ChaosBench/leaderboard.html">Leaderboard</a></p></li>
 </ul>
 </section>
-<section id="motivation">
-<h2>Motivation<a class="headerlink" href="#motivation" title="Permalink to this heading">#</a></h2>
-<p>1️⃣ <strong>Collapse to Climatology</strong>. Performing comparable or worse than climatology renders these state-of-the-art-models operationally unusable
-<img alt="Collapse" src="_images/all_rmse_sota.png" /></p>
-<p>2️⃣ <strong>Blurring Artifact</strong>. Averaged-out forecasts is of little use when one attempts to predict extreme events requiring high-fidelity on the S2S scale (e.g., droughts, hurricanes)
-<img alt="Blurring" src="_images/preds_climax_q700_direct_Task1.png" /></p>
-</section>
 <div class="toctree-wrapper compound">
 </div>
 </section>
@@ -463,11 +457,11 @@ <h2>Motivation<a class="headerlink" href="#motivation" title="Permalink to this
 
 <div class="prev-next-area">
     <a class="right-next"
-       href="quickstart.html"
+       href="motivation.html"
        title="next page">
       <div class="prev-next-info">
         <p class="prev-next-subtitle">next</p>
-        <p class="prev-next-title">Quickstart</p>
+        <p class="prev-next-title">Motivation</p>
       </div>
       <i class="fa-solid fa-angle-right"></i>
     </a>
@@ -491,7 +485,6 @@ <h2>Motivation<a class="headerlink" href="#motivation" title="Permalink to this
 <li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#getting-started">Getting Started</a></li>
 <li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#build-your-own-model">Build Your Own Model</a></li>
 <li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#benchmarking">Benchmarking</a></li>
-<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#motivation">Motivation</a></li>
 </ul>
   </nav></div>
 

diff --git a/_sources/README.md b/_sources/README.md
@@ -22,6 +22,7 @@ Dataset 🤗: https://huggingface.co/datasets/LEAP/ChaosBench
 4️⃣ __Large-Scale Benchmarking__. Systematic evaluation for state-of-the-art ML-based weather models like PanguWeather, FourcastNetV2, ViT/ClimaX, and Graphcast
 
 ## Getting Started
+- [Motivation](https://leap-stc.github.io/ChaosBench/motivation.html)
 - [Quickstart](https://leap-stc.github.io/ChaosBench/quickstart.html)
 - [Dataset Overview](https://leap-stc.github.io/ChaosBench/dataset.html)
 - [Task Overview](https://leap-stc.github.io/ChaosBench/task.html)
@@ -33,12 +34,4 @@ Dataset 🤗: https://huggingface.co/datasets/LEAP/ChaosBench
 
 ## Benchmarking
 - [Baseline Models](https://leap-stc.github.io/ChaosBench/baseline.html)
-- [Leaderboard](https://leap-stc.github.io/ChaosBench/leaderboard.html)
-
-
-## Motivation
-1️⃣ __Collapse to Climatology__. Performing comparable or worse than climatology renders these state-of-the-art-models operationally unusable
-![Collapse](docs/all_rmse_sota.png)
-
-2️⃣ __Blurring Artifact__. Averaged-out forecasts is of little use when one attempts to predict extreme events requiring high-fidelity on the S2S scale (e.g., droughts, hurricanes)
-![Blurring](docs/preds_climax_q700_direct_Task1.png)
+- [Leaderboard](https://leap-stc.github.io/ChaosBench/leaderboard.html)
diff --git a/_sources/motivation.md b/_sources/motivation.md
@@ -0,0 +1,8 @@
+# Motivation
+Our benchmark is one of the first to perform large-scale evaluation on existing state-of-the-art models, and finds methods originally developed for weather-scale applications _fails_ on S2S task, including:
+
+1️⃣ __Collapse to Climatology__. Performing comparable or worse than climatology renders these state-of-the-art-models operationally unusable
+![Collapse](../docs/all_rmse_sota.png)
+
+2️⃣ __Blurring Artifact__. Averaged-out forecasts is of little use when one attempts to predict extreme events requiring high-fidelity on the S2S scale (e.g., droughts, hurricanes)
+![Blurring](../docs/preds_climax_q700_direct_Task1.png)
diff --git a/_sources/quickstart.md b/_sources/quickstart.md
@@ -13,7 +13,7 @@ mkdir data
 **Step 4**: Initialize the space by running
 ```
 cd ChaosBench/data/
-wget https://huggingface.co/datasets/juannat7/ChaosBench/blob/main/process.sh
+wget https://huggingface.co/datasets/LEAP/ChaosBench/blob/main/process.sh
 chmod +x process.sh
 ```
 **Step 5**: Download the data 

diff --git a/baseline.html b/baseline.html
@@ -167,6 +167,7 @@
             </li>
         </ul>
         <ul class="current nav bd-sidenav">
+<li class="toctree-l1"><a class="reference internal" href="motivation.html">Motivation</a></li>
 <li class="toctree-l1"><a class="reference internal" href="quickstart.html">Quickstart</a></li>
 <li class="toctree-l1"><a class="reference internal" href="dataset.html">Dataset Information</a></li>
 <li class="toctree-l1"><a class="reference internal" href="task.html">Task Overview</a></li>

diff --git a/dataset.html b/dataset.html
@@ -167,6 +167,7 @@
             </li>
         </ul>
         <ul class="current nav bd-sidenav">
+<li class="toctree-l1"><a class="reference internal" href="motivation.html">Motivation</a></li>
 <li class="toctree-l1"><a class="reference internal" href="quickstart.html">Quickstart</a></li>
 <li class="toctree-l1 current active"><a class="current reference internal" href="#">Dataset Information</a></li>
 <li class="toctree-l1"><a class="reference internal" href="task.html">Task Overview</a></li>

diff --git a/evaluation.html b/evaluation.html
@@ -165,6 +165,7 @@
             </li>
         </ul>
         <ul class="current nav bd-sidenav">
+<li class="toctree-l1"><a class="reference internal" href="motivation.html">Motivation</a></li>
 <li class="toctree-l1"><a class="reference internal" href="quickstart.html">Quickstart</a></li>
 <li class="toctree-l1"><a class="reference internal" href="dataset.html">Dataset Information</a></li>
 <li class="toctree-l1"><a class="reference internal" href="task.html">Task Overview</a></li>

diff --git a/genindex.html b/genindex.html
@@ -164,6 +164,7 @@
             </li>
         </ul>
         <ul class="nav bd-sidenav">
+<li class="toctree-l1"><a class="reference internal" href="motivation.html">Motivation</a></li>
 <li class="toctree-l1"><a class="reference internal" href="quickstart.html">Quickstart</a></li>
 <li class="toctree-l1"><a class="reference internal" href="dataset.html">Dataset Information</a></li>
 <li class="toctree-l1"><a class="reference internal" href="task.html">Task Overview</a></li>

diff --git a/leaderboard.html b/leaderboard.html
@@ -164,6 +164,7 @@
             </li>
         </ul>
         <ul class="current nav bd-sidenav">
+<li class="toctree-l1"><a class="reference internal" href="motivation.html">Motivation</a></li>
 <li class="toctree-l1"><a class="reference internal" href="quickstart.html">Quickstart</a></li>
 <li class="toctree-l1"><a class="reference internal" href="dataset.html">Dataset Information</a></li>
 <li class="toctree-l1"><a class="reference internal" href="task.html">Task Overview</a></li>