Deploying to gh-pages from @ a23f7ee 🚀

feed.xml

@@ -1,4 +1,4 @@
-<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" xml:lang="en"><generator uri="https://jekyllrb.com/" version="4.3.2">Jekyll</generator><link href="https://shaoleiren.github.io/feed.xml" rel="self" type="application/atom+xml"/><link href="https://shaoleiren.github.io/" rel="alternate" type="text/html" hreflang="en"/><updated>2023-11-10T20:10:40+00:00</updated><id>https://shaoleiren.github.io/feed.xml</id><title type="html">blank</title><subtitle></subtitle><entry><title type="html">a post with bibliography</title><link href="https://shaoleiren.github.io/blog/2023/post-bibliography/" rel="alternate" type="text/html" title="a post with bibliography"/><published>2023-07-12T13:56:00+00:00</published><updated>2023-07-12T13:56:00+00:00</updated><id>https://shaoleiren.github.io/blog/2023/post-bibliography</id><content type="html" xml:base="https://shaoleiren.github.io/blog/2023/post-bibliography/"><![CDATA[<p>This post shows how to add bibliography to simple blog posts. If you would like something more academic, check the <a href="/blog/2021/distill/">distill style post</a>.</p>]]></content><author><name></name></author><category term="sample-posts"/><category term="formatting"/><category term="bib"/><summary type="html"><![CDATA[an example of a blog post with bibliography]]></summary></entry><entry><title type="html">a post with jupyter notebook</title><link href="https://shaoleiren.github.io/blog/2023/jupyter-notebook/" rel="alternate" type="text/html" title="a post with jupyter notebook"/><published>2023-07-04T12:57:00+00:00</published><updated>2023-07-04T12:57:00+00:00</updated><id>https://shaoleiren.github.io/blog/2023/jupyter-notebook</id><content type="html" xml:base="https://shaoleiren.github.io/blog/2023/jupyter-notebook/"><![CDATA[<p>To include a jupyter notebook in a post, you can use the following code:</p> <div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code>{::nomarkdown}
+<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" xml:lang="en"><generator uri="https://jekyllrb.com/" version="4.3.2">Jekyll</generator><link href="https://shaoleiren.github.io/feed.xml" rel="self" type="application/atom+xml"/><link href="https://shaoleiren.github.io/" rel="alternate" type="text/html" hreflang="en"/><updated>2023-11-16T20:55:07+00:00</updated><id>https://shaoleiren.github.io/feed.xml</id><title type="html">blank</title><subtitle></subtitle><entry><title type="html">a post with bibliography</title><link href="https://shaoleiren.github.io/blog/2023/post-bibliography/" rel="alternate" type="text/html" title="a post with bibliography"/><published>2023-07-12T13:56:00+00:00</published><updated>2023-07-12T13:56:00+00:00</updated><id>https://shaoleiren.github.io/blog/2023/post-bibliography</id><content type="html" xml:base="https://shaoleiren.github.io/blog/2023/post-bibliography/"><![CDATA[<p>This post shows how to add bibliography to simple blog posts. If you would like something more academic, check the <a href="/blog/2021/distill/">distill style post</a>.</p>]]></content><author><name></name></author><category term="sample-posts"/><category term="formatting"/><category term="bib"/><summary type="html"><![CDATA[an example of a blog post with bibliography]]></summary></entry><entry><title type="html">a post with jupyter notebook</title><link href="https://shaoleiren.github.io/blog/2023/jupyter-notebook/" rel="alternate" type="text/html" title="a post with jupyter notebook"/><published>2023-07-04T12:57:00+00:00</published><updated>2023-07-04T12:57:00+00:00</updated><id>https://shaoleiren.github.io/blog/2023/jupyter-notebook</id><content type="html" xml:base="https://shaoleiren.github.io/blog/2023/jupyter-notebook/"><![CDATA[<p>To include a jupyter notebook in a post, you can use the following code:</p> <div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code>{::nomarkdown}
 {% assign jupyter_path = "assets/jupyter/blog.ipynb" | relative_url %}
 {% capture notebook_exists %}{% file_exists assets/jupyter/blog.ipynb %}{% endcapture %}
 {% if notebook_exists == "true" %}

index.html

@@ -12,13 +12,13 @@
   <span class="na">month</span> <span class="p">=</span> <span class="s">{}</span><span class="p">,</span>
   <span class="na">year</span> <span class="p">=</span> <span class="s">{2023}</span><span class="p">,</span>
   <span class="na">url</span> <span class="p">=</span> <span class="s">{https://arxiv.org/abs/2307.05494}</span><span class="p">,</span>
-<span class="p">}</span></code></pre></figure> </div> </div> </div> </li> <li> <div class="row"> <div class="col-sm-2 abbr"><abbr class="badge">NeurIPS</abbr></div> <div id="Learning_AnytimeConstrainedRL_NeurIPS_2023" class="col-sm-10"> <div class="title"><a href="">Anytime-Competitive Reinforcement Learning with Policy Prior</a></div> <div class="author"> Jianyi Yang, Pengfei Li, Tongxin Li, Adam Wierman, and <em>Shaolei Ren</em> </div> <div class="periodical"> <em>NeurIPS</em>, 2023 </div> <div class="periodical"> </div> <div class="links"> <a class="abstract btn btn-sm z-depth-0" role="button">Abstract</a> <a class="bibtex btn btn-sm z-depth-0" role="button">Bib</a> <a href="" class="btn btn-sm z-depth-0" role="button">HTML</a> </div> <div class="badges"> </div> <div class="abstract hidden"> <p>This paper studies the problem of Anytime-Competitive Markov Decision Process (A-CMDP). Existing works on Constrained Markov Decision Processes (CMDPs) aim to optimize the expected reward while constraining the expected cost over random dynamics, but the cost in a specific episode can still be unsatisfactorily high. In contrast, the goal of A-CMDP is to optimize the expected reward while guaranteeing a bounded cost in each round of any episode against a policy prior. We propose a new algorithm, called Anytime-Competitive Reinforcement Learning (ACRL), which provably guarantees the anytime cost constraints. The regret analysis shows the policy asymptotically matches the optimal reward achievable under anytime constraints. Experiments on the application of carbon-intelligent computing verify the reward performance and cost constraint guarantee of ACRL.</p> </div> <div class="bibtex hidden"> <figure class="highlight"><pre><code class="language-bibtex" data-lang="bibtex"><span class="nc">@article</span><span class="p">{</span><span class="nl">Learning_AnytimeConstrainedRL_NeurIPS_2023</span><span class="p">,</span>
+<span class="p">}</span></code></pre></figure> </div> </div> </div> </li> <li> <div class="row"> <div class="col-sm-2 abbr"><abbr class="badge">NeurIPS</abbr></div> <div id="Learning_AnytimeConstrainedRL_NeurIPS_2023" class="col-sm-10"> <div class="title"><a href="https://arxiv.org/abs/2311.01568" rel="external nofollow noopener" target="_blank">Anytime-Competitive Reinforcement Learning with Policy Prior</a></div> <div class="author"> Jianyi Yang, Pengfei Li, Tongxin Li, Adam Wierman, and <em>Shaolei Ren</em> </div> <div class="periodical"> <em>NeurIPS</em>, 2023 </div> <div class="periodical"> </div> <div class="links"> <a class="abstract btn btn-sm z-depth-0" role="button">Abstract</a> <a class="bibtex btn btn-sm z-depth-0" role="button">Bib</a> <a href="https://arxiv.org/abs/2311.01568" class="btn btn-sm z-depth-0" role="button" rel="external nofollow noopener" target="_blank">HTML</a> </div> <div class="badges"> </div> <div class="abstract hidden"> <p>This paper studies the problem of Anytime-Competitive Markov Decision Process (A-CMDP). Existing works on Constrained Markov Decision Processes (CMDPs) aim to optimize the expected reward while constraining the expected cost over random dynamics, but the cost in a specific episode can still be unsatisfactorily high. In contrast, the goal of A-CMDP is to optimize the expected reward while guaranteeing a bounded cost in each round of any episode against a policy prior. We propose a new algorithm, called Anytime-Competitive Reinforcement Learning (ACRL), which provably guarantees the anytime cost constraints. The regret analysis shows the policy asymptotically matches the optimal reward achievable under anytime constraints. Experiments on the application of carbon-intelligent computing verify the reward performance and cost constraint guarantee of ACRL.</p> </div> <div class="bibtex hidden"> <figure class="highlight"><pre><code class="language-bibtex" data-lang="bibtex"><span class="nc">@article</span><span class="p">{</span><span class="nl">Learning_AnytimeConstrainedRL_NeurIPS_2023</span><span class="p">,</span>
   <span class="na">title</span> <span class="p">=</span> <span class="s">{Anytime-Competitive Reinforcement Learning with Policy Prior}</span><span class="p">,</span>
   <span class="na">author</span> <span class="p">=</span> <span class="s">{Yang, Jianyi and Li, Pengfei and Li, Tongxin and Wierman, Adam and Ren, Shaolei}</span><span class="p">,</span>
   <span class="na">journal</span> <span class="p">=</span> <span class="s">{NeurIPS}</span><span class="p">,</span>
   <span class="na">month</span> <span class="p">=</span> <span class="s">{}</span><span class="p">,</span>
   <span class="na">year</span> <span class="p">=</span> <span class="s">{2023}</span><span class="p">,</span>
-  <span class="na">url</span> <span class="p">=</span> <span class="s">{}</span><span class="p">,</span>
+  <span class="na">url</span> <span class="p">=</span> <span class="s">{https://arxiv.org/abs/2311.01568}</span><span class="p">,</span>
 <span class="p">}</span></code></pre></figure> </div> </div> </div> </li> <li> <div class="row"> <div class="col-sm-2 abbr"><abbr class="badge">NeurIPS</abbr></div> <div id="SOCO_RCL_NeurIPS_2023" class="col-sm-10"> <div class="title"><a href="https://arxiv.org/abs/2310.20098" rel="external nofollow noopener" target="_blank">Robust Learning for Smoothed Online Convex Optimization with Feedback Delay</a></div> <div class="author"> Pengfei Li, Jianyi Yang, Adam Wierman, and <em>Shaolei Ren</em> </div> <div class="periodical"> <em>NeurIPS</em>, 2023 </div> <div class="periodical"> </div> <div class="links"> <a class="abstract btn btn-sm z-depth-0" role="button">Abstract</a> <a class="bibtex btn btn-sm z-depth-0" role="button">Bib</a> <a href="https://arxiv.org/abs/2310.20098" class="btn btn-sm z-depth-0" role="button" rel="external nofollow noopener" target="_blank">HTML</a> </div> <div class="badges"> </div> <div class="abstract hidden"> <p>We study a general form of Smoothed Online Convex Optimization, a.k.a. SOCO, including multi-step switching costs and feedback delay. We propose a novel machine learning (ML) augmented online algorithm, Robustness-Constrained Learning (RCL), which combines untrusted ML predictions with a trusted expert online algorithm via constrained projection to robustify the ML prediction. Specifically, we prove that RCL is able to guarantee (1+lambda)-competitiveness against any given expert for any lambda&gt;0, while also explicitly training the ML model in a robustification-aware manner to improve the average-case performance. Importantly, RCL is the first ML-augmented algorithm with a provable robustness guarantee in the case of multi-step switching cost and feedback delay. We demonstrate the improvement of RCL in both robustness and average performance using battery management for electrifying transportation as a case study.</p> </div> <div class="bibtex hidden"> <figure class="highlight"><pre><code class="language-bibtex" data-lang="bibtex"><span class="nc">@article</span><span class="p">{</span><span class="nl">SOCO_RCL_NeurIPS_2023</span><span class="p">,</span>
   <span class="na">title</span> <span class="p">=</span> <span class="s">{Robust Learning for Smoothed Online Convex Optimization with Feedback Delay}</span><span class="p">,</span>
   <span class="na">author</span> <span class="p">=</span> <span class="s">{Li, Pengfei and Yang, Jianyi and Wierman, Adam and Ren, Shaolei}</span><span class="p">,</span>