Then $Y^{(i)}(t)$ models the $i$-th derivative of $y(t)$. In this section, we consider choices relating to the "diffusion"$\textcolor{#4063D8}{\Gamma}$. If you're more interested in the drift matrix$\textcolor{#389826}{A}$ check out the Priors section, and for info on the initial distribution $\textcolor{purple}{ \mathcal{N} \left( \mu_0, \Sigma_0 \right) }$ check out the Initialization section.
Time-fixed, isotropic diffusion, which is (optionally) quasi-maximum-likelihood-estimated.
This is the recommended diffusion when using fixed steps.
By default with calibrate=true, all covariances are re-scaled at the end of the solve with the MLE diffusion. Set calibrate=false to skip this step, e.g. when setting the initial_diffusion and then estimating the diffusion outside of the solver (e.g. with Fenrir.jl).
A multi-variate version of DynamicDiffusion, where instead of an isotropic matrix, a diagonal matrix is estimated. This can be helpful to get more expressive posterior covariances when using the EK0, since the individual dimensions can be adjusted separately.
A multi-variate version of FixedDiffusion, where instead of an isotropic matrix, a diagonal matrix is estimated. This can be helpful to get more expressive posterior covariances when using the EK0, since the individual dimensions can be adjusted separately.
[1] N. Bosch, P. Hennig, F. Tronarp: Calibrated Adaptive Probabilistic ODE Solvers (2021) (link)
Settings
This document was generated with Documenter.jl version 0.27.25 on Tuesday 4 July 2023. Using Julia version 1.9.1.
+\end{aligned}\]
Then $Y^{(i)}(t)$ models the $i$-th derivative of $y(t)$. In this section, we consider choices relating to the "diffusion"$\textcolor{#4063D8}{\Gamma}$. If you're more interested in the drift matrix$\textcolor{#389826}{A}$ check out the Priors section, and for info on the initial distribution $\textcolor{purple}{ \mathcal{N} \left( \mu_0, \Sigma_0 \right) }$ check out the Initialization section.
Time-fixed, isotropic diffusion, which is (optionally) quasi-maximum-likelihood-estimated.
This is the recommended diffusion when using fixed steps.
By default with calibrate=true, all covariances are re-scaled at the end of the solve with the MLE diffusion. Set calibrate=false to skip this step, e.g. when setting the initial_diffusion and then estimating the diffusion outside of the solver (e.g. with Fenrir.jl).
A multi-variate version of DynamicDiffusion, where instead of an isotropic matrix, a diagonal matrix is estimated. This can be helpful to get more expressive posterior covariances when using the EK0, since the individual dimensions can be adjusted separately.
A multi-variate version of FixedDiffusion, where instead of an isotropic matrix, a diagonal matrix is estimated. This can be helpful to get more expressive posterior covariances when using the EK0, since the individual dimensions can be adjusted separately.
In-place and square-root implementation of predict which saves the result into x_out.
Only works with PSDMatrices.PSDMatrix types as Ah, Qh, and in the covariances of x_curr and x_out (both of type Gaussian). To prevent allocations, a cache matrix cachemat of size $D \times 2D$ (where $D \times D$ is the size of Ah and Qh) needs to be passed.
In-place and square-root implementation of predict which saves the result into x_out.
Only works with PSDMatrices.PSDMatrix types as Ah, Qh, and in the covariances of x_curr and x_out (both of type Gaussian). To prevent allocations, a cache matrix cachemat of size $D \times 2D$ (where $D \times D$ is the size of Ah and Qh) needs to be passed.
Update step in Kalman filtering for linear dynamics models.
Given a Gaussian $x = \mathcal{N}(μ, Σ)$ and a measurement $z = \mathcal{N}(\hat{z}, S)$, with $S = H Σ H^T$, compute
\[\begin{aligned}
K &= Σ^P H^T S^{-1}, \\
μ^F &= μ + K (0 - \hat{z}), \\
Σ^F &= Σ - K S K^T,
-\end{aligned}\]
and return an updated state \mathcal{N}(μ^F, Σ^F). Note that this assumes zero-measurements. When called with ProbNumDiffEq.SquarerootMatrix type arguments it performs the update in Joseph / square-root form.
For better performance, we recommend to use the non-allocating update!.
In-place and square-root implementation of update which saves the result into x_out.
Implemented in Joseph Form to retain the PSDMatrix covariances:
\[\begin{aligned}
+\end{aligned}\]
and return an updated state \mathcal{N}(μ^F, Σ^F). Note that this assumes zero-measurements. When called with ProbNumDiffEq.SquarerootMatrix type arguments it performs the update in Joseph / square-root form.
For better performance, we recommend to use the non-allocating update!.
Update step of the Kalman smoother, aka. Rauch-Tung-Striebel smoother, for linear dynamics models.
Given Gaussians $x_n = \mathcal{N}(μ_{n}, Σ_{n})$ and $x_{n+1} = \mathcal{N}(μ_{n+1}^S, Σ_{n+1}^S)$, compute
\[\begin{aligned}
μ_{n+1}^P &= A μ_n^F, \\
P_{n+1}^P &= A Σ_n^F A + Q, \\
G &= Σ_n^S A^T (Σ_{n+1}^P)^{-1}, \\
μ_n^S &= μ_n^F + G (μ_{n+1}^S - μ_{n+1}^P), \\
Σ_n^S &= (I - G A) Σ_n^F (I - G A)^T + G Q G^T + G Σ_{n+1}^S G^T,
-\end{aligned}\]
and return a smoothed state \mathcal{N}(μ_n^S, Σ_n^S). When called with ProbNumDiffEq.SquarerootMatrix type arguments it performs the update in Joseph / square-root form.
For better performance, we recommend to use the non-allocating smooth!.
This document was generated with Documenter.jl version 0.27.25 on Tuesday 4 July 2023. Using Julia version 1.9.1.
+\end{aligned}\]
and return a smoothed state \mathcal{N}(μ_n^S, Σ_n^S). When called with ProbNumDiffEq.SquarerootMatrix type arguments it performs the update in Joseph / square-root form.
For better performance, we recommend to use the non-allocating smooth!.
ProbNumDiffEq.jl builds directly on OrdinaryDiffEq.jl to benefit from its iterator interface, flexible step-size control, and efficient Jacobian calculations. But, this requires extending non-public APIs. This page is meant to provide an overview on which parts exactly ProbNumDiffEq.jl builds on.
For more discussion on the pros and cons of building on OrdinaryDiffEq.jl, see this thread on discourse.
./src/caches.jl implements the cache and its main constructor: OrdinaryDiffEq.alg_cache
Initialization and perform_step!: via OrdinaryDiffEq.initialize! and OrdinaryDiffEq.perform_step!. Implemented in ./src/perform_step.jl.
Custom postamble by overloading OrdinaryDiffEq.postamble! (which should always call OrdinaryDiffEq._postamble!). This is where we do the "smoothing" of the solution. Implemented in ./src/integrator_utils.jl.
Custom saving by overloading OrdinaryDiffEq.savevalues! (which should always call OrdinaryDiffEq._savevalues!). Implemented in ./src/integrator_utils.jl.
DiffEqBase.__init is currently overloaded to transform OOP problems into IIP problems (in ./src/solve.jl).
The solution object:ProbODESolution <: AbstractProbODESolution <: DiffEqBase.AbstractODESolution
./src/solution.jl implements the main parts. Note that the main constructor DiffEqBase.build_solution is called by OrdinaryDiffEq.__init - so OrdinaryDiffEq.jl has control over its inputs.
There is also MeanProbODESolution <: DiffEqBase.AbstractODESolution: It allows handling the mean of a probabilistic ODE solution the same way one would handle any "standard" ODE solution - e.g. it is compatible with DiffEqDevTools.appxtrue.
AbstractODEFilterPosterior <: DiffEqBase.AbstractDiffEqInterpolation is the current interpolant, but it does not actually fully handle the interpolation right now. This part might be subject to change soon.
Plot recipe in ./src/solution_plotting.jl
Sampling in ./src/solution_sampling.jl
DiffEqBase.prepare_alg(::EK1{0}); closely follows a similar function implemented in OrdinaryDiffEq.jl ./src/alg_utils.jl
DiffEqDevTools.appxtrue is overloaded to work with ProbODESolution (by just doing mean(sol)). This also enables DiffEqDevTools.WorkPrecision to work out of th box.
Settings
This document was generated with Documenter.jl version 0.27.25 on Tuesday 4 July 2023. Using Julia version 1.9.1.
+Implementation via OrdinaryDiffEq.jl · ProbNumDiffEq.jl
ProbNumDiffEq.jl builds directly on OrdinaryDiffEq.jl to benefit from its iterator interface, flexible step-size control, and efficient Jacobian calculations. But, this requires extending non-public APIs. This page is meant to provide an overview on which parts exactly ProbNumDiffEq.jl builds on.
For more discussion on the pros and cons of building on OrdinaryDiffEq.jl, see this thread on discourse.
./src/caches.jl implements the cache and its main constructor: OrdinaryDiffEq.alg_cache
Initialization and perform_step!: via OrdinaryDiffEq.initialize! and OrdinaryDiffEq.perform_step!. Implemented in ./src/perform_step.jl.
Custom postamble by overloading OrdinaryDiffEq.postamble! (which should always call OrdinaryDiffEq._postamble!). This is where we do the "smoothing" of the solution. Implemented in ./src/integrator_utils.jl.
Custom saving by overloading OrdinaryDiffEq.savevalues! (which should always call OrdinaryDiffEq._savevalues!). Implemented in ./src/integrator_utils.jl.
DiffEqBase.__init is currently overloaded to transform OOP problems into IIP problems (in ./src/solve.jl).
The solution object:ProbODESolution <: AbstractProbODESolution <: DiffEqBase.AbstractODESolution
./src/solution.jl implements the main parts. Note that the main constructor DiffEqBase.build_solution is called by OrdinaryDiffEq.__init - so OrdinaryDiffEq.jl has control over its inputs.
There is also MeanProbODESolution <: DiffEqBase.AbstractODESolution: It allows handling the mean of a probabilistic ODE solution the same way one would handle any "standard" ODE solution - e.g. it is compatible with DiffEqDevTools.appxtrue.
AbstractODEFilterPosterior <: DiffEqBase.AbstractDiffEqInterpolation is the current interpolant, but it does not actually fully handle the interpolation right now. This part might be subject to change soon.
Plot recipe in ./src/solution_plotting.jl
Sampling in ./src/solution_sampling.jl
DiffEqBase.prepare_alg(::EK1{0}); closely follows a similar function implemented in OrdinaryDiffEq.jl ./src/alg_utils.jl
DiffEqDevTools.appxtrue is overloaded to work with ProbODESolution (by just doing mean(sol)). This also enables DiffEqDevTools.WorkPrecision to work out of th box.
Settings
This document was generated with Documenter.jl version 0.27.25 on Tuesday 1 August 2023. Using Julia version 1.9.2.
ProbNumDiffEq.jl provides probabilistic numerical solvers to the DifferentialEquations.jl ecosystem. The implemented ODE filters solve differential equations via Bayesian filtering and smoothing and compute not just a single point estimate of the true solution, but a posterior distribution that contains an estimate of its numerical approximation error.
probdiffeq: Fast and feature-rich filtering-based probabilistic ODE solvers in JAX.
ProbNum: Probabilistic numerics in Python. It has not only probabilistic ODE solvers, but also probabilistic linear solvers, Bayesian quadrature, and many filtering and smoothing implementations.
Fenrir.jl: Parameter-inference in ODEs with probabilistic ODE solvers. This package builds on ProbNumDiffEq.jl to provide a negative marginal log-likelihood function, which can then be used with an optimizer or with MCMC for parameter inference.
Settings
This document was generated with Documenter.jl version 0.27.25 on Tuesday 4 July 2023. Using Julia version 1.9.1.
ProbNumDiffEq.jl provides probabilistic numerical solvers to the DifferentialEquations.jl ecosystem. The implemented ODE filters solve differential equations via Bayesian filtering and smoothing and compute not just a single point estimate of the true solution, but a posterior distribution that contains an estimate of its numerical approximation error.
probdiffeq: Fast and feature-rich filtering-based probabilistic ODE solvers in JAX.
ProbNum: Probabilistic numerics in Python. It has not only probabilistic ODE solvers, but also probabilistic linear solvers, Bayesian quadrature, and many filtering and smoothing implementations.
Fenrir.jl: Parameter-inference in ODEs with probabilistic ODE solvers. This package builds on ProbNumDiffEq.jl to provide a negative marginal log-likelihood function, which can then be used with an optimizer or with MCMC for parameter inference.
Settings
This document was generated with Documenter.jl version 0.27.25 on Tuesday 1 August 2023. Using Julia version 1.9.2.
It turns out that we can also compute higher-order derivatives of $y$ with the chain rule, and then use these to better initialize $Y^{(i)}(0)$. This, done efficiently with Taylor-mode autodiff by using TaylorIntegration.jl, is what TaylorModeInit does. See also [1].
In the vast majority of cases, just stick to the exact Taylor-mode initialization TaylorModeInit!
Exact initialization via Taylor-mode automatic differentiation.
This is the recommended initialization method!
It uses TaylorIntegration.jl to efficiently compute the higher-order derivatives of the solution at the initial value, via Taylor-mode automatic differentiation.
In some special cases it can happen that TaylorIntegration.jl is incompatible with the given problem (typically because the problem definition does not allow for elements of type Taylor). If this happens, try ClassicSolverInit.
References
[1] N. Krämer, P. Hennig: Stable Implementation of Probabilistic ODE Solvers (2020)
Initialization via regression on a few steps of a classic ODE solver.
In a nutshell, instead of specifying $\mu_0$ exactly and setting $\Sigma_0=0$ (which is what TaylorModeInit does), use a classic ODE solver to compute a few steps of the solution, and then regress on the computed values (by running a smoother) to compute $\mu_0$ and $\Sigma_0$ as the mean and covariance of the smoothing posterior at time 0. See also [2].
The initial value and derivative are set directly from the given initial value problem; optionally the second derivative can also be set via automatic differentiation by setting init_on_ddu=true.
Arguments
alg: The solver to be used. Can be any solver from OrdinaryDiffEq.jl.
init_on_ddu: If true, the second derivative is also initialized exactly via automatic differentiation with ForwardDiff.jl.
References
[2] M. Schober, S. Särkkä, and P. Hennig: A Probabilistic Model for the Numerical Solution of Initial Value Problems (2018)
[1] N. Krämer, P. Hennig: Stable Implementation of Probabilistic ODE Solvers (2020) (link) [2] M. Schober, S. Särkkä, and P. Hennig: A Probabilistic Model for the Numerical Solution of Initial Value Problems (2018) (link)
Settings
This document was generated with Documenter.jl version 0.27.25 on Tuesday 4 July 2023. Using Julia version 1.9.1.
+\end{aligned}\]
It turns out that we can also compute higher-order derivatives of $y$ with the chain rule, and then use these to better initialize $Y^{(i)}(0)$. This, done efficiently with Taylor-mode autodiff by using TaylorIntegration.jl, is what TaylorModeInit does. See also [1].
In the vast majority of cases, just stick to the exact Taylor-mode initialization TaylorModeInit!
Exact initialization via Taylor-mode automatic differentiation.
This is the recommended initialization method!
It uses TaylorIntegration.jl to efficiently compute the higher-order derivatives of the solution at the initial value, via Taylor-mode automatic differentiation.
In some special cases it can happen that TaylorIntegration.jl is incompatible with the given problem (typically because the problem definition does not allow for elements of type Taylor). If this happens, try ClassicSolverInit.
References
[4] Krämer et al, "Stable Implementation of Probabilistic ODE Solvers" (2020)
Initialization via regression on a few steps of a classic ODE solver.
In a nutshell, instead of specifying $\mu_0$ exactly and setting $\Sigma_0=0$ (which is what TaylorModeInit does), use a classic ODE solver to compute a few steps of the solution, and then regress on the computed values (by running a smoother) to compute $\mu_0$ and $\Sigma_0$ as the mean and covariance of the smoothing posterior at time 0. See also [2].
The initial value and derivative are set directly from the given initial value problem; optionally the second derivative can also be set via automatic differentiation by setting init_on_ddu=true.
Arguments
alg: The solver to be used. Can be any solver from OrdinaryDiffEq.jl.
init_on_ddu: If true, the second derivative is also initialized exactly via automatic differentiation with ForwardDiff.jl.
References
[4] Krämer et al, "Stable Implementation of Probabilistic ODE Solvers" (2020)
[5] Schober et al, "A probabilistic model for the numerical solution of initial value problems", Statistics and Computing (2019)
Then $Y^{(i)}(t)$ models the $i$-th derivative of $y(t)$. In this section, we consider choices relating to the drift matrix$\textcolor{#389826}{A}$. If you're more interested in the diffusion$\textcolor{#4063D8}{\Gamma}$ check out the Diffusion models and calibration section, and for info on the initial distribution $\textcolor{purple}{ \mathcal{N} \left( \mu_0, \Sigma_0 \right) }$ check out the Initialization section.
If you're unsure which prior to use, just stick to the integrated Wiener process prior IWP! This is also the default choice for all solvers. The other priors are rather experimental / niche at the time of writing.
This is the recommended prior! It is the most well-tested prior, both in this package and in the probabilistic numerics literature in general (see the references). It is also the prior that has the most efficient implementation.
The IWP can be created without specifying the dimension of the Wiener process, in which case it will be inferred from the dimension of the ODE during the solve. This is typically the preferred usage.
In math
\[\begin{aligned}
+\end{aligned}\]
Then $Y^{(i)}(t)$ models the $i$-th derivative of $y(t)$. In this section, we consider choices relating to the drift matrix$\textcolor{#389826}{A}$. If you're more interested in the diffusion$\textcolor{#4063D8}{\Gamma}$ check out the Diffusion models and calibration section, and for info on the initial distribution $\textcolor{purple}{ \mathcal{N} \left( \mu_0, \Sigma_0 \right) }$ check out the Initialization section.
If you're unsure which prior to use, just stick to the integrated Wiener process prior IWP! This is also the default choice for all solvers. The other priors are rather experimental / niche at the time of writing.
This is the recommended prior! It is the most well-tested prior, both in this package and in the probabilistic numerics literature in general (see the references). It is also the prior that has the most efficient implementation.
The IWP can be created without specifying the dimension of the Wiener process, in which case it will be inferred from the dimension of the ODE during the solve. This is typically the preferred usage.
As with the IWP, the IOUP can be created without specifying its dimension, in which case it will be inferred from the dimension of the ODE during the solve. This is typically the preferred usage. The rate parameter however always needs to be specified.
As with the IWP, the Matern can be created without specifying its dimension, in which case it will be inferred from the dimension of the ODE during the solve. This is typically the preferred usage. The lengthscale parameter however always needs to be specified.
F. Tronarp, H. Kersting, S. Särkkä and P. Hennig. Probabilistic solutions to ordinary differential equations as
+ nonlinear Bayesian filtering: a new perspective. Statistics and Computing 29, 1297-1315 (2019).
This document was generated with Documenter.jl version 0.27.25 on Tuesday 1 August 2023. Using Julia version 1.9.2.
diff --git a/dev/refs.bib b/dev/refs.bib
new file mode 100644
index 000000000..136453adc
--- /dev/null
+++ b/dev/refs.bib
@@ -0,0 +1,172 @@
+@misc{bosch23expint,
+ title = {Probabilistic Exponential Integrators},
+ author = {Nathanael Bosch and Philipp Hennig and Filip Tronarp},
+ year = 2023,
+ eprint = {2305.14978},
+ archivePrefix ={arXiv},
+ primaryClass = {math.NA}
+}
+
+@InProceedings{krämer21highdim,
+ title = {Probabilistic {ODE} Solutions in Millions of Dimensions},
+ author = {Krämer, Nicholas and Bosch, Nathanael and Schmidt, Jonathan
+ and Hennig, Philipp},
+ booktitle = {Proceedings of the 39th International Conference on Machine
+ Learning},
+ pages = {11634--11649},
+ year = 2022,
+ editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and
+ Szepesvari, Csaba and Niu, Gang and Sabato, Sivan},
+ volume = 162,
+ series = {Proceedings of Machine Learning Research},
+ month = {17--23 Jul},
+ publisher = {PMLR},
+ pdf = {https://proceedings.mlr.press/v162/kramer22b/kramer22b.pdf},
+ url = {https://proceedings.mlr.press/v162/kramer22b.html},
+}
+
+@InProceedings{bosch22pickandmix,
+ title = {Pick-and-Mix Information Operators for Probabilistic {ODE}
+ Solvers},
+ author = {Bosch, Nathanael and Tronarp, Filip and Hennig, Philipp},
+ booktitle = {Proceedings of The 25th International Conference on Artificial
+ Intelligence and Statistics},
+ pages = {10015--10027},
+ year = 2022,
+ editor = {Camps-Valls, Gustau and Ruiz, Francisco J. R. and Valera,
+ Isabel},
+ volume = 151,
+ series = {Proceedings of Machine Learning Research},
+ month = {28--30 Mar},
+ publisher = {PMLR},
+ pdf = {https://proceedings.mlr.press/v151/bosch22a/bosch22a.pdf},
+ url = {https://proceedings.mlr.press/v151/bosch22a.html},
+}
+
+@InProceedings{bosch20capos,
+ title = {Calibrated Adaptive Probabilistic {ODE} Solvers},
+ author = {Bosch, Nathanael and Hennig, Philipp and Tronarp, Filip},
+ booktitle = {Proceedings of The 24th International Conference on Artificial
+ Intelligence and Statistics},
+ pages = {3466--3474},
+ year = 2021,
+ editor = {Banerjee, Arindam and Fukumizu, Kenji},
+ volume = 130,
+ series = {Proceedings of Machine Learning Research},
+ month = {13--15 Apr},
+ publisher = {PMLR},
+ pdf = {http://proceedings.mlr.press/v130/bosch21a/bosch21a.pdf},
+ url = {http://proceedings.mlr.press/v130/bosch21a.html},
+}
+
+@article{tronarp20bayes,
+ author = {Tronarp, Filip and Särkkä, Simo and Hennig, Philipp},
+ title = {Bayesian {ODE} solvers: the maximum a posteriori estimate},
+ journal = {Statistics and Computing},
+ year = 2021,
+ month = {Mar},
+ day = 03,
+ volume = 31,
+ number = 3,
+ pages = 23,
+ issn = {1573-1375},
+ doi = {10.1007/s11222-021-09993-7},
+ url = {https://doi.org/10.1007/s11222-021-09993-7}
+}
+
+@article{tronarp18probsol,
+ author = {Filip Tronarp and Hans Kersting and Simo Särkkä and Philipp
+ Hennig},
+ title = {Probabilistic solutions to ordinary differential equations as
+ nonlinear {B}ayesian filtering: a new perspective},
+ year = 2019,
+ volume = 29,
+ number = 6,
+ pages = {1297-1315},
+ doi = {10.1007/s11222-019-09900-1},
+ url = {https://doi.org/10.1007/s11222-019-09900-1},
+ journal = {Statistics and Computing},
+}
+
+@article{kraemer20stableimplementation,
+ author = {Krämer, Nicholas and Hennig, Philipp},
+ title = {Stable Implementation of Probabilistic {ODE} Solvers},
+ journal = {CoRR},
+ year = 2020,
+ url = {http://arxiv.org/abs/2012.10106v1},
+ archivePrefix ={arXiv},
+ eprint = {2012.10106},
+ primaryClass = {stat.ML},
+}
+
+@article{kersting18convergencerates,
+ author = {Kersting, Hans and Sullivan, T. J. and Hennig, Philipp},
+ title = {Convergence rates of {G}aussian {ODE} filters},
+ journal = {Statistics and Computing},
+ year = 2020,
+ month = {Nov},
+ day = 01,
+ volume = 30,
+ number = 6,
+ pages = {1791-1816},
+ issn = {1573-1375},
+ doi = {10.1007/s11222-020-09972-4},
+ url = {https://doi.org/10.1007/s11222-020-09972-4}
+}
+
+@article{schober16probivp,
+ author = "Schober, Michael and Särkkä, Simo and Hennig, Philipp",
+ title = "A probabilistic model for the numerical solution of initial
+ value problems",
+ journal = "Statistics and Computing",
+ year = 2019,
+ month = "Jan",
+ day = 01,
+ volume = 29,
+ number = 1,
+ pages = "99--122",
+ issn = "1573-1375",
+ doi = "10.1007/s11222-017-9798-7",
+ url = "https://doi.org/10.1007/s11222-017-9798-7"
+}
+
+@InProceedings{tronarp22fenrir,
+ title = {Fenrir: Physics-Enhanced Regression for Initial Value
+ Problems},
+ author = {Tronarp, Filip and Bosch, Nathanael and Hennig, Philipp},
+ booktitle = {Proceedings of the 39th International Conference on Machine
+ Learning},
+ pages = {21776--21794},
+ year = 2022,
+ editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and
+ Szepesvari, Csaba and Niu, Gang and Sabato, Sivan},
+ volume = 162,
+ series = {Proceedings of Machine Learning Research},
+ month = {17--23 Jul},
+ publisher = {PMLR},
+ pdf = {https://proceedings.mlr.press/v162/tronarp22a/tronarp22a.pdf},
+ url = {https://proceedings.mlr.press/v162/tronarp22a.html},
+}
+
+@inproceedings{kersting16active,
+ author = {Kersting, Hans and Hennig, Philipp},
+ title = {Active Uncertainty Calibration in Bayesian ODE Solvers},
+ year = 2016,
+ isbn = 9780996643115,
+ publisher = {AUAI Press},
+ booktitle = {Proceedings of the Thirty-Second Conference on Uncertainty in
+ Artificial Intelligence},
+ pages = {309–318},
+ numpages = 10,
+ series = {UAI'16},
+ url = {http://www.auai.org/uai2016/proceedings/papers/163.pdf},
+}
+
+@book{hennig22probnum,
+ place = {Cambridge},
+ title = {Probabilistic Numerics: Computation as Machine Learning},
+ DOI = {10.1017/9781316681411},
+ publisher = {Cambridge University Press},
+ author = {Hennig, Philipp and Osborne, Michael A. and Kersting, Hans P.},
+ year = 2022,
+}
diff --git a/dev/search/index.html b/dev/search/index.html
index 6684db678..d470552c9 100644
--- a/dev/search/index.html
+++ b/dev/search/index.html
@@ -1,2 +1,2 @@
-Search · ProbNumDiffEq.jl
Loading search...
Settings
This document was generated with Documenter.jl version 0.27.25 on Tuesday 4 July 2023. Using Julia version 1.9.1.
+Search · ProbNumDiffEq.jl
Loading search...
Settings
This document was generated with Documenter.jl version 0.27.25 on Tuesday 1 August 2023. Using Julia version 1.9.2.
diff --git a/dev/search_index.js b/dev/search_index.js
index 001901e71..56deb6e72 100644
--- a/dev/search_index.js
+++ b/dev/search_index.js
@@ -1,3 +1,3 @@
var documenterSearchIndex = {"docs":
-[{"location":"benchmarks/vanderpol/#Van-der-Pol-benchmark","page":"Stiff ODE: Van der Pol","title":"Van der Pol benchmark","text":"","category":"section"},{"location":"benchmarks/vanderpol/","page":"Stiff ODE: Van der Pol","title":"Stiff ODE: Van der Pol","text":"using LinearAlgebra, Statistics, InteractiveUtils\nusing DiffEqDevTools, ParameterizedFunctions, SciMLBase, OrdinaryDiffEq, Plots\nusing ProbNumDiffEq\n\n# Plotting theme\ntheme(:dao;\n markerstrokewidth=0.5,\n legend=:outertopright,\n bottom_margin=5Plots.mm,\n size = (1000, 400),\n)","category":"page"},{"location":"benchmarks/vanderpol/#Van-der-Pol-problem-definition","page":"Stiff ODE: Van der Pol","title":"Van der Pol problem definition","text":"","category":"section"},{"location":"benchmarks/vanderpol/","page":"Stiff ODE: Van der Pol","title":"Stiff ODE: Van der Pol","text":"function vanderpol!(du, u, p, t)\n du[1] = u[2]\n du[2] = p[1] * ((1 - u[1]^2) * u[2] - u[1])\nend\np = [1e5]\ntspan = (0.0, 6.3)\nu0 = [2.0, 0.0]\nprob = ODEProblem(vanderpol!, u0, tspan, p)\n\ntest_sol = solve(prob, RadauIIA5(), abstol=1/10^14, reltol=1/10^14, dense=false)\nplot(test_sol, title=\"Van der Pol Solution\", legend=false, ylims=(-2.5, 2.5))","category":"page"},{"location":"benchmarks/vanderpol/","page":"Stiff ODE: Van der Pol","title":"Stiff ODE: Van der Pol","text":"(Image: )","category":"page"},{"location":"benchmarks/vanderpol/#EK1-accross-orders","page":"Stiff ODE: Van der Pol","title":"EK1 accross orders","text":"","category":"section"},{"location":"benchmarks/vanderpol/","page":"Stiff ODE: Van der Pol","title":"Stiff ODE: Van der Pol","text":"DENSE = false;\nSAVE_EVERYSTEP = false;\n\n_setups = [\n \"EK1($order)\" => Dict(:alg => EK1(order=order, smooth=DENSE))\n for order in 2:6\n]\n\nlabels = first.(_setups)\nsetups = last.(_setups)\n\nabstols = 1.0 ./ 10.0 .^ (6:13)\nreltols = 1.0 ./ 10.0 .^ (3:10)\n\nwp = WorkPrecisionSet(\n prob, abstols, reltols, setups;\n names = labels,\n #print_names = true,\n appxsol = test_sol,\n dense = DENSE,\n save_everystep = SAVE_EVERYSTEP,\n numruns = 10,\n maxiters = Int(1e7),\n timeseries_errors = false,\n verbose = false,\n)\n\nplot(wp, palette=Plots.palette([:blue, :red], length(_setups)), xticks = 10.0 .^ (-16:1:5))","category":"page"},{"location":"benchmarks/vanderpol/","page":"Stiff ODE: Van der Pol","title":"Stiff ODE: Van der Pol","text":"(Image: )","category":"page"},{"location":"benchmarks/vanderpol/#Solving-the-first-vs-second-order-ODE","page":"Stiff ODE: Van der Pol","title":"Solving the first- vs second-order ODE","text":"","category":"section"},{"location":"benchmarks/vanderpol/","page":"Stiff ODE: Van der Pol","title":"Stiff ODE: Van der Pol","text":"function vanderpol2!(ddu, du, u, p, t)\n ddu[1] = p[1] * ((1 - u[1]^2) * du[1] - u[1])\nend\np = [1e5]\ntspan = (0.0, 6.3)\nu0 = [2.0]\ndu0 = [0.0]\nprob2 = SecondOrderODEProblem(vanderpol2!, du0, u0, tspan, p)\n\ntest_sol2 = solve(prob2, RadauIIA5(), abstol=1/10^14, reltol=1/10^14, dense=false)\nplot(test_sol2, title=\"Van der Pol Solution (2nd order)\", legend=false, ylims=(-2.5, 2.5))","category":"page"},{"location":"benchmarks/vanderpol/","page":"Stiff ODE: Van der Pol","title":"Stiff ODE: Van der Pol","text":"(Image: )","category":"page"},{"location":"benchmarks/vanderpol/","page":"Stiff ODE: Van der Pol","title":"Stiff ODE: Van der Pol","text":"DENSE = false;\nSAVE_EVERYSTEP = false;\n\n_setups = [\n \"EK1(2) 1st order\" => Dict(:alg => EK1(order=2, smooth=DENSE))\n \"EK1(3) 1st order\" => Dict(:alg => EK1(order=3, smooth=DENSE))\n \"EK1(4) 1st order\" => Dict(:alg => EK1(order=4, smooth=DENSE))\n \"EK1(5) 1st order\" => Dict(:alg => EK1(order=5, smooth=DENSE))\n \"EK1(3) 2nd order\" => Dict(:prob_choice => 2, :alg => EK1(order=3, smooth=DENSE))\n \"EK1(4) 2nd order\" => Dict(:prob_choice => 2, :alg => EK1(order=4, smooth=DENSE))\n \"EK1(5) 2nd order\" => Dict(:prob_choice => 2, :alg => EK1(order=5, smooth=DENSE))\n \"EK1(5) 2nd order\" => Dict(:prob_choice => 2, :alg => EK1(order=6, smooth=DENSE))\n]\n\nlabels = first.(_setups)\nsetups = last.(_setups)\n\nabstols = 1.0 ./ 10.0 .^ (6:13)\nreltols = 1.0 ./ 10.0 .^ (3:10)\n\nwp = WorkPrecisionSet(\n [prob, prob2], abstols, reltols, setups;\n names = labels,\n #print_names = true,\n appxsol = [test_sol, test_sol2],\n dense = DENSE,\n save_everystep = SAVE_EVERYSTEP,\n numruns = 10,\n maxiters = Int(1e7),\n timeseries_errors = false,\n verbose = false,\n)\n\nplot(wp, color=[1 1 1 1 2 2 2 2], xticks = 10.0 .^ (-16:1:5))","category":"page"},{"location":"benchmarks/vanderpol/","page":"Stiff ODE: Van der Pol","title":"Stiff ODE: Van der Pol","text":"(Image: )","category":"page"},{"location":"benchmarks/vanderpol/#Conclusion","page":"Stiff ODE: Van der Pol","title":"Conclusion","text":"","category":"section"},{"location":"benchmarks/vanderpol/","page":"Stiff ODE: Van der Pol","title":"Stiff ODE: Van der Pol","text":"Use the EK1 to solve stiff problems, with orders leq 6 depending on the error tolerance.\nWhen the problem is actually a second-order ODE, as is the case for the Van der Pol system here, solve it as a second-order ODE.","category":"page"},{"location":"benchmarks/vanderpol/#Appendix","page":"Stiff ODE: Van der Pol","title":"Appendix","text":"","category":"section"},{"location":"benchmarks/vanderpol/","page":"Stiff ODE: Van der Pol","title":"Stiff ODE: Van der Pol","text":"Computer information:","category":"page"},{"location":"benchmarks/vanderpol/","page":"Stiff ODE: Van der Pol","title":"Stiff ODE: Van der Pol","text":"InteractiveUtils.versioninfo()","category":"page"},{"location":"benchmarks/vanderpol/","page":"Stiff ODE: Van der Pol","title":"Stiff ODE: Van der Pol","text":"Julia Version 1.8.5\nCommit 17cfb8e65ea (2023-01-08 06:45 UTC)\nPlatform Info:\n OS: Linux (x86_64-linux-gnu)\n CPU: 12 × Intel(R) Core(TM) i7-6800K CPU @ 3.40GHz\n WORD_SIZE: 64\n LIBM: libopenlibm\n LLVM: libLLVM-13.0.1 (ORCJIT, broadwell)\n Threads: 12 on 12 virtual cores\nEnvironment:\n JULIA_NUM_THREADS = auto\n JULIA_STACKTRACE_MINIMAL = true","category":"page"},{"location":"benchmarks/rober/#ROBER-benchmark","page":"DAE: ROBER","title":"ROBER benchmark","text":"","category":"section"},{"location":"benchmarks/rober/","page":"DAE: ROBER","title":"DAE: ROBER","text":"Adapted from SciMLBenchmarks.jl.","category":"page"},{"location":"benchmarks/rober/","page":"DAE: ROBER","title":"DAE: ROBER","text":"using LinearAlgebra, Statistics, InteractiveUtils\nusing DiffEqDevTools, ParameterizedFunctions, SciMLBase, OrdinaryDiffEq, Sundials, Plots\nusing ModelingToolkit\nusing ProbNumDiffEq\n\n# Plotting theme\ntheme(:dao;\n markerstrokewidth=0.5,\n legend=:outertopright,\n bottom_margin=5Plots.mm,\n size = (1000, 400),\n)","category":"page"},{"location":"benchmarks/rober/#ROBER-problem-definition","page":"DAE: ROBER","title":"ROBER problem definition","text":"","category":"section"},{"location":"benchmarks/rober/","page":"DAE: ROBER","title":"DAE: ROBER","text":"@variables t y₁(t)=1.0 y₂(t)=0.0 y₃(t)=0.0\n@parameters k₁=0.04 k₂=3e7 k₃=1e4\nD = Differential(t)\neqs = [\n D(y₁) ~ -k₁*y₁ + k₃*y₂*y₃\n D(y₂) ~ k₁*y₁ - k₃*y₂*y₃ - k₂*y₂^2\n 0 ~ y₁ + y₂ + y₃ - 1\n]\n@named sys = ODESystem(eqs)\nmmprob = ODEProblem(sys,[],(0.0,1e5))\ndaeprob = DAEProblem(sys,[D(y₁)=>-0.04, D(y₂)=>0.04, D(y₃)=>0.0],[],(0.0,1e5)) # can't handle this yet\nodaeprob = ODAEProblem(structural_simplify(sys),[],(0.0,1e5)) # can't handle this yet\n\nref_sol = solve(daeprob,IDA(),abstol=1/10^14,reltol=1/10^14,dense=false)\nplot(ref_sol, vars=[y₁,y₂,y₃], title=\"ROBER Solution\", legend=false, ylims=(0, 1))","category":"page"},{"location":"benchmarks/rober/","page":"DAE: ROBER","title":"DAE: ROBER","text":"(Image: )","category":"page"},{"location":"benchmarks/rober/#EK1-accross-orders","page":"DAE: ROBER","title":"EK1 accross orders","text":"","category":"section"},{"location":"benchmarks/rober/","page":"DAE: ROBER","title":"DAE: ROBER","text":"DENSE = false;\nSAVE_EVERYSTEP = false;\n\n_setups = [\n \"EK1($order)\" => Dict(:alg => EK1(order=order, smooth=DENSE))\n for order in 2:6\n]\n\nlabels = first.(_setups)\nsetups = last.(_setups)\n\n# works:\n# abstols = 1.0 ./ 10.0 .^ (4:10)\n# reltols = 1.0 ./ 10.0 .^ (1:7)\n# test:\nabstols = 1.0 ./ 10.0 .^ (4:9)\nreltols = 1.0 ./ 10.0 .^ (1:6)\n\nwp = WorkPrecisionSet(\n mmprob, abstols, reltols, setups;\n names = labels,\n #print_names = true,\n appxsol = ref_sol,\n dense = DENSE,\n save_everystep = SAVE_EVERYSTEP,\n numruns = 10,\n maxiters = Int(1e7),\n timeseries_errors = false,\n verbose = false,\n)\n\nplot(wp, palette=Plots.palette([:blue, :red], length(_setups)), xticks = 10.0 .^ (-16:1:5),\n xlims = (2e-15, 3e-7), ylims = (1e-2, 6e-1))","category":"page"},{"location":"benchmarks/rober/","page":"DAE: ROBER","title":"DAE: ROBER","text":"(Image: )","category":"page"},{"location":"benchmarks/rober/#Conclusion","page":"DAE: ROBER","title":"Conclusion","text":"","category":"section"},{"location":"benchmarks/rober/","page":"DAE: ROBER","title":"DAE: ROBER","text":"The EK1 can solve mass-matrix DAEs! But it only really works well for low errors.\nOrder 3 seems to work well here. But the order-to-error-tolerance heuristic should in principle still hold: lower tolerance level rightarrow higher order.","category":"page"},{"location":"benchmarks/rober/#Appendix","page":"DAE: ROBER","title":"Appendix","text":"","category":"section"},{"location":"benchmarks/rober/","page":"DAE: ROBER","title":"DAE: ROBER","text":"Computer information:","category":"page"},{"location":"benchmarks/rober/","page":"DAE: ROBER","title":"DAE: ROBER","text":"InteractiveUtils.versioninfo()","category":"page"},{"location":"benchmarks/rober/","page":"DAE: ROBER","title":"DAE: ROBER","text":"Julia Version 1.8.5\nCommit 17cfb8e65ea (2023-01-08 06:45 UTC)\nPlatform Info:\n OS: Linux (x86_64-linux-gnu)\n CPU: 12 × Intel(R) Core(TM) i7-6800K CPU @ 3.40GHz\n WORD_SIZE: 64\n LIBM: libopenlibm\n LLVM: libLLVM-13.0.1 (ORCJIT, broadwell)\n Threads: 12 on 12 virtual cores\nEnvironment:\n JULIA_NUM_THREADS = auto\n JULIA_STACKTRACE_MINIMAL = true","category":"page"},{"location":"filtering/#Gaussian-Filtering-and-Smoothing","page":"Filtering and Smoothing","title":"Gaussian Filtering and Smoothing","text":"","category":"section"},{"location":"filtering/#Predict","page":"Filtering and Smoothing","title":"Predict","text":"","category":"section"},{"location":"filtering/","page":"Filtering and Smoothing","title":"Filtering and Smoothing","text":"ProbNumDiffEq.predict\nProbNumDiffEq.predict!","category":"page"},{"location":"filtering/#ProbNumDiffEq.predict","page":"Filtering and Smoothing","title":"ProbNumDiffEq.predict","text":"predict(x::Gaussian, A::AbstractMatrix, Q::AbstractMatrix)\n\nPrediction step in Kalman filtering for linear dynamics models.\n\nGiven a Gaussian x = mathcalN(μ Σ), compute and return mathcalN(A μ A Σ A^T + Q).\n\nSee also the non-allocating square-root version predict!.\n\n\n\n\n\n","category":"function"},{"location":"filtering/#ProbNumDiffEq.predict!","page":"Filtering and Smoothing","title":"ProbNumDiffEq.predict!","text":"predict!(x_out, x_curr, Ah, Qh, cachemat)\n\nIn-place and square-root implementation of predict which saves the result into x_out.\n\nOnly works with PSDMatrices.PSDMatrix types as Ah, Qh, and in the covariances of x_curr and x_out (both of type Gaussian). To prevent allocations, a cache matrix cachemat of size D times 2D (where D times D is the size of Ah and Qh) needs to be passed.\n\nSee also: predict.\n\n\n\n\n\n","category":"function"},{"location":"filtering/#Update","page":"Filtering and Smoothing","title":"Update","text":"","category":"section"},{"location":"filtering/","page":"Filtering and Smoothing","title":"Filtering and Smoothing","text":"ProbNumDiffEq.update\nProbNumDiffEq.update!","category":"page"},{"location":"filtering/#ProbNumDiffEq.update","page":"Filtering and Smoothing","title":"ProbNumDiffEq.update","text":"update(x, measurement, H)\n\nUpdate step in Kalman filtering for linear dynamics models.\n\nGiven a Gaussian x = mathcalN(μ Σ) and a measurement z = mathcalN(hatz S), with S = H Σ H^T, compute\n\nbeginaligned\nK = Σ^P H^T S^-1 \nμ^F = μ + K (0 - hatz) \nΣ^F = Σ - K S K^T\nendaligned\n\nand return an updated state \\mathcal{N}(μ^F, Σ^F). Note that this assumes zero-measurements. When called with ProbNumDiffEq.SquarerootMatrix type arguments it performs the update in Joseph / square-root form.\n\nFor better performance, we recommend to use the non-allocating update!.\n\n\n\n\n\n","category":"function"},{"location":"filtering/#ProbNumDiffEq.update!","page":"Filtering and Smoothing","title":"ProbNumDiffEq.update!","text":"update!(x_out, x_pred, measurement, H, K_cache, M_cache, S_cache)\n\nIn-place and square-root implementation of update which saves the result into x_out.\n\nImplemented in Joseph Form to retain the PSDMatrix covariances:\n\nbeginaligned\nK = Σ^P H^T S^-1 \nμ^F = μ + K (0 - hatz) \nsqrtΣ^F = (I - KH) sqrt(Σ)\nendaligned\n\nwhere sqrtM denotes the left square-root of a matrix M, i.e. M = sqrtM sqrtM^T.\n\nTo prevent allocations, write into caches K_cache and M_cache, both of size D × D, and S_cache of same type as measurement.Σ.\n\nSee also: update.\n\n\n\n\n\n","category":"function"},{"location":"filtering/#Smooth","page":"Filtering and Smoothing","title":"Smooth","text":"","category":"section"},{"location":"filtering/","page":"Filtering and Smoothing","title":"Filtering and Smoothing","text":"ProbNumDiffEq.smooth\nProbNumDiffEq.smooth!","category":"page"},{"location":"filtering/#ProbNumDiffEq.smooth","page":"Filtering and Smoothing","title":"ProbNumDiffEq.smooth","text":"smooth(x_curr, x_next_smoothed, A, Q)\n\nUpdate step of the Kalman smoother, aka. Rauch-Tung-Striebel smoother, for linear dynamics models.\n\nGiven Gaussians x_n = mathcalN(μ_n Σ_n) and x_n+1 = mathcalN(μ_n+1^S Σ_n+1^S), compute\n\nbeginaligned\nμ_n+1^P = A μ_n^F \nP_n+1^P = A Σ_n^F A + Q \nG = Σ_n^S A^T (Σ_n+1^P)^-1 \nμ_n^S = μ_n^F + G (μ_n+1^S - μ_n+1^P) \nΣ_n^S = (I - G A) Σ_n^F (I - G A)^T + G Q G^T + G Σ_n+1^S G^T\nendaligned\n\nand return a smoothed state \\mathcal{N}(μ_n^S, Σ_n^S). When called with ProbNumDiffEq.SquarerootMatrix type arguments it performs the update in Joseph / square-root form.\n\nFor better performance, we recommend to use the non-allocating smooth!.\n\n\n\n\n\n","category":"function"},{"location":"filtering/#ProbNumDiffEq.smooth!","page":"Filtering and Smoothing","title":"ProbNumDiffEq.smooth!","text":"smooth!(x_curr, x_next, Ah, Qh, cache, diffusion=1)\n\nIn-place and square-root implementation of smooth which overwrites x_curr.\n\nImplemented in Joseph form to preserve square-root structure. It requires access to the solvers cache to prevent allocations.\n\nSee also: smooth.\n\n\n\n\n\n","category":"function"},{"location":"solvers/#Solvers","page":"Solvers","title":"Solvers","text":"","category":"section"},{"location":"solvers/","page":"Solvers","title":"Solvers","text":"ProbNumDiffEq.jl provides two solvers: the EK1 and the EK0. Both based on extended Kalman filtering and smoothing, but the latter relies on evaluating the Jacobian of the vector field. For the best results, use the EK1.","category":"page"},{"location":"solvers/","page":"Solvers","title":"Solvers","text":"All solvers are compatible with DAEs in mass-matrix ODE form. They also specialize on second-order ODEs: If the problem is of type SecondOrderODEProblem, it solves the second-order problem directly; this is more efficient than solving the transformed first-order problem and provides more meaningful posteriors [1].","category":"page"},{"location":"solvers/#API","page":"Solvers","title":"API","text":"","category":"section"},{"location":"solvers/","page":"Solvers","title":"Solvers","text":"EK1\nEK0","category":"page"},{"location":"solvers/#ProbNumDiffEq.EK1","page":"Solvers","title":"ProbNumDiffEq.EK1","text":"EK1(; order=3,\n smooth=true,\n prior=IWP(order),\n diffusionmodel=DynamicDiffusion(),\n initialization=TaylorModeInit(),\n kwargs...)\n\nGaussian ODE filter with first-order vector field linearization.\n\nArguments\n\norder::Integer: Order of the integrated Wiener process (IWP) prior.\nsmooth::Bool: Turn smoothing on/off; smoothing is required for dense output.\nprior::AbstractODEFilterPrior: Prior to be used by the ODE filter. By default, uses a 3-times integrated Wiener process prior IWP(3). See also: Priors.\ndiffusionmodel::ProbNumDiffEq.AbstractDiffusion: See Diffusion models and calibration.\ninitialization::ProbNumDiffEq.InitializationScheme: See Initialization.\n\nSome additional kwargs relating to implicit solvers are supported; check out DifferentialEquations.jl's Extra Options page. Right now, we support autodiff, chunk_size, and diff_type. In particular, autodiff=false can come in handy to use finite differences instead of ForwardDiff.jl to compute Jacobians.\n\nExamples\n\njulia> solve(prob, EK1())\n\nReferences\n\n\n\n\n\n","category":"type"},{"location":"solvers/#ProbNumDiffEq.EK0","page":"Solvers","title":"ProbNumDiffEq.EK0","text":"EK0(; order=3,\n smooth=true,\n prior=IWP(order),\n diffusionmodel=DynamicDiffusion(),\n initialization=TaylorModeInit())\n\nGaussian ODE filter with zeroth-order vector field linearization.\n\nArguments\n\norder::Integer: Order of the integrated Wiener process (IWP) prior.\nsmooth::Bool: Turn smoothing on/off; smoothing is required for dense output.\nprior::AbstractODEFilterPrior: Prior to be used by the ODE filter. By default, uses a 3-times integrated Wiener process prior IWP(3). See also: Priors.\ndiffusionmodel::ProbNumDiffEq.AbstractDiffusion: See Diffusion models and calibration.\ninitialization::ProbNumDiffEq.InitializationScheme: See Initialization.\n\nExamples\n\njulia> solve(prob, EK0())\n\nReferences\n\n\n\n\n\n","category":"type"},{"location":"solvers/#Probabilistic-Exponential-Integrators","page":"Solvers","title":"Probabilistic Exponential Integrators","text":"","category":"section"},{"location":"solvers/","page":"Solvers","title":"Solvers","text":"ExpEK\nRosenbrockExpEK","category":"page"},{"location":"solvers/#ProbNumDiffEq.ExpEK","page":"Solvers","title":"ProbNumDiffEq.ExpEK","text":"ExpEK(; L, order=3, kwargs...)\n\nProbabilistic exponential integrator\n\nProbabilistic exponential integrators are a class of integrators for semi-linear stiff ODEs that provide improved stability by essentially solving the linear part of the ODE exactly. In probabilistic numerics, this amounts to including the linear part into the prior model of the solver.\n\nExpEK is therefore just a short-hand for EK0 with IOUP prior:\n\nExpEK(; order=3, L, kwargs...) = EK0(; prior=IOUP(order, L), kwargs...)\n\nSee also RosenbrockExpEK, EK0, EK1.\n\nArguments\n\nSee EK0 for available keyword arguments.\n\nExamples\n\njulia> prob = ODEProblem((du, u, p, t) -> (@. du = - u + sin(u)), [1.0], (0.0, 10.0))\njulia> solve(prob, ExpEK(L=-1))\n\nReference\n\n[1] N. Bosch, P. Hennig, F. Tronarp: Probabilistic Exponential Integrators (2023) (link)\n\n\n\n\n\n","category":"function"},{"location":"solvers/#ProbNumDiffEq.RosenbrockExpEK","page":"Solvers","title":"ProbNumDiffEq.RosenbrockExpEK","text":"RosenbrockExpEK(; order=3, kwargs...)\n\nProbabilistic Rosenbrock-type exponential integrator\n\nA probabilistic exponential integrator similar to ExpEK, but with automatic linearization along the mean numerical solution. This brings the advantage that the linearity does not need to be specified manually, and the more accurate local linearization can sometimes also improve stability; but since the \"prior\" is adjusted at each step the probabilistic interpretation becomes more complicated.\n\nRosenbrockExpEK is just a short-hand for EK1 with appropriete IOUP prior:\n\nRosenbrockExpEK(; order=3, kwargs...) = EK1(; prior=IOUP(order, update_rate_parameter=true), kwargs...)\n\nSee also ExpEK, EK0, EK1.\n\nArguments\n\nSee EK1 for available keyword arguments.\n\nExamples\n\njulia> prob = ODEProblem((du, u, p, t) -> (@. du = - u + sin(u)), [1.0], (0.0, 10.0))\njulia> solve(prob, RosenbrockExpEK())\n\nReference\n\n[1] N. Bosch, P. Hennig, F. Tronarp: Probabilistic Exponential Integrators (2023) (link)\n\n\n\n\n\n","category":"function"},{"location":"solvers/#solversrefs","page":"Solvers","title":"References","text":"","category":"section"},{"location":"solvers/","page":"Solvers","title":"Solvers","text":"[1] N. Bosch, F. Tronarp, P. Hennig: Pick-and-Mix Information Operators for Probabilistic ODE Solvers (2022) (link)","category":"page"},{"location":"tutorials/exponential_integrators/#Probabilistic-Exponential-Integrators","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"","category":"section"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"Exponential integrators are a class of numerical methods for solving semi-linear ordinary differential equations (ODEs) of the form","category":"page"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"beginaligned\ndoty(t) = L y(t) + f(y(t) t) quad y(0) = y_0\nendaligned","category":"page"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"where L is a linear operator and f is a nonlinear function. In a nutshell, exponential integrators solve the linear part of the ODE exactly, and only approximate the nonlinear part. Probabilistic exponential integrators are the probabilistic numerics approach to exponential integrators.","category":"page"},{"location":"tutorials/exponential_integrators/#Example","page":"Probabilistic Exponential Integrators","title":"Example","text":"","category":"section"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"Let's consider a simple semi-linear ODE","category":"page"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"beginaligned\ndoty(t) = - y(t) + sin(y(t)) quad y(0) = 10\nendaligned","category":"page"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"We can solve this ODE reasonably well with the standard EK1 and adaptive steps (the default):","category":"page"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"using ProbNumDiffEq, Plots, LinearAlgebra\ntheme(:default; palette=[\"#4063D8\", \"#389826\", \"#9558B2\", \"#CB3C33\"])\n\nf(du, u, p, t) = (@. du = -u + sin(u))\nu0 = [1.0]\ntspan = (0.0, 20.0)\nprob = ODEProblem(f, u0, tspan)\n\nref = solve(prob, EK1(), abstol=1e-10, reltol=1e-10)\nplot(ref, color=:black, linestyle=:dash, label=\"Reference\")","category":"page"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"But for fixed (large) step sizes this ODE is more challenging: The explicit EK0 method oscillates and diverges due to the stiffness of the ODE, and the semi-implicit EK1 method is stable but the solution is not very accurate.","category":"page"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"STEPSIZE = 4\nDM = FixedDiffusion() # recommended for fixed steps\n\n# we don't smooth the EK0 here to show the oscillations more clearly\nsol0 = solve(prob, EK0(smooth=false, diffusionmodel=DM), adaptive=false, dt=STEPSIZE, dense=false)\nsol1 = solve(prob, EK1(diffusionmodel=DM), adaptive=false, dt=STEPSIZE)\n\nplot(ylims=(0.3, 1.05))\nplot!(ref, color=:black, linestyle=:dash, label=\"Reference\")\nplot!(sol0, denseplot=false, marker=:o, markersize=2, label=\"EK0\", color=1)\nplot!(sol1, denseplot=false, marker=:o, markersize=2, label=\"EK1\", color=2)","category":"page"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"Probabilistic exponential integrators leverage the semi-linearity of the ODE to compute more accurate solutions for the same fixed step size. You can use either the ExpEK method and provide the linear part (with the keyword argument L), or the RosenbrockExpEK to automatically linearize along the mean of the numerical solution:","category":"page"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"sol_exp = solve(prob, ExpEK(L=-1, diffusionmodel=DM), adaptive=false, dt=STEPSIZE)\nsol_ros = solve(prob, RosenbrockExpEK(diffusionmodel=DM), adaptive=false, dt=STEPSIZE)\n\nplot(ylims=(0.3, 1.05))\nplot!(ref, color=:black, linestyle=:dash, label=\"Reference\")\nplot!(sol_exp, denseplot=false, marker=:o, markersize=2, label=\"ExpEK\", color=3)\nplot!(sol_ros, denseplot=false, marker=:o, markersize=2, label=\"RosenbrockExpEK\", color=4)","category":"page"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"The solutions are indeed much more accurate than those of the standard EK1, for the same fixed step size!","category":"page"},{"location":"tutorials/exponential_integrators/#Background:-Integrated-Ornstein-Uhlenbeck-priors","page":"Probabilistic Exponential Integrators","title":"Background: Integrated Ornstein-Uhlenbeck priors","text":"","category":"section"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"Probabilistic exponential integrators \"solve the linear part exactly\" by including it into the prior model of the solver. Namely, the solver chooses a (q-times) integrated Ornstein-Uhlenbeck prior with rate parameter equal to the linearity. The ExpEK solver is just a short-hand for an EK0 with appropriate prior:","category":"page"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"ExpEK(order=3, L=-1) == EK0(prior=IOUP(3, -1))","category":"page"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"Similarly, the RosenbrockExpEK solver is also just a short-hand:","category":"page"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"RosenbrockExpEK(order=3) == EK1(prior=IOUP(3, update_rate_parameter=true))","category":"page"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"This means that you can also construct other probabilistic exponential integrators by hand! In this example the EK1 with IOUP prior with rate parameter -1 performs extremely well:","category":"page"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"sol_expek1 = solve(prob, EK1(prior=IOUP(3, -1), diffusionmodel=DM), adaptive=false, dt=STEPSIZE)\n\nplot(ylims=(0.3, 1.05))\nplot!(ref, color=:black, linestyle=:dash, label=\"Reference\")\nplot!(sol_expek1, denseplot=false, marker=:o, markersize=2, label=\"EK1 + IOUP\")","category":"page"},{"location":"tutorials/exponential_integrators/#References","page":"Probabilistic Exponential Integrators","title":"References","text":"","category":"section"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"[1] N. Bosch, P. Hennig, F. Tronarp: Probabilistic Exponential Integrators (2023) (link)","category":"page"},{"location":"benchmarks/lotkavolterra/#Lotka-Volterra-benchmark","page":"Non-stiff ODE: Lotka-Volterra","title":"Lotka-Volterra benchmark","text":"","category":"section"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"Adapted from SciMLBenchmarks.jl.","category":"page"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"using LinearAlgebra, Statistics, InteractiveUtils\nusing DiffEqDevTools, ParameterizedFunctions, SciMLBase, OrdinaryDiffEq, Plots\nusing ProbNumDiffEq\n\n# Plotting theme\ntheme(:dao;\n markerstrokewidth=0.5,\n legend=:outertopright,\n bottom_margin=5Plots.mm,\n size = (1000, 400),\n)","category":"page"},{"location":"benchmarks/lotkavolterra/#Lotka-Volterra-problem-definition","page":"Non-stiff ODE: Lotka-Volterra","title":"Lotka-Volterra problem definition","text":"","category":"section"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"f = @ode_def LotkaVolterra begin\n dx = a*x - b*x*y\n dy = -c*y + d*x*y\nend a b c d\np = [1.5, 1, 3, 1]\ntspan = (0.0, 10.0)\nu0 = [1.0, 1.0]\nprob = ODEProblem{true, SciMLBase.FullSpecialize}(f, u0, tspan, p)\n\ntest_sol = solve(prob, Vern7(), abstol=1/10^14, reltol=1/10^14, dense=false)\nplot(test_sol, title=\"Lotka-Volterra Solution\", legend=false)","category":"page"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"(Image: )","category":"page"},{"location":"benchmarks/lotkavolterra/#EK0-accross-orders","page":"Non-stiff ODE: Lotka-Volterra","title":"EK0 accross orders","text":"","category":"section"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"DENSE = false;\nSAVE_EVERYSTEP = false;\n\n_setups = [\n \"EK0(order=$order)\" => Dict(:alg => EK0(order=order, smooth=DENSE))\n for order in 1:7\n]\n\nlabels = first.(_setups)\nsetups = last.(_setups)\n\nabstols = 1.0 ./ 10.0 .^ (4:13)\nreltols = 1.0 ./ 10.0 .^ (1:10)\n\nwp = WorkPrecisionSet(\n prob, abstols, reltols, setups;\n names = labels,\n #print_names = true,\n appxsol = test_sol,\n dense = DENSE,\n save_everystep = SAVE_EVERYSTEP,\n numruns = 10,\n maxiters = Int(1e7),\n timeseries_errors = false,\n verbose = false,\n)\n\nplot(wp, palette=Plots.palette([:blue, :red], length(_setups)), xticks = 10.0 .^ (-16:1:5))","category":"page"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"(Image: )","category":"page"},{"location":"benchmarks/lotkavolterra/#EK1-accross-orders","page":"Non-stiff ODE: Lotka-Volterra","title":"EK1 accross orders","text":"","category":"section"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"DENSE = false;\nSAVE_EVERYSTEP = false;\n\n_setups = [\n \"EK1(order=$order)\" => Dict(:alg => EK1(order=order, smooth=DENSE))\n for order in 1:7\n]\n\nlabels = first.(_setups)\nsetups = last.(_setups)\n\nabstols = 1.0 ./ 10.0 .^ (4:13)\nreltols = 1.0 ./ 10.0 .^ (1:10)\n\nwp = WorkPrecisionSet(\n prob, abstols, reltols, setups;\n names = labels,\n #print_names = true,\n appxsol = test_sol,\n dense = DENSE,\n save_everystep = SAVE_EVERYSTEP,\n numruns = 10,\n maxiters = Int(1e7),\n timeseries_errors = false,\n verbose = false,\n)\n\nplot(wp, palette=Plots.palette([:blue, :red], length(_setups)), xticks = 10.0 .^ (-16:1:5))","category":"page"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"(Image: )","category":"page"},{"location":"benchmarks/lotkavolterra/#EK0-vs.-EK1","page":"Non-stiff ODE: Lotka-Volterra","title":"EK0 vs. EK1","text":"","category":"section"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"DENSE = false;\nSAVE_EVERYSTEP = false;\n\n_setups = [\n \"EK0(order=2)\" => Dict(:alg => EK0(order=2, smooth=DENSE))\n \"EK0(order=3)\" => Dict(:alg => EK0(order=3, smooth=DENSE))\n \"EK0(order=4)\" => Dict(:alg => EK0(order=4, smooth=DENSE))\n \"EK0(order=5)\" => Dict(:alg => EK0(order=5, smooth=DENSE))\n \"EK1(order=2)\" => Dict(:alg => EK1(order=2, smooth=DENSE))\n \"EK1(order=3)\" => Dict(:alg => EK1(order=3, smooth=DENSE))\n \"EK1(order=4)\" => Dict(:alg => EK1(order=4, smooth=DENSE))\n \"EK1(order=5)\" => Dict(:alg => EK1(order=5, smooth=DENSE))\n]\n\nlabels = first.(_setups)\nsetups = last.(_setups)\n\nabstols = 1.0 ./ 10.0 .^ (4:14)\nreltols = 1.0 ./ 10.0 .^ (1:11)\n\nwp = WorkPrecisionSet(\n prob, abstols, reltols, setups;\n names = labels,\n #print_names = true,\n appxsol = test_sol,\n dense = DENSE,\n save_everystep = SAVE_EVERYSTEP,\n numruns = 10,\n maxiters = Int(1e7),\n timeseries_errors = false,\n verbose = false,\n)\n\nplot(wp, color=[1 1 1 1 2 2 2 2], xticks = 10.0 .^ (-16:1:5))","category":"page"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"(Image: )","category":"page"},{"location":"benchmarks/lotkavolterra/#DynamicDiffusion-vs-FixedDiffusion","page":"Non-stiff ODE: Lotka-Volterra","title":"DynamicDiffusion vs FixedDiffusion","text":"","category":"section"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"DENSE = false;\nSAVE_EVERYSTEP = false;\n\n_setups = [\n \"EK1(2) dynamic\" => Dict(:alg => EK1(order=2, smooth=DENSE, diffusionmodel=DynamicDiffusion()))\n \"EK1(3) dynamic\" => Dict(:alg => EK1(order=3, smooth=DENSE, diffusionmodel=DynamicDiffusion()))\n \"EK1(5) dynamic\" => Dict(:alg => EK1(order=5, smooth=DENSE, diffusionmodel=DynamicDiffusion()))\n \"EK1(2) fixed\" => Dict(:alg => EK1(order=2, smooth=DENSE, diffusionmodel=FixedDiffusion()))\n \"EK1(3) fixed\" => Dict(:alg => EK1(order=3, smooth=DENSE, diffusionmodel=FixedDiffusion()))\n \"EK1(5) fixed\" => Dict(:alg => EK1(order=5, smooth=DENSE, diffusionmodel=FixedDiffusion()))\n]\n\nlabels = first.(_setups)\nsetups = last.(_setups)\n\nabstols = 1.0 ./ 10.0 .^ (4:15)\nreltols = 1.0 ./ 10.0 .^ (1:12)\n\nwp = WorkPrecisionSet(\n prob, abstols, reltols, setups;\n names = labels,\n #print_names = true,\n appxsol = test_sol,\n dense = DENSE,\n save_everystep = SAVE_EVERYSTEP,\n numruns = 10,\n maxiters = Int(1e7),\n timeseries_errors = false,\n verbose = false,\n)\n\nplot(wp, color=[2 2 2 3 3 3], xticks = 10.0 .^ (-16:1:5))","category":"page"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"(Image: )","category":"page"},{"location":"benchmarks/lotkavolterra/#TaylorModeInit-vs-ClassicSolverInit","page":"Non-stiff ODE: Lotka-Volterra","title":"TaylorModeInit vs ClassicSolverInit","text":"","category":"section"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"DENSE = false;\nSAVE_EVERYSTEP = false;\n\n_setups = [\n \"EK1(2) TaylorInit\" => Dict(:alg => EK1(order=2, smooth=DENSE, initialization=TaylorModeInit()))\n \"EK1(3) TaylorInit\" => Dict(:alg => EK1(order=3, smooth=DENSE, initialization=TaylorModeInit()))\n \"EK1(5) TaylorInit\" => Dict(:alg => EK1(order=5, smooth=DENSE, initialization=TaylorModeInit()))\n \"EK1(2) Tsit5Init\" => Dict(:alg => EK1(order=2, smooth=DENSE, initialization=ClassicSolverInit()))\n \"EK1(3) Tsit5Init\" => Dict(:alg => EK1(order=3, smooth=DENSE, initialization=ClassicSolverInit()))\n \"EK1(5) Tsit5Init\" => Dict(:alg => EK1(order=5, smooth=DENSE, initialization=ClassicSolverInit()))\n \"EK1(2) Tsit5Init+ddu\" => Dict(:alg => EK1(order=2, smooth=DENSE, initialization=ClassicSolverInit(init_on_ddu=true)))\n \"EK1(3) Tsit5Init+ddu\" => Dict(:alg => EK1(order=3, smooth=DENSE, initialization=ClassicSolverInit(init_on_ddu=true)))\n \"EK1(5) Tsit5Init+ddu\" => Dict(:alg => EK1(order=5, smooth=DENSE, initialization=ClassicSolverInit(init_on_ddu=true)))\n]\n\nlabels = first.(_setups)\nsetups = last.(_setups)\n\nabstols = 1.0 ./ 10.0 .^ (4:15)\nreltols = 1.0 ./ 10.0 .^ (1:12)\n\nwp = WorkPrecisionSet(\n prob, abstols, reltols, setups;\n names = labels,\n #print_names = true,\n appxsol = test_sol,\n dense = DENSE,\n save_everystep = SAVE_EVERYSTEP,\n numruns = 10,\n maxiters = Int(1e7),\n timeseries_errors = false,\n verbose = false,\n)\n\nplot(wp, color=[2 2 2 4 4 4 5 5 5], xticks = 10.0 .^ (-16:1:5))","category":"page"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"(Image: )","category":"page"},{"location":"benchmarks/lotkavolterra/#Conclusion","page":"Non-stiff ODE: Lotka-Volterra","title":"Conclusion","text":"","category":"section"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"Use the EK1! It seems to be strictly better than the EK0 here. Though note that by using ParameterizedFunctions.jl, the Jacobian of the vector field is available analytically.\nOrders behave as in classic solvers: Use low order for low accuracy, medium order for medium accuracy, high order for high accuracy.\nMost likely, the default choice of diffusionmodel=DynamicDiffusion and initialization=TaylorModeInit are fine.","category":"page"},{"location":"benchmarks/lotkavolterra/#Appendix","page":"Non-stiff ODE: Lotka-Volterra","title":"Appendix","text":"","category":"section"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"Computer information:","category":"page"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"InteractiveUtils.versioninfo()","category":"page"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"Julia Version 1.8.5\nCommit 17cfb8e65ea (2023-01-08 06:45 UTC)\nPlatform Info:\n OS: Linux (x86_64-linux-gnu)\n CPU: 12 × Intel(R) Core(TM) i7-6800K CPU @ 3.40GHz\n WORD_SIZE: 64\n LIBM: libopenlibm\n LLVM: libLLVM-13.0.1 (ORCJIT, broadwell)\n Threads: 12 on 12 virtual cores\nEnvironment:\n JULIA_NUM_THREADS = auto\n JULIA_STACKTRACE_MINIMAL = true","category":"page"},{"location":"initialization/#Initialization","page":"Initialization","title":"Initialization","text":"","category":"section"},{"location":"initialization/","page":"Initialization","title":"Initialization","text":"The notion of \"initialization\" relates to the prior part of the model.","category":"page"},{"location":"initialization/#Background:-The-prior","page":"Initialization","title":"Background: The prior","text":"","category":"section"},{"location":"initialization/","page":"Initialization","title":"Initialization","text":"We model the ODE solution y(t) with a Gauss–Markov prior. More precisely, let","category":"page"},{"location":"initialization/","page":"Initialization","title":"Initialization","text":"beginaligned\nY(t) = left Y^(0)(t) Y^(1)(t) dots Y^(q)(t) right\nendaligned","category":"page"},{"location":"initialization/","page":"Initialization","title":"Initialization","text":"be the solution to the SDE","category":"page"},{"location":"initialization/","page":"Initialization","title":"Initialization","text":"beginaligned\ntextd Y^(i)(t) = Y^(i+1)(t) textdt qquad i = 0 dots q-1 \ntextd Y^(q)(t) = textcolor389826A Y(t) textdt + textcolor4063D8Gamma textdW(t) \nY(0) sim textcolor9558B2 mathcalN left( mu_0 Sigma_0 right) \nendaligned","category":"page"},{"location":"initialization/","page":"Initialization","title":"Initialization","text":"Then Y^(i)(t) models the i-th derivative of y(t). In this section, we consider the initial distribution textcolorpurple mathcalN left( mu_0 Sigma_0 right) . If you're more interested in the drift matrix textcolor389826A check out the Priors section, and for more info on the diffusion textcolor4063D8Gamma check out the Diffusion models and calibration section.","category":"page"},{"location":"initialization/#Setting-the-initial-distribution","page":"Initialization","title":"Setting the initial distribution","text":"","category":"section"},{"location":"initialization/","page":"Initialization","title":"Initialization","text":"Let's assume an initial value problem of the form","category":"page"},{"location":"initialization/","page":"Initialization","title":"Initialization","text":"beginaligned\ndoty(t) = f(y(t) t) qquad 0 T \ny(0) = y_0\nendaligned","category":"page"},{"location":"initialization/","page":"Initialization","title":"Initialization","text":"It is clear that this contains quite some information for Y(0): The initial value y_0 and the vector field f imply","category":"page"},{"location":"initialization/","page":"Initialization","title":"Initialization","text":"beginaligned\nY^(0)(0) = y_0 \nY^(1)(0) = f(y_0 0)\nendaligned","category":"page"},{"location":"initialization/","page":"Initialization","title":"Initialization","text":"It turns out that we can also compute higher-order derivatives of y with the chain rule, and then use these to better initialize Y^(i)(0). This, done efficiently with Taylor-mode autodiff by using TaylorIntegration.jl, is what TaylorModeInit does. See also [1].","category":"page"},{"location":"initialization/","page":"Initialization","title":"Initialization","text":"In the vast majority of cases, just stick to the exact Taylor-mode initialization TaylorModeInit!","category":"page"},{"location":"initialization/#API","page":"Initialization","title":"API","text":"","category":"section"},{"location":"initialization/","page":"Initialization","title":"Initialization","text":"TaylorModeInit\nClassicSolverInit","category":"page"},{"location":"initialization/#ProbNumDiffEq.TaylorModeInit","page":"Initialization","title":"ProbNumDiffEq.TaylorModeInit","text":"TaylorModeInit()\n\nExact initialization via Taylor-mode automatic differentiation.\n\nThis is the recommended initialization method!\n\nIt uses TaylorIntegration.jl to efficiently compute the higher-order derivatives of the solution at the initial value, via Taylor-mode automatic differentiation.\n\nIn some special cases it can happen that TaylorIntegration.jl is incompatible with the given problem (typically because the problem definition does not allow for elements of type Taylor). If this happens, try ClassicSolverInit.\n\nReferences\n\n[1] N. Krämer, P. Hennig: Stable Implementation of Probabilistic ODE Solvers (2020)\n\n\n\n\n\n","category":"type"},{"location":"initialization/#ProbNumDiffEq.ClassicSolverInit","page":"Initialization","title":"ProbNumDiffEq.ClassicSolverInit","text":"ClassicSolverInit(; alg=OrdinaryDiffEq.Tsit5(), init_on_ddu=false)\n\nInitialization via regression on a few steps of a classic ODE solver.\n\nIn a nutshell, instead of specifying mu_0 exactly and setting Sigma_0=0 (which is what TaylorModeInit does), use a classic ODE solver to compute a few steps of the solution, and then regress on the computed values (by running a smoother) to compute mu_0 and Sigma_0 as the mean and covariance of the smoothing posterior at time 0. See also [2].\n\nThe initial value and derivative are set directly from the given initial value problem; optionally the second derivative can also be set via automatic differentiation by setting init_on_ddu=true.\n\nArguments\n\nalg: The solver to be used. Can be any solver from OrdinaryDiffEq.jl.\ninit_on_ddu: If true, the second derivative is also initialized exactly via automatic differentiation with ForwardDiff.jl.\n\nReferences\n\n[2] M. Schober, S. Särkkä, and P. Hennig: A Probabilistic Model for the Numerical Solution of Initial Value Problems (2018)\n\n\n\n\n\n","category":"type"},{"location":"initialization/#initrefs","page":"Initialization","title":"References","text":"","category":"section"},{"location":"initialization/","page":"Initialization","title":"Initialization","text":"[1] N. Krämer, P. Hennig: Stable Implementation of Probabilistic ODE Solvers (2020) (link) [2] M. Schober, S. Särkkä, and P. Hennig: A Probabilistic Model for the Numerical Solution of Initial Value Problems (2018) (link)","category":"page"},{"location":"tutorials/dae/#Solving-DAEs-with-Probabilistic-Numerics","page":"Differential Algebraic Equations","title":"Solving DAEs with Probabilistic Numerics","text":"","category":"section"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"ProbNumDiffEq.jl provides probabilistic numerical solvers for differential algebraic equations (DAEs). Currently, we recommend using the semi-implicit EK1 algorithm.","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"note: Note\nFor a more general tutorial on DAEs check out the DifferentialEquations.jl DAE tutorial.","category":"page"},{"location":"tutorials/dae/#Solving-mass-matrix-DAEs-with-the-EK1","page":"Differential Algebraic Equations","title":"Solving mass-matrix DAEs with the EK1","text":"","category":"section"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"First, define the DAE (here the ROBER problem) as an ODE problem with singular mass matrix:","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"using ProbNumDiffEq, Plots, LinearAlgebra, OrdinaryDiffEq, ModelingToolkit\n\nfunction rober(du, u, p, t)\n y₁, y₂, y₃ = u\n k₁, k₂, k₃ = p\n du[1] = -k₁ * y₁ + k₃ * y₂ * y₃\n du[2] = k₁ * y₁ - k₃ * y₂ * y₃ - k₂ * y₂^2\n du[3] = y₁ + y₂ + y₃ - 1\n nothing\nend\nM = [1 0 0\n 0 1 0\n 0 0 0]\nf = ODEFunction(rober, mass_matrix=M)\nprob_mm = ODEProblem(f, [1.0, 0.0, 0.0], (0.0, 1e5), (0.04, 3e7, 1e4))","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"We can solve this problem directly with the EK1:","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"sol = solve(prob_mm, EK1(), reltol=1e-8, abstol=1e-8)\nplot(\n sol,\n xscale=:log10,\n tspan=(1e-6, 1e5),\n layout=(3, 1),\n legend=false,\n ylabel=[\"u₁(t)\" \"u₂(t)\" \"u₃(t)\"],\n xlabel=[\"\" \"\" \"t\"],\n denseplot=false,\n)","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"Looks good!","category":"page"},{"location":"tutorials/dae/#Solving-an-Index-3-DAE-directly","page":"Differential Algebraic Equations","title":"Solving an Index-3 DAE directly","text":"","category":"section"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"The following is based on the \"Automatic Index Reduction of DAEs\" tutorial by ModelingToolkit.jl, which demonstrates how the classic Rodas4 solver fails to solve a DAE due to the fact that it is of index 3; which is why ModelingToolkit's automatic index reduction is so useful.","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"It turns out that our probabilistic numerical solvers can directly solve the index-3 DAE!","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"First, define the pendulum problem as in the tutorial:","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"function pendulum!(du, u, p, t)\n x, dx, y, dy, T = u\n g, L = p\n du[1] = dx\n du[2] = T * x\n du[3] = dy\n du[4] = T * y - g\n du[5] = x^2 + y^2 - L^2\nend\npendulum_fun! = ODEFunction(pendulum!, mass_matrix=Diagonal([1, 1, 1, 1, 0]))\nu0 = [1.0, 0, 0, 0, 0];\np = [9.8, 1];\ntspan = (0, 5.0);\npendulum_prob = ODEProblem(pendulum_fun!, u0, tspan, p)","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"We can try to solve it directly with one of the classic mass-matrix DAE solvers from OrdinaryDiffEq.jl:","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"solve(pendulum_prob, Rodas4())","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"It does not work! This is because of the index of the DAE; see e.g. this explenation from the tutorial.","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"Does this also hold for the EK1 solver? Let's find out:","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"sol = solve(pendulum_prob, EK1())","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"Nope! The EK1 is able to solve the index-3 DAE directly. Pretty cool!","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"plot(sol)","category":"page"},{"location":"tutorials/dae/#Is-index-reduction-still-worth-it?","page":"Differential Algebraic Equations","title":"Is index-reduction still worth it?","text":"","category":"section"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"The point of the \"Automatic Index Reduction of DAEs\" tutorial is to demonstrate ModelingToolkit's utility for automatic index reduction, which enables the classic implicit Runge-Kutta solvers such as Rodas5 to solve this DAE. Let's see if that still helps in this context here.","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"First, modelingtoolkitize the problem:","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"traced_sys = modelingtoolkitize(pendulum_prob)","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"(how cool is this latex output ?!?)","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"Next, lower the DAE index and simplify it with MTK's dae_index_lowering and structural_simplify:","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"simplified_sys = structural_simplify(dae_index_lowering(traced_sys))","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"Let's build two different ODE problems, and check how well we can solve each:","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"prob_index3 = ODEProblem(traced_sys, Pair[], tspan)\nprob_index1 = ODEProblem(simplified_sys, Pair[], tspan)\n\nsol3 = solve(prob_index3, EK1())\nsol1 = solve(prob_index1, EK1())\n\ntruesol = solve(prob_index1, Rodas4(), abstol=1e-10, reltol=1e-10)\n\nsol1_final_error = norm(sol1.u[end] - truesol.u[end])\nsol1_f_evals = sol1.stats.nf\nsol3_final_error = norm(sol3.u[end] - truesol.u[end])\nsol3_f_evals = sol3.stats.nf\n@info \"Results\" sol1_final_error sol1_f_evals sol3_final_error sol3_f_evals","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"The error for the index-1 DAE solve is much lower. So it seems that, even if the index-3 DAE could also be solved directly, index lowering might still be beneficial when solving DAEs with the EK1!","category":"page"},{"location":"tutorials/dae/#References","page":"Differential Algebraic Equations","title":"References","text":"","category":"section"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"[1] N. Bosch, F. Tronarp, P. Hennig: Pick-and-Mix Information Operators for Probabilistic ODE Solvers (2022) (link)","category":"page"},{"location":"diffusions/#Diffusion-models-and-calibration","page":"Diffusion models and calibration","title":"Diffusion models and calibration","text":"","category":"section"},{"location":"diffusions/","page":"Diffusion models and calibration","title":"Diffusion models and calibration","text":"The notion of \"diffusion\" and \"calibration\" relates to the prior part of the model.","category":"page"},{"location":"diffusions/#Background:-The-prior","page":"Diffusion models and calibration","title":"Background: The prior","text":"","category":"section"},{"location":"diffusions/","page":"Diffusion models and calibration","title":"Diffusion models and calibration","text":"We model the ODE solution y(t) with a Gauss–Markov prior. More precisely, let","category":"page"},{"location":"diffusions/","page":"Diffusion models and calibration","title":"Diffusion models and calibration","text":"beginaligned\nY(t) = left Y^(0)(t) Y^(1)(t) dots Y^(q)(t) right\nendaligned","category":"page"},{"location":"diffusions/","page":"Diffusion models and calibration","title":"Diffusion models and calibration","text":"be the solution to the SDE","category":"page"},{"location":"diffusions/","page":"Diffusion models and calibration","title":"Diffusion models and calibration","text":"beginaligned\ntextd Y^(i)(t) = Y^(i+1)(t) textdt qquad i = 0 dots q-1 \ntextd Y^(q)(t) = textcolor389826A Y(t) textdt + textcolor4063D8Gamma textdW(t) \nY(0) sim textcolorpurple mathcalN left( mu_0 Sigma_0 right) \nendaligned","category":"page"},{"location":"diffusions/","page":"Diffusion models and calibration","title":"Diffusion models and calibration","text":"Then Y^(i)(t) models the i-th derivative of y(t). In this section, we consider choices relating to the \"diffusion\" textcolor4063D8Gamma. If you're more interested in the drift matrix textcolor389826A check out the Priors section, and for info on the initial distribution textcolorpurple mathcalN left( mu_0 Sigma_0 right) check out the Initialization section.","category":"page"},{"location":"diffusions/#Diffusion-and-calibration","page":"Diffusion models and calibration","title":"Diffusion and calibration","text":"","category":"section"},{"location":"diffusions/","page":"Diffusion models and calibration","title":"Diffusion models and calibration","text":"We call textcolor4063D8Gamma the \"diffusion\" parameter. Since it is typically not known we need to estimate it; this is called \"calibration\".","category":"page"},{"location":"diffusions/","page":"Diffusion models and calibration","title":"Diffusion models and calibration","text":"There are a few different choices for how to model and estimate textcolor4063D8Gamma:","category":"page"},{"location":"diffusions/","page":"Diffusion models and calibration","title":"Diffusion models and calibration","text":"FixedDiffusion assumes an isotropic, time-fixed textcolor4063D8Gamma = sigma cdot I_d,\nDynamicDiffusion assumes an isotropic, time-varying textcolor4063D8Gamma(t) = sigma(t) cdot I_d (recommended),\nFixedMVDiffusion assumes a diagonal, time-fixed textcolor4063D8Gamma = operatornamediag(sigma_1 dots sigma_d),\nDynamicMVDiffusion assumes a diagonal, time-varying textcolor4063D8Gamma(t) = operatornamediag(sigma_1(t) dots sigma_d(t)).","category":"page"},{"location":"diffusions/","page":"Diffusion models and calibration","title":"Diffusion models and calibration","text":"Or more compactly:","category":"page"},{"location":"diffusions/","page":"Diffusion models and calibration","title":"Diffusion models and calibration","text":" Isotropic: Diagonal (only for the EK0)\nTime-varying DynamicDiffusion DynamicMVDiffusion\nTime-fixed FixedDiffusion FixedMVDiffusion","category":"page"},{"location":"diffusions/","page":"Diffusion models and calibration","title":"Diffusion models and calibration","text":"For more details on diffusions and calibration, check out this paper [1].","category":"page"},{"location":"diffusions/#API","page":"Diffusion models and calibration","title":"API","text":"","category":"section"},{"location":"diffusions/","page":"Diffusion models and calibration","title":"Diffusion models and calibration","text":"DynamicDiffusion\nFixedDiffusion\nDynamicMVDiffusion\nFixedMVDiffusion","category":"page"},{"location":"diffusions/#ProbNumDiffEq.DynamicDiffusion","page":"Diffusion models and calibration","title":"ProbNumDiffEq.DynamicDiffusion","text":"DynamicDiffusion()\n\nTime-varying, isotropic diffusion, which is quasi-maximum-likelihood-estimated at each step.\n\nThis is the recommended diffusion when using adaptive step-size selection, and in particular also when solving stiff systems.\n\n\n\n\n\n","category":"type"},{"location":"diffusions/#ProbNumDiffEq.FixedDiffusion","page":"Diffusion models and calibration","title":"ProbNumDiffEq.FixedDiffusion","text":"FixedDiffusion(; initial_diffusion=1.0, calibrate=true)\n\nTime-fixed, isotropic diffusion, which is (optionally) quasi-maximum-likelihood-estimated.\n\nThis is the recommended diffusion when using fixed steps.\n\nBy default with calibrate=true, all covariances are re-scaled at the end of the solve with the MLE diffusion. Set calibrate=false to skip this step, e.g. when setting the initial_diffusion and then estimating the diffusion outside of the solver (e.g. with Fenrir.jl).\n\n\n\n\n\n","category":"type"},{"location":"diffusions/#ProbNumDiffEq.DynamicMVDiffusion","page":"Diffusion models and calibration","title":"ProbNumDiffEq.DynamicMVDiffusion","text":"DynamicMVDiffusion()\n\nTime-varying, diagonal diffusion, which is quasi-maximum-likelihood-estimated at each step.\n\nOnly works with the EK0!\n\nA multi-variate version of DynamicDiffusion, where instead of an isotropic matrix, a diagonal matrix is estimated. This can be helpful to get more expressive posterior covariances when using the EK0, since the individual dimensions can be adjusted separately.\n\nSee also [1].\n\n\n\n\n\n","category":"type"},{"location":"diffusions/#ProbNumDiffEq.FixedMVDiffusion","page":"Diffusion models and calibration","title":"ProbNumDiffEq.FixedMVDiffusion","text":"FixedMVDiffusion(; initial_diffusion=1.0, calibrate=true)\n\nTime-fixed, diagonal diffusion, which is quasi-maximum-likelihood-estimated at each step.\n\nOnly works with the EK0!\n\nA multi-variate version of FixedDiffusion, where instead of an isotropic matrix, a diagonal matrix is estimated. This can be helpful to get more expressive posterior covariances when using the EK0, since the individual dimensions can be adjusted separately.\n\nSee also [1].\n\n\n\n\n\n","category":"type"},{"location":"diffusions/#diffusionrefs","page":"Diffusion models and calibration","title":"References","text":"","category":"section"},{"location":"diffusions/","page":"Diffusion models and calibration","title":"Diffusion models and calibration","text":"[1] N. Bosch, P. Hennig, F. Tronarp: Calibrated Adaptive Probabilistic ODE Solvers (2021) (link)","category":"page"},{"location":"priors/#Priors","page":"Priors","title":"Priors","text":"","category":"section"},{"location":"priors/","page":"Priors","title":"Priors","text":"We model the ODE solution y(t) with a Gauss–Markov prior. More precisely, let","category":"page"},{"location":"priors/","page":"Priors","title":"Priors","text":"beginaligned\nY(t) = left Y^(0)(t) Y^(1)(t) dots Y^(q)(t) right\nendaligned","category":"page"},{"location":"priors/","page":"Priors","title":"Priors","text":"be the solution to the SDE","category":"page"},{"location":"priors/","page":"Priors","title":"Priors","text":"beginaligned\ntextd Y^(i)(t) = Y^(i+1)(t) textdt qquad i = 0 dots q-1 \ntextd Y^(q)(t) = textcolor389826A Y(t) textdt + textcolor4063D8Gamma textdW(t) \nY(0) sim textcolorpurple mathcalN left( mu_0 Sigma_0 right) \nendaligned","category":"page"},{"location":"priors/","page":"Priors","title":"Priors","text":"Then Y^(i)(t) models the i-th derivative of y(t). In this section, we consider choices relating to the drift matrix textcolor389826A. If you're more interested in the diffusion textcolor4063D8Gamma check out the Diffusion models and calibration section, and for info on the initial distribution textcolorpurple mathcalN left( mu_0 Sigma_0 right) check out the Initialization section.","category":"page"},{"location":"priors/","page":"Priors","title":"Priors","text":"If you're unsure which prior to use, just stick to the integrated Wiener process prior IWP! This is also the default choice for all solvers. The other priors are rather experimental / niche at the time of writing.","category":"page"},{"location":"priors/#API","page":"Priors","title":"API","text":"","category":"section"},{"location":"priors/","page":"Priors","title":"Priors","text":"IWP\nIOUP\nMatern","category":"page"},{"location":"priors/#ProbNumDiffEq.IWP","page":"Priors","title":"ProbNumDiffEq.IWP","text":"IWP([wiener_process_dimension::Integer,] num_derivatives::Integer)\n\nIntegrated Wiener process.\n\nThis is the recommended prior! It is the most well-tested prior, both in this package and in the probabilistic numerics literature in general (see the references). It is also the prior that has the most efficient implementation.\n\nThe IWP can be created without specifying the dimension of the Wiener process, in which case it will be inferred from the dimension of the ODE during the solve. This is typically the preferred usage.\n\nIn math\n\nbeginaligned\ntextd Y^(i)(t) = Y^(i+1)(t) textdt qquad i = 0 dots q-1 \ntextd Y^(q)(t) = Gamma textdW(t)\nendaligned\n\nExamples\n\njulia> solve(prob, EK1(prior=IWP(2)))\n\n\n\n\n\n","category":"type"},{"location":"priors/#ProbNumDiffEq.IOUP","page":"Priors","title":"ProbNumDiffEq.IOUP","text":"IOUP([wiener_process_dimension::Integer,]\n num_derivatives::Integer,\n rate_parameter::Union{Number,AbstractVector,AbstractMatrix})\n\nIntegrated Ornstein–Uhlenbeck process.\n\nAs with the IWP, the IOUP can be created without specifying its dimension, in which case it will be inferred from the dimension of the ODE during the solve. This is typically the preferred usage. The rate parameter however always needs to be specified.\n\nIn math\n\nbeginaligned\ntextd Y^(i)(t) = Y^(i+1)(t) textdt qquad i = 0 dots q-1 \ntextd Y^(q)(t) = L Y^(q)(t) textdt + Gamma textdW(t)\nendaligned\n\nwhere L is the rate_parameter.\n\nExamples\n\njulia> solve(prob, EK1(prior=IOUP(2, -1)))\n\n\n\n\n\n","category":"type"},{"location":"priors/#ProbNumDiffEq.Matern","page":"Priors","title":"ProbNumDiffEq.Matern","text":"Matern([wiener_process_dimension::Integer,]\n num_derivatives::Integer,\n lengthscale::Number)\n\nMatern process.\n\nAs with the IWP, the Matern can be created without specifying its dimension, in which case it will be inferred from the dimension of the ODE during the solve. This is typically the preferred usage. The lengthscale parameter however always needs to be specified.\n\nIn math\n\nbeginaligned\ntextd Y^(i)(t) = Y^(i+1)(t) textdt qquad i = 0 dots q-1 \ntextd Y^(q)(t) = - sum_j=0^q left(\n beginpmatrix q+1 j endpmatrix\n left( fracsqrt2q - 1l right)^q-j\n Y^(j)(t) right) textdt + Gamma textdW(t)\nendaligned\n\nwhere l is the lengthscale.\n\nExamples\n\njulia> solve(prob, EK1(prior=Matern(2, 1)))\n\n\n\n\n\n","category":"type"},{"location":"tutorials/dynamical_odes/#Second-Order-ODEs-and-Energy-Preservation","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"","category":"section"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"In this tutorial we consider an energy-preserving, physical dynamical system, given by a second-order ODE.","category":"page"},{"location":"tutorials/dynamical_odes/#TL;DR:","page":"Second Order ODEs and Energy Preservation","title":"TL;DR:","text":"","category":"section"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"To efficiently solve second-order ODEs, just define the problem as a SecondOrderODEProblem.\nTo preserve constant quantities, use the ManifoldUpdate callback; same syntax as DiffEqCallback.jl's ManifoldProjection.","category":"page"},{"location":"tutorials/dynamical_odes/#Simulating-the-Hénon-Heiles-system","page":"Second Order ODEs and Energy Preservation","title":"Simulating the Hénon-Heiles system","text":"","category":"section"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"The Hénon-Heiles model describes the motion of a star around a galactic center, restricted to a plane. It is given by a second-order ODE","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"beginaligned\nddotx = - x - 2 x y \nddoty = y^2 - y - x^2\nendaligned","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"Our goal is to numerically simulate this system on a time span t in 0 T, starting with initial values x(0)=0, y(0) = 01, dotx(0) = 05, doty(0) = 0.","category":"page"},{"location":"tutorials/dynamical_odes/#Transforming-the-problem-into-a-first-order-ODE","page":"Second Order ODEs and Energy Preservation","title":"Transforming the problem into a first-order ODE","text":"","category":"section"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"A very common approach is to first transform the problem into a first-order ODE by introducing a new variable","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"u = dxdyxy","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"to obtain","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"beginaligned\ndotu_1(t) = - u_3 - 2 u_3 u_4 \ndotu_2(t) = u_4^2 - u_4 - u_4^2 \ndotu_3(t) = u_1 \ndotu_4(t) = u_2\nendaligned","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"This first-order ODE can then be solved using any conventional ODE solver - including our EK1:","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"using ProbNumDiffEq, Plots\n\nfunction Hénon_Heiles(du, u, p, t)\n du[1] = -u[3] - 2 * u[3] * u[4]\n du[2] = u[4]^2 - u[4] - u[3]^2\n du[3] = u[1]\n du[4] = u[2]\nend\nu0, du0 = [0.0, 0.1], [0.5, 0.0]\ntspan = (0.0, 100.0)\nprob = ODEProblem(Hénon_Heiles, [du0; u0], tspan)\nsol = solve(prob, EK1());\nplot(sol, vars=(3, 4)) # where `vars=(3,4)` is used to plot x agains y","category":"page"},{"location":"tutorials/dynamical_odes/#Solving-the-second-order-ODE-directly","page":"Second Order ODEs and Energy Preservation","title":"Solving the second-order ODE directly","text":"","category":"section"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"Instead of first transforming the problem, we can also solve it directly as a second-order ODE, by defining it as a SecondOrderODEProblem.","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"note: Note\nThe SecondOrderODEProblem type is not defined in ProbNumDiffEq.jl but is provided by SciMLBase.jl. For more information, check out the DifferentialEquations.jl documentation on Dynamical, Hamiltonian and 2nd Order ODE Problems.","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"function Hénon_Heiles2(ddu, du, u, p, t)\n ddu[1] = -u[1] - 2 * u[1] * u[2]\n ddu[2] = u[2]^2 - u[2] - u[1]^2\nend\nprob2 = SecondOrderODEProblem(Hénon_Heiles2, du0, u0, tspan)\nsol2 = solve(prob2, EK1());\nplot(sol2, vars=(3, 4))","category":"page"},{"location":"tutorials/dynamical_odes/#Benchmark:-Solving-second-order-ODEs-is-*faster*","page":"Second Order ODEs and Energy Preservation","title":"Benchmark: Solving second order ODEs is faster","text":"","category":"section"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"Solving second-order ODEs is not just a matter of convenience - in fact, SciMLBase's SecondOrderODEProblem is neatly designed in such a way that all the classic solvers from OrdinaryDiffEq.jl can handle it by solving the corresponding first-order ODE. But, transforming the ODE to first order increases the dimensionality of the problem, and comes therefore at increased computational cost; this also motivates classic specialized solvers for second-order ODEs.","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"The probabilistic numerical solvers from ProbNumDiffEq.jl have the same internal state representation for first and second order ODEs; all that changes is the measurement model [1]. As a result, we can use the EK1 both for first and second order ODEs, but it automatically specializes on the latter to provide a 2x performance boost:","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"julia> @btime solve(prob, EK1(order=3), adaptive=false, dt=1e-2);\n 766.312 ms (400362 allocations: 173.38 MiB)\n\njulia> @btime solve(prob2, EK1(order=4), adaptive=false, dt=1e-2);\n 388.301 ms (510676 allocations: 102.78 MiB)","category":"page"},{"location":"tutorials/dynamical_odes/#Energy-preservation","page":"Second Order ODEs and Energy Preservation","title":"Energy preservation","text":"","category":"section"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"In addition to the ODE given above, we know that the solution of the Hénon-Heiles model has to preserve energy over time. The total energy can be expressed as the sum of the potential and kinetic energies, given by","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"beginaligned\noperatornamePotentialEnergy(xy) = frac12 left( x^2 + y^2 + 2 x^2 y - frac2y^33 right) \noperatornameKineticEnergy(dotx doty) = frac12 left( dotx^2 + doty^2 right)\nendaligned","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"In code:","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"PotentialEnergy(x, y) = 1 // 2 * (x^2 + y^2 + 2x^2 * y - 2 // 3 * y^3)\nKineticEnergy(dx, dy) = 1 // 2 * (dx^2 + dy^2)\nE(dx, dy, x, y) = PotentialEnergy(x, y) + KineticEnergy(dx, dy)\nE(u) = E(u...); # convenient shorthand","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"So, let's have a look at how the total energy changes over time when we numerically simulate the Hénon-Heiles model over a long period of time: Standard solve","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"longprob = remake(prob2, tspan=(0.0, 1e3))\nlongsol = solve(longprob, EK1(smooth=false), dense=false)\nplot(longsol.t, E.(longsol.u))","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"It visibly loses energy over time, from an initial 0.12967 to a final 0.12899. Let's fix this to get a physically more meaningful solution.","category":"page"},{"location":"tutorials/dynamical_odes/#Energy-preservation-with-the-ManifoldUpdate-callback","page":"Second Order ODEs and Energy Preservation","title":"Energy preservation with the ManifoldUpdate callback","text":"","category":"section"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"In the language of ODE filters, preserving energy over time amounts to just another measurement model [1]. The most convenient way of updating on this additional zero measurement with ProbNumDiffEq.jl is with the ManifoldUpdate callback.","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"note: Note\nThe ManifoldUpdate callback can be thought of a probabilistic counterpart to the ManifoldProjection callback provided by DiffEqCallbacks.jl.","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"To do so, first define a (vector-valued) residual function, here chosen to be the difference between the current energy and the initial energy, and build a ManifoldUpdate callback","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"residual(u) = [E(u) - E(du0..., u0...)]\ncb = ManifoldUpdate(residual)","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"Then, solve the ODE with this callback","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"longsol_preserving = solve(longprob, EK1(smooth=false), dense=false, callback=cb)\nplot(longsol.t, E.(longsol.u))\nplot!(longsol_preserving.t, E.(longsol_preserving.u))","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"Voilà! With the ManifoldUpdate callback we could preserve the energy over time and obtain a more truthful probabilistic numerical long-term simulation of the Hénon-Heiles model.","category":"page"},{"location":"tutorials/dynamical_odes/#References","page":"Second Order ODEs and Energy Preservation","title":"References","text":"","category":"section"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"[1] N. Bosch, F. Tronarp, P. Hennig: Pick-and-Mix Information Operators for Probabilistic ODE Solvers (2022)","category":"page"},{"location":"benchmarks/multi-language-wrappers/#ProbNumDiffEq.jl-vs.-various-solver-packages","page":"Multi-Language Wrapper Benchmark","title":"ProbNumDiffEq.jl vs. various solver packages","text":"","category":"section"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"Adapted from SciMLBenchmarks.jl multi-language wrapper benchmark.","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"# Imports\nusing LinearAlgebra, Statistics, InteractiveUtils\nusing StaticArrays, DiffEqDevTools, ParameterizedFunctions, Plots, SciMLBase, OrdinaryDiffEq\nusing ODEInterface, ODEInterfaceDiffEq, Sundials, SciPyDiffEq, deSolveDiffEq, MATLABDiffEq, LSODA\nusing LoggingExtras\n\nusing ProbNumDiffEq","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"# Plotting theme\ntheme(:dao;\n markerstrokewidth=0.5,\n legend=:outertopright,\n bottom_margin=5Plots.mm,\n size = (1000, 400),\n)","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"# Constants used throughout this benchmark\nconst DENSE = false # used to decide if we smooth or not\nconst SAVE_EVERYSTEP = false;","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"_COLORS = Dict(\n \"Julia\" => :LightGreen,\n \"Julia (static)\" => :DarkGreen,\n \"Hairer\" => :Red,\n \"MATLAB\" => :Orange,\n \"SciPy\" => :Yellow,\n \"deSolve\" => :Blue,\n \"Sundials\" => :Purple,\n \"liblsoda\" => :Purple,\n \"ProbNumDiffEq\" => :Darkgray,\n)\ntocolor(n) = _COLORS[split(n, ':')[1]];","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"deprecated_filter(log_args) = !contains(log_args.message, \"deprecated\")\nfiltered_logger = ActiveFilteredLogger(deprecated_filter, global_logger());","category":"page"},{"location":"benchmarks/multi-language-wrappers/#Non-Stiff-Problem-1:-Lotka-Volterra","page":"Multi-Language Wrapper Benchmark","title":"Non-Stiff Problem 1: Lotka-Volterra","text":"","category":"section"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"f = @ode_def LotkaVolterra begin\n dx = a*x - b*x*y\n dy = -c*y + d*x*y\nend a b c d\np = [1.5, 1, 3, 1]\ntspan = (0.0, 10.0)\nu0 = [1.0, 1.0]\nprob = ODEProblem{true,SciMLBase.FullSpecialize()}(f,u0,tspan,p)\nstaticprob = ODEProblem{false,SciMLBase.FullSpecialize()}(f,SVector{2}(u0),tspan,SVector{4}(p))\n\nsol = solve(prob,Vern7(),abstol=1/10^14,reltol=1/10^14,dense=false)\ntest_sol = sol\nplot(sol, title=\"Lotka-Volterra Solution\", legend=false)","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"(Image: )","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"_setups = [\n \"Julia: DP5\" => Dict(:alg=>DP5())\n \"Julia: Tsit5\" => Dict(:alg=>Tsit5())\n \"Julia: Vern7\" => Dict(:alg=>Vern7())\n \"Hairer: dopri5\" => Dict(:alg=>ODEInterfaceDiffEq.dopri5())\n \"MATLAB: ode45\" => Dict(:alg=>MATLABDiffEq.ode45())\n \"MATLAB: ode113\" => Dict(:alg=>MATLABDiffEq.ode113())\n \"SciPy: RK45\" => Dict(:alg=>SciPyDiffEq.RK45())\n \"SciPy: LSODA\" => Dict(:alg=>SciPyDiffEq.LSODA())\n \"SciPy: odeint\" => Dict(:alg=>SciPyDiffEq.odeint())\n \"deSolve: lsoda\" => Dict(:alg=>deSolveDiffEq.lsoda())\n \"deSolve: ode45\" => Dict(:alg=>deSolveDiffEq.ode45())\n \"Sundials: Adams\" => Dict(:alg=>Sundials.CVODE_Adams())\n \"ProbNumDiffEq: EK0(2)\" => Dict(:alg=>EK0(order=2, smooth=DENSE))\n \"ProbNumDiffEq: EK0(3)\" => Dict(:alg=>EK0(order=3, smooth=DENSE))\n \"ProbNumDiffEq: EK1(3)\" => Dict(:alg=>EK1(order=3, smooth=DENSE))\n \"ProbNumDiffEq: EK1(5)\" => Dict(:alg=>EK1(order=5, smooth=DENSE))\n]\n\nlabels = first.(_setups)\nsetups = last.(_setups)\ncolors = tocolor.(labels) |> permutedims\n\nabstols = 1.0 ./ 10.0 .^ (6:13)\nreltols = 1.0 ./ 10.0 .^ (3:10)\n\nwp = with_logger(filtered_logger) do\n WorkPrecisionSet(\n [prob, staticprob], abstols, reltols, setups;\n names = labels,\n #print_names = true,\n appxsol = [test_sol, test_sol],\n dense = DENSE,\n save_everystep = SAVE_EVERYSTEP,\n numruns = 10,\n maxiters = Int(1e7),\n timeseries_errors = false,\n verbose = false,\n )\nend\n\nplot(\n wp,\n title = \"Non-stiff 1: Lotka-Volterra\",\n color = colors,\n xticks = 10.0 .^ (-16:1:5),\n yticks = 10.0 .^ (-6:1:5),\n)","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"(Image: )","category":"page"},{"location":"benchmarks/multi-language-wrappers/#Non-Stiff-Problem-2:-Rigid-Body","page":"Multi-Language Wrapper Benchmark","title":"Non-Stiff Problem 2: Rigid Body","text":"","category":"section"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"f = @ode_def RigidBodyBench begin\n dy1 = -2*y2*y3\n dy2 = 1.25*y1*y3\n dy3 = -0.5*y1*y2 + 0.25*sin(t)^2\nend\nu0 = [1.0;0.0;0.9]\ntspan = (0.0, 10.0)\nprob = ODEProblem{true,SciMLBase.FullSpecialize()}(f,u0,tspan)\nstaticprob = ODEProblem{false,SciMLBase.FullSpecialize()}(f,SVector{3}(u0),tspan)\nsol = solve(prob,Vern7(),abstol=1/10^14,reltol=1/10^14,dense=false)\ntest_sol = sol\nplot(sol, title=\"Rigid Body Solution\", legend=false)","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"(Image: )","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"_setups = [\n \"Julia: DP5\" => Dict(:alg=>DP5())\n \"Julia: Tsit5\" => Dict(:alg=>Tsit5())\n \"Julia: Vern7\" => Dict(:alg=>Vern7())\n \"Hairer: dopri5\" => Dict(:alg=>dopri5())\n \"MATLAB: ode45\" => Dict(:alg=>MATLABDiffEq.ode45())\n \"MATLAB: ode113\" => Dict(:alg=>MATLABDiffEq.ode113())\n \"SciPy: RK45\" => Dict(:alg=>SciPyDiffEq.RK45())\n \"SciPy: LSODA\" => Dict(:alg=>SciPyDiffEq.LSODA())\n \"SciPy: odeint\" => Dict(:alg=>SciPyDiffEq.odeint())\n \"deSolve: lsoda\" => Dict(:alg=>deSolveDiffEq.lsoda())\n \"deSolve: ode45\" => Dict(:alg=>deSolveDiffEq.ode45())\n \"Sundials: Adams\" => Dict(:alg=>CVODE_Adams())\n \"ProbNumDiffEq: EK0(2)\" => Dict(:alg=>EK0(order=2, smooth=DENSE))\n \"ProbNumDiffEq: EK0(3)\" => Dict(:alg=>EK0(order=3, smooth=DENSE))\n \"ProbNumDiffEq: EK1(3)\" => Dict(:alg=>EK1(order=3, smooth=DENSE))\n \"ProbNumDiffEq: EK1(5)\" => Dict(:alg=>EK1(order=5, smooth=DENSE))\n]\n\nlabels = first.(_setups)\nsetups = last.(_setups)\ncolors = tocolor.(labels) |> permutedims\n\nabstols = 1.0 ./ 10.0 .^ (6:13)\nreltols = 1.0 ./ 10.0 .^ (3:10)\n\nwp = with_logger(filtered_logger) do\n WorkPrecisionSet(\n [prob,staticprob], abstols, reltols, setups;\n names = labels,\n #print_names = true,\n appxsol = [test_sol, test_sol],\n dense = DENSE,\n save_everystep = SAVE_EVERYSTEP,\n numruns = 10,\n maxiters = Int(1e7),\n timeseries_errors = false,\n verbose = false\n )\nend\n\nplot(\n wp,\n title = \"Non-stiff 2: Rigid-Body\",\n color = colors,\n xticks = 10.0 .^ (-12:1:5),\n yticks = 10.0 .^ (-6:1:5),\n)","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"(Image: )","category":"page"},{"location":"benchmarks/multi-language-wrappers/#Stiff-Problem-1:-ROBER","page":"Multi-Language Wrapper Benchmark","title":"Stiff Problem 1: ROBER","text":"","category":"section"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"rober = @ode_def begin\n dy₁ = -k₁*y₁+k₃*y₂*y₃\n dy₂ = k₁*y₁-k₂*y₂^2-k₃*y₂*y₃\n dy₃ = k₂*y₂^2\nend k₁ k₂ k₃\nu0 = [1.0,0.0,0.0]\np = [0.04,3e7,1e4]\nprob = ODEProblem{true,SciMLBase.FullSpecialize()}(rober,u0,(0.0,1e5),p)\nstaticprob = ODEProblem{false,SciMLBase.FullSpecialize()}(rober,SVector{3}(u0),(0.0,1e5),SVector{3}(p))\nsol = solve(prob,CVODE_BDF(),abstol=1/10^14,reltol=1/10^14,dense=false)\ntest_sol = sol\nplot(sol, title=\"ROBER Solution\", legend=false, xlims=(1e0, 1e5))","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"(Image: )","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"_setups = [\n \"Julia: Rosenbrock23\" => Dict(:alg=>Rosenbrock23())\n \"Julia: Rodas4\" => Dict(:alg=>Rodas4())\n \"Julia: Rodas5\" => Dict(:alg=>Rodas5())\n \"Hairer: rodas\" => Dict(:alg=>rodas())\n \"Hairer: radau\" => Dict(:alg=>radau())\n \"MATLAB: ode23s\" => Dict(:alg=>MATLABDiffEq.ode23s())\n \"MATLAB: ode15s\" => Dict(:alg=>MATLABDiffEq.ode15s())\n \"SciPy: LSODA\" => Dict(:alg=>SciPyDiffEq.LSODA())\n \"SciPy: BDF\" => Dict(:alg=>SciPyDiffEq.BDF())\n \"SciPy: odeint\" => Dict(:alg=>SciPyDiffEq.odeint())\n \"deSolve: lsoda\" => Dict(:alg=>deSolveDiffEq.lsoda())\n \"Sundials: CVODE\" => Dict(:alg=>CVODE_BDF())\n \"ProbNumDiffEq: EK1(2)\" => Dict(:alg=>EK1(order=2, smooth=DENSE))\n \"ProbNumDiffEq: EK1(3)\" => Dict(:alg=>EK1(order=3, smooth=DENSE))\n \"ProbNumDiffEq: EK1(5)\" => Dict(:alg=>EK1(order=5, smooth=DENSE))\n]\n\nlabels = first.(_setups)\nsetups = last.(_setups)\ncolors = tocolor.(labels) |> permutedims\n\nabstols = 1.0 ./ 10.0 .^ (5:12)\nreltols = 1.0 ./ 10.0 .^ (2:9)\n\nwp = with_logger(filtered_logger) do\n WorkPrecisionSet(\n [prob, staticprob], abstols, reltols, setups;\n names = labels,\n #print_names = true,\n dense = DENSE,\n verbose = false,\n save_everystep = SAVE_EVERYSTEP,\n appxsol = [test_sol, test_sol],\n maxiters=Int(1e5)\n )\nend\n\nplot(\n wp,\n title = \"Stiff 1: ROBER\",\n color = colors,\n xticks = 10.0 .^ (-16:1:4),\n yticks = 10.0 .^ (-6:1:5),\n)","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"(Image: )","category":"page"},{"location":"benchmarks/multi-language-wrappers/#Stiff-Problem-2:-HIRES","page":"Multi-Language Wrapper Benchmark","title":"Stiff Problem 2: HIRES","text":"","category":"section"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"f = @ode_def Hires begin\n dy1 = -1.71*y1 + 0.43*y2 + 8.32*y3 + 0.0007\n dy2 = 1.71*y1 - 8.75*y2\n dy3 = -10.03*y3 + 0.43*y4 + 0.035*y5\n dy4 = 8.32*y2 + 1.71*y3 - 1.12*y4\n dy5 = -1.745*y5 + 0.43*y6 + 0.43*y7\n dy6 = -280.0*y6*y8 + 0.69*y4 + 1.71*y5 -\n 0.43*y6 + 0.69*y7\n dy7 = 280.0*y6*y8 - 1.81*y7\n dy8 = -280.0*y6*y8 + 1.81*y7\nend\n\nu0 = zeros(8)\nu0[1] = 1\nu0[8] = 0.0057\nprob = ODEProblem{true,SciMLBase.FullSpecialize()}(f,u0,(0.0,321.8122))\nstaticprob = ODEProblem{false,SciMLBase.FullSpecialize()}(f,SVector{8}(u0),(0.0,321.8122))\n\nsol = solve(prob,Rodas5(),abstol=1/10^14,reltol=1/10^14, dense=false)\ntest_sol = sol\nplot(sol, title=\"ROBER Solution\", legend=false)","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"(Image: )","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"_setups = [\n \"Julia: Rosenbrock23\" => Dict(:alg=>Rosenbrock23())\n \"Julia: Rodas4\" => Dict(:alg=>Rodas4())\n \"Julia: radau\" => Dict(:alg=>RadauIIA5())\n \"Hairer: rodas\" => Dict(:alg=>rodas())\n \"Hairer: radau\" => Dict(:alg=>radau())\n \"MATLAB: ode23s\" => Dict(:alg=>MATLABDiffEq.ode23s())\n \"MATLAB: ode15s\" => Dict(:alg=>MATLABDiffEq.ode15s())\n \"SciPy: LSODA\" => Dict(:alg=>SciPyDiffEq.LSODA())\n \"SciPy: BDF\" => Dict(:alg=>SciPyDiffEq.BDF())\n \"SciPy: odeint\" => Dict(:alg=>SciPyDiffEq.odeint())\n \"deSolve: lsoda\" => Dict(:alg=>deSolveDiffEq.lsoda())\n \"Sundials: CVODE\" => Dict(:alg=>CVODE_BDF())\n \"ProbNumDiffEq: EK1(2)\" => Dict(:alg=>EK1(order=2, smooth=DENSE))\n \"ProbNumDiffEq: EK1(3)\" => Dict(:alg=>EK1(order=3, smooth=DENSE))\n \"ProbNumDiffEq: EK1(5)\" => Dict(:alg=>EK1(order=5, smooth=DENSE))\n]\n\nlabels = first.(_setups)\nsetups = last.(_setups)\ncolors = tocolor.(labels) |> permutedims\n\nabstols = 1.0 ./ 10.0 .^ (5:10)\nreltols = 1.0 ./ 10.0 .^ (1:6)\n\nwp = with_logger(filtered_logger) do\n WorkPrecisionSet(\n [prob, staticprob], abstols, reltols, setups;\n names = labels,\n #print_names = true,\n dense = false,\n verbose = false,\n save_everystep = false,\n appxsol = [test_sol, test_sol],\n maxiters = Int(1e5),\n numruns=100\n )\nend\n\nplot(\n wp,\n title = \"Stiff 2: Hires\",\n color=colors,\n xticks = 10.0 .^ (-16:1:4),\n yticks = 10.0 .^ (-6:1:5),\n)","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"(Image: )","category":"page"},{"location":"benchmarks/multi-language-wrappers/#Appendix","page":"Multi-Language Wrapper Benchmark","title":"Appendix","text":"","category":"section"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"Computer information:","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"InteractiveUtils.versioninfo()","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"Julia Version 1.8.5\nCommit 17cfb8e65ea (2023-01-08 06:45 UTC)\nPlatform Info:\n OS: Linux (x86_64-linux-gnu)\n CPU: 12 × Intel(R) Core(TM) i7-6800K CPU @ 3.40GHz\n WORD_SIZE: 64\n LIBM: libopenlibm\n LLVM: libLLVM-13.0.1 (ORCJIT, broadwell)\n Threads: 12 on 12 virtual cores\nEnvironment:\n JULIA_NUM_THREADS = auto\n JULIA_STACKTRACE_MINIMAL = true","category":"page"},{"location":"tutorials/getting_started/#Solving-ODEs-with-Probabilistic-Numerics","page":"Getting Started","title":"Solving ODEs with Probabilistic Numerics","text":"","category":"section"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"In this tutorial we solve a simple non-linear ordinary differential equation (ODE) with the probabilistic numerical ODE solvers implemented in this package.","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"note: Note\nIf you never used DifferentialEquations.jl, check out their \"Getting Started with Differential Equations in Julia\" tutorial. It explains how to define and solve ODE problems and how to analyze the solution, so it's a great starting point. Most of ProbNumDiffEq.jl works exaclty as you would expect from DifferentialEquations.jl – just with some added uncertainties and related functionality on top!","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"In this tutorial, we consider a Fitzhugh-Nagumo model described by an ODE of the form","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"beginaligned\ndoty_1 = c (y_1 - fracy_1^33 + y_2) \ndoty_2 = -frac1c (y_1 - a - b y_2)\nendaligned","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"on a time span t in 0 T, with initial value y(0) = y_0. In the following, we","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"define the problem with explicit choices of initial values, integration domains, and parameters,\nsolve the problem with our ODE filters, and\nvisualize the results and the corresponding uncertainties.","category":"page"},{"location":"tutorials/getting_started/#TL;DR:-Just-use-DifferentialEquations.jl-with-the-EK1-algorithm","page":"Getting Started","title":"TL;DR: Just use DifferentialEquations.jl with the EK1 algorithm","text":"","category":"section"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"using ProbNumDiffEq, Plots\n\nfunction fitz(du, u, p, t)\n a, b, c = p\n du[1] = c * (u[1] - u[1]^3 / 3 + u[2])\n du[2] = -(1 / c) * (u[1] - a - b * u[2])\nend\nu0 = [-1.0; 1.0]\ntspan = (0.0, 20.0)\np = (0.2, 0.2, 3.0)\nprob = ODEProblem(fitz, u0, tspan, p)\n\nusing Logging; Logging.disable_logging(Logging.Warn); # hide\nsol = solve(prob, EK1())\nLogging.disable_logging(Logging.Debug) # hide\nplot(sol)","category":"page"},{"location":"tutorials/getting_started/#Step-1:-Define-the-problem","page":"Getting Started","title":"Step 1: Define the problem","text":"","category":"section"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"First, import ProbNumDiffEq.jl","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"using ProbNumDiffEq","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"Then, set up the ODEProblem exactly as you would in DifferentialEquations.jl. Define the vector field","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"function fitz(du, u, p, t)\n a, b, c = p\n du[1] = c * (u[1] - u[1]^3 / 3 + u[2])\n du[2] = -(1 / c) * (u[1] - a - b * u[2])\nend\nnothing # hide","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"and then the ODEProblem, with initial value u0, time span tspan, and parameters p","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"u0 = [-1.0; 1.0]\ntspan = (0.0, 20.0)\np = (0.2, 0.2, 3.0)\nprob = ODEProblem(fitz, u0, tspan, p)\nnothing # hide","category":"page"},{"location":"tutorials/getting_started/#Step-2:-Solve-the-problem","page":"Getting Started","title":"Step 2: Solve the problem","text":"","category":"section"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"To solve the ODE we just use DifferentialEquations.jl's solve interface, together with one of the algorithms implemented in this package. For now, let's use EK1:","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"sol = solve(prob, EK1())\n# nothing # hide","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"That's it! we just computed a probabilistic numerical ODE solution!","category":"page"},{"location":"tutorials/getting_started/#Step-3:-Analyze-the-solution","page":"Getting Started","title":"Step 3: Analyze the solution","text":"","category":"section"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"Let's plot the result with Plots.jl.","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"using Plots\nplot(sol)","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"Looks good! Looks like the EK1 managed to solve the Fitzhugh-Nagumo problem quite well.","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"tip: Tip\nTo learn more about plotting ODE solutions, check out the plotting tutorial for DifferentialEquations.jl + Plots.jl provided here. Most of that works exactly as expected with ProbNumDiffEq.jl.","category":"page"},{"location":"tutorials/getting_started/#Plot-the-probabilistic-error-estimates","page":"Getting Started","title":"Plot the probabilistic error estimates","text":"","category":"section"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"The plot above looks like a standard ODE solution – but it's not! The numerical errors are just so small that we can't see them in the plot, and the probabilistic error estimates are too. We can visualize them by plotting the errors and error estimates directly:","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"using OrdinaryDiffEq, Statistics\nreference = solve(prob, Vern9(), abstol=1e-9, reltol=1e-9, saveat=sol.t)\nerrors = reduce(hcat, mean.(sol.pu) .- reference.u)'\nerror_estimates = reduce(hcat, std.(sol.pu))'\nplot(sol.t, errors, label=\"error\", color=[1 2], xlabel=\"t\", ylabel=\"err\")\nplot!(sol.t, zero(errors), ribbon=3error_estimates, label=\"error estimate\",\n color=[1 2], alpha=0.2)","category":"page"},{"location":"tutorials/getting_started/#More-about-the-ProbabilisticODESolution","page":"Getting Started","title":"More about the ProbabilisticODESolution","text":"","category":"section"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"The solution object returned by ProbNumDiffEq.jl mostly behaves just like any other ODESolution in DifferentialEquations.jl – with some added uncertainties and related functionality on top. So, sol can be indexed","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"sol[1]\nsol[end]","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"and has fields sol.t and sol.u which store the time points and mean estimates:","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"sol.t[end]\nsol.u[end]","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"But since sol is a probabilistic numerical ODE solution, it contains a Gaussian distributions over solution values. The marginals of this posterior are stored in sol.pu:","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"sol.pu[end]","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"You can compute means, covariances, and standard deviations via Statistics.jl:","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"using Statistics\nmean(sol.pu[5])\ncov(sol.pu[5])\nstd(sol.pu[5])","category":"page"},{"location":"tutorials/getting_started/#Dense-output","page":"Getting Started","title":"Dense output","text":"","category":"section"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"Probabilistic numerical ODE solvers approximate the posterior distribution","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"p Big( y(t) big y(0) = y_0 doty(t_i) = f_theta(y(t_i) t_i) Big)","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"which describes a posterior not just for the discrete steps but for any t in the continuous space t in 0 T; in classic ODE solvers, this is also known as \"interpolation\" or \"dense output\". The probabilistic solutions returned by our solvers can be interpolated as usual by treating them as functions, but they return Gaussian distributions","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"sol(0.45)\nmean(sol(0.45))","category":"page"},{"location":"tutorials/getting_started/#Next-steps","page":"Getting Started","title":"Next steps","text":"","category":"section"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"Check out one of the other tutorials:","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"\"Second Order ODEs and Energy Preservation\" explains how to solve second-order ODEs more efficiently while also better perserving energy or other conserved quantities;\n\"Solving DAEs with Probabilistic Numerics\" demonstrates how to solve differential algebraic equatios in a probabilistic numerical way.","category":"page"},{"location":"tutorials/fenrir/#Parameter-Inference-with-ProbNumDiffEq.jl-and-Fenrir.jl","page":"Parameter Inference","title":"Parameter Inference with ProbNumDiffEq.jl and Fenrir.jl","text":"","category":"section"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"note: Note\nThis is mostly just a copy from the tutorial included in the Fenrir.jl documentation, so have a look there too!","category":"page"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"using LinearAlgebra\nusing OrdinaryDiffEq, ProbNumDiffEq, Plots\nusing Fenrir\nusing Optimization, OptimizationOptimJL\nstack(x) = copy(reduce(hcat, x)') # convenient\nnothing # hide","category":"page"},{"location":"tutorials/fenrir/#The-parameter-inference-problem-in-general","page":"Parameter Inference","title":"The parameter inference problem in general","text":"","category":"section"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"Let's assume we have an initial value problem (IVP)","category":"page"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"beginaligned\ndoty = f_theta(y t) qquad y(t_0) = y_0\nendaligned","category":"page"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"which we observe through a set mathcalD = u(t_n)_n=1^N of noisy data points","category":"page"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"beginaligned\nu(t_n) = H y(t_n) + v_n qquad v_n sim mathcalN(0 R)\nendaligned","category":"page"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"The question of interest is: How can we compute the marginal likelihood p(mathcalD mid theta)? Short answer: We can't. It's intractable, because computing the true IVP solution exactly y(t) is intractable. What we can do however is compute an approximate marginal likelihood. This is what Fenrir.jl provides. For details, check out the paper.","category":"page"},{"location":"tutorials/fenrir/#The-specific-problem,-in-code","page":"Parameter Inference","title":"The specific problem, in code","text":"","category":"section"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"Let's assume that the true underlying dynamics are given by a FitzHugh-Nagumo model","category":"page"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"function f(du, u, p, t)\n a, b, c = p\n du[1] = c*(u[1] - u[1]^3/3 + u[2])\n du[2] = -(1/c)*(u[1] - a - b*u[2])\nend\nu0 = [-1.0, 1.0]\ntspan = (0.0, 20.0)\np = (0.2, 0.2, 3.0)\ntrue_prob = ODEProblem(f, u0, tspan, p)","category":"page"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"from which we generate some artificial noisy data","category":"page"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"true_sol = solve(true_prob, Vern9(), abstol=1e-10, reltol=1e-10)\n\ntimes = 1:0.5:20\nobservation_noise_var = 1e-1\nodedata = [true_sol(t) .+ sqrt(observation_noise_var) * randn(length(u0)) for t in times]\n\nplot(true_sol, color=:black, linestyle=:dash, label=[\"True Solution\" \"\"])\nscatter!(times, stack(odedata), markersize=2, markerstrokewidth=0.1, color=1, label=[\"Noisy Data\" \"\"])","category":"page"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"Our goal is then to recover the true parameter p (and thus also the true trajectoy plotted above) the noisy data.","category":"page"},{"location":"tutorials/fenrir/#Computing-the-negative-log-likelihood","page":"Parameter Inference","title":"Computing the negative log-likelihood","text":"","category":"section"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"To do parameter inference - be it maximum-likelihod, maximum a posteriori, or full Bayesian inference with MCMC - we need to evaluate the likelihood of given a parameter estimate theta_textest. This is exactly what Fenrir.jl's fenrir_nll provides:","category":"page"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"p_est = (0.1, 0.1, 2.0)\nprob = remake(true_prob, p=p_est)\ndata = (t=times, u=odedata)\nκ² = 1e10\nnll, _, _ = fenrir_nll(prob, data, observation_noise_var, κ²; dt=1e-1)\nnll","category":"page"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"This is the negative marginal log-likelihood of the parameter p_est. You can use it as any other NLL: Optimize it to compute maximum-likelihood estimates or MAPs, or plug it into MCMC to sample from the posterior. In our paper we compute MLEs by pairing Fenrir with Optimization.jl and ForwardDiff.jl. Let's quickly explore how to do this next.","category":"page"},{"location":"tutorials/fenrir/#Maximum-likelihood-parameter-inference","page":"Parameter Inference","title":"Maximum-likelihood parameter inference","text":"","category":"section"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"To compute a maximum-likelihood estimate (MLE), we just need to maximize theta to p(mathcalD mid theta) - that is, minimize the nll from above. We use Optimization.jl for this. First, define a loss function and create an OptimizationProblem","category":"page"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"function loss(x, _)\n ode_params = x[begin:end-1]\n prob = remake(true_prob, p=ode_params)\n κ² = exp(x[end]) # the diffusion parameter of the EK1\n return fenrir_nll(prob, data, observation_noise_var, κ²; dt=1e-1)\nend\n\nfun = OptimizationFunction(loss, Optimization.AutoForwardDiff())\noptprob = OptimizationProblem(\n fun, [p_est..., 1e0];\n lb=[0.0, 0.0, 0.0, -10], ub=[1.0, 1.0, 5.0, 20] # lower and upper bounds\n)","category":"page"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"Then, just solve it! Here we use LBFGS:","category":"page"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"optsol = solve(optprob, LBFGS())\np_mle = optsol.u[1:3]\np_mle # hide","category":"page"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"Success! The computed MLE is quite close to the true parameter which we used to generate the data. As a final step, let's plot the true solution, the data, and the result of the MLE:","category":"page"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"plot(true_sol, color=:black, linestyle=:dash, label=[\"True Solution\" \"\"])\nscatter!(times, stack(odedata), markersize=2, markerstrokewidth=0.1, color=1, label=[\"Noisy Data\" \"\"])\nmle_sol = solve(remake(true_prob, p=p_mle), EK1())\nplot!(mle_sol, color=3, label=[\"MLE-parameter Solution\" \"\"])","category":"page"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"Looks good!","category":"page"},{"location":"tutorials/fenrir/#Reference","page":"Parameter Inference","title":"Reference","text":"","category":"section"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"[1] F. Tronarp, N. Bosch, P. Hennig: Fenrir: Fenrir: Physics-Enhanced Regression for Initial Value Problems (2022)","category":"page"},{"location":"#Probabilistic-Numerical-Differential-Equation-Solvers","page":"Home","title":"Probabilistic Numerical Differential Equation Solvers","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"(Image: Banner)","category":"page"},{"location":"","page":"Home","title":"Home","text":"ProbNumDiffEq.jl provides probabilistic numerical solvers to the DifferentialEquations.jl ecosystem. The implemented ODE filters solve differential equations via Bayesian filtering and smoothing and compute not just a single point estimate of the true solution, but a posterior distribution that contains an estimate of its numerical approximation error.","category":"page"},{"location":"","page":"Home","title":"Home","text":"For a short intro video, check out our poster presentation at JuliaCon2021.","category":"page"},{"location":"#Installation","page":"Home","title":"Installation","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"Run Julia, enter ] to bring up Julia's package manager, and add the ProbNumDiffEq.jl package:","category":"page"},{"location":"","page":"Home","title":"Home","text":"julia> ]\n(v1.9) pkg> add ProbNumDiffEq","category":"page"},{"location":"#Getting-Started","page":"Home","title":"Getting Started","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"For a quick introduction check out the \"Solving ODEs with Probabilistic Numerics\" tutorial.","category":"page"},{"location":"#Features","page":"Home","title":"Features","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"Two extended Kalman filtering-based probabilistic solvers: the explicit EK0 and semi-implicit EK1.\nAdaptive step-size selection with PI control; fully compatible with DifferentialEquations.jl's timestepping options\nOnline uncertainty calibration for multiple different diffusion models (see \"Diffusion models and calibration\")\nDense output\nSampling from the solution\nCallback support\nConvenient plotting through a Plots.jl recipe\nAutomatic differentiation via ForwardDiff.jl\nArbitrary precision via Julia's built-in arbitrary precision arithmetic\nSpecialized solvers for second-order ODEs (see Second Order ODEs and Energy Preservation)\nCompatible with DAEs in mass-matrix ODE form (see Solving DAEs with Probabilistic Numerics)","category":"page"},{"location":"#references","page":"Home","title":"References","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"The main references for this package include:","category":"page"},{"location":"","page":"Home","title":"Home","text":"N. Bosch, P. Hennig, F. Tronarp: Probabilistic Exponential Integrators (2023)\nN. Bosch, F. Tronarp, P. Hennig: Pick-and-Mix Information Operators for Probabilistic ODE Solvers (2022)\nN. Krämer, N. Bosch, J. Schmidt, P. Hennig: Probabilistic ODE Solutions in Millions of Dimensions (2022)\nN. Bosch, P. Hennig, F. Tronarp: Calibrated Adaptive Probabilistic ODE Solvers (2021)\nF. Tronarp, S. Särkkä, and P. Hennig: Bayesian ODE Solvers: The Maximum A Posteriori Estimate (2021)\nN. Krämer, P. Hennig: Stable Implementation of Probabilistic ODE Solvers (2020)\nH. Kersting, T. J. Sullivan, and P. Hennig: Convergence Rates of Gaussian Ode Filters (2020)\nF. Tronarp, H. Kersting, S. Särkkä, and P. Hennig: Probabilistic Solutions To Ordinary Differential Equations As Non-Linear Bayesian Filtering: A New Perspective (2019)\nM. Schober, S. Särkkä, and P. Hennig: A Probabilistic Model for the Numerical Solution of Initial Value Problems (2018)","category":"page"},{"location":"","page":"Home","title":"Home","text":"More references on ODE filters and on probabilistic numerics in general can be found on probabilistic-numerics.org .","category":"page"},{"location":"#Related-packages","page":"Home","title":"Related packages","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"probdiffeq: Fast and feature-rich filtering-based probabilistic ODE solvers in JAX.\nProbNum: Probabilistic numerics in Python. It has not only probabilistic ODE solvers, but also probabilistic linear solvers, Bayesian quadrature, and many filtering and smoothing implementations.\nFenrir.jl: Parameter-inference in ODEs with probabilistic ODE solvers. This package builds on ProbNumDiffEq.jl to provide a negative marginal log-likelihood function, which can then be used with an optimizer or with MCMC for parameter inference.","category":"page"},{"location":"implementation/#Solver-Implementation-via-OrdinaryDiffEq.jl","page":"Implementation via OrdinaryDiffEq.jl","title":"Solver Implementation via OrdinaryDiffEq.jl","text":"","category":"section"},{"location":"implementation/","page":"Implementation via OrdinaryDiffEq.jl","title":"Implementation via OrdinaryDiffEq.jl","text":"ProbNumDiffEq.jl builds directly on OrdinaryDiffEq.jl to benefit from its iterator interface, flexible step-size control, and efficient Jacobian calculations. But, this requires extending non-public APIs. This page is meant to provide an overview on which parts exactly ProbNumDiffEq.jl builds on.","category":"page"},{"location":"implementation/","page":"Implementation via OrdinaryDiffEq.jl","title":"Implementation via OrdinaryDiffEq.jl","text":"For more discussion on the pros and cons of building on OrdinaryDiffEq.jl, see this thread on discourse.","category":"page"},{"location":"implementation/#Building-on-OrdinaryDiffEq.jl","page":"Implementation via OrdinaryDiffEq.jl","title":"Building on OrdinaryDiffEq.jl","text":"","category":"section"},{"location":"implementation/","page":"Implementation via OrdinaryDiffEq.jl","title":"Implementation via OrdinaryDiffEq.jl","text":"ProbNumDiffEq.jl shares most of OrdinaryDiffEq.jl's implementation. In particular:","category":"page"},{"location":"implementation/","page":"Implementation via OrdinaryDiffEq.jl","title":"Implementation via OrdinaryDiffEq.jl","text":"OrdinaryDiffEq.__init builds the cache and the integrator, and calls OrdinaryDiffEq.initialize!\nOrdinaryDiffEq.solve! implements the actual iterator structure, with\nOrdinaryDiffEq.loopheader!\nOrdinaryDiffEq.perform_step!\nOrdinaryDiffEq.loopfooter!\nOrdinaryDiffEq.postamble!","category":"page"},{"location":"implementation/","page":"Implementation via OrdinaryDiffEq.jl","title":"Implementation via OrdinaryDiffEq.jl","text":"ProbNumDiffEq.jl builds around this structure and overloads some of the parts:","category":"page"},{"location":"implementation/","page":"Implementation via OrdinaryDiffEq.jl","title":"Implementation via OrdinaryDiffEq.jl","text":"Algorithms: EK0/EK1 <: AbstractEK <: OrdinaryDiffEq.OrdinaryDiffEqAdaptiveAlgorithm\n./src/algorithms.jl provides the algorithms themselves\n./src/alg_utils.jl implements many traits (e.g. relating to autodiff, implicitness, step-size control)\nCache: EKCache <: AbstractODEFilterCache <: OrdinaryDiffEq.OrdinaryDiffEqCache\n./src/caches.jl implements the cache and its main constructor: OrdinaryDiffEq.alg_cache\nInitialization and perform_step!: via OrdinaryDiffEq.initialize! and OrdinaryDiffEq.perform_step!. Implemented in ./src/perform_step.jl.\nCustom postamble by overloading OrdinaryDiffEq.postamble! (which should always call OrdinaryDiffEq._postamble!). This is where we do the \"smoothing\" of the solution. Implemented in ./src/integrator_utils.jl. \nCustom saving by overloading OrdinaryDiffEq.savevalues! (which should always call OrdinaryDiffEq._savevalues!). Implemented in ./src/integrator_utils.jl.","category":"page"},{"location":"implementation/#Building-on-DiffEqBase.jl","page":"Implementation via OrdinaryDiffEq.jl","title":"Building on DiffEqBase.jl","text":"","category":"section"},{"location":"implementation/","page":"Implementation via OrdinaryDiffEq.jl","title":"Implementation via OrdinaryDiffEq.jl","text":"DiffEqBase.__init is currently overloaded to transform OOP problems into IIP problems (in ./src/solve.jl).\nThe solution object: ProbODESolution <: AbstractProbODESolution <: DiffEqBase.AbstractODESolution\n./src/solution.jl implements the main parts. Note that the main constructor DiffEqBase.build_solution is called by OrdinaryDiffEq.__init - so OrdinaryDiffEq.jl has control over its inputs.\nThere is also MeanProbODESolution <: DiffEqBase.AbstractODESolution: It allows handling the mean of a probabilistic ODE solution the same way one would handle any \"standard\" ODE solution - e.g. it is compatible with DiffEqDevTools.appxtrue.\nAbstractODEFilterPosterior <: DiffEqBase.AbstractDiffEqInterpolation is the current interpolant, but it does not actually fully handle the interpolation right now. This part might be subject to change soon.\nPlot recipe in ./src/solution_plotting.jl\nSampling in ./src/solution_sampling.jl\nDiffEqBase.prepare_alg(::EK1{0}); closely follows a similar function implemented in OrdinaryDiffEq.jl ./src/alg_utils.jl\nthis also required DiffEqBase.remake(::EK1)","category":"page"},{"location":"implementation/#Other-packages","page":"Implementation via OrdinaryDiffEq.jl","title":"Other packages","text":"","category":"section"},{"location":"implementation/","page":"Implementation via OrdinaryDiffEq.jl","title":"Implementation via OrdinaryDiffEq.jl","text":"DiffEqDevTools.appxtrue is overloaded to work with ProbODESolution (by just doing mean(sol)). This also enables DiffEqDevTools.WorkPrecision to work out of th box.","category":"page"}]
+[{"location":"references/#references","page":"References","title":"References","text":"","category":"section"},{"location":"references/","page":"References","title":"References","text":"*","category":"page"},{"location":"benchmarks/vanderpol/#Van-der-Pol-benchmark","page":"Stiff ODE: Van der Pol","title":"Van der Pol benchmark","text":"","category":"section"},{"location":"benchmarks/vanderpol/","page":"Stiff ODE: Van der Pol","title":"Stiff ODE: Van der Pol","text":"using LinearAlgebra, Statistics, InteractiveUtils\nusing DiffEqDevTools, ParameterizedFunctions, SciMLBase, OrdinaryDiffEq, Plots\nusing ProbNumDiffEq\n\n# Plotting theme\ntheme(:dao;\n markerstrokewidth=0.5,\n legend=:outertopright,\n bottom_margin=5Plots.mm,\n size = (1000, 400),\n)","category":"page"},{"location":"benchmarks/vanderpol/#Van-der-Pol-problem-definition","page":"Stiff ODE: Van der Pol","title":"Van der Pol problem definition","text":"","category":"section"},{"location":"benchmarks/vanderpol/","page":"Stiff ODE: Van der Pol","title":"Stiff ODE: Van der Pol","text":"function vanderpol!(du, u, p, t)\n du[1] = u[2]\n du[2] = p[1] * ((1 - u[1]^2) * u[2] - u[1])\nend\np = [1e5]\ntspan = (0.0, 6.3)\nu0 = [2.0, 0.0]\nprob = ODEProblem(vanderpol!, u0, tspan, p)\n\ntest_sol = solve(prob, RadauIIA5(), abstol=1/10^14, reltol=1/10^14, dense=false)\nplot(test_sol, title=\"Van der Pol Solution\", legend=false, ylims=(-2.5, 2.5))","category":"page"},{"location":"benchmarks/vanderpol/","page":"Stiff ODE: Van der Pol","title":"Stiff ODE: Van der Pol","text":"(Image: )","category":"page"},{"location":"benchmarks/vanderpol/#EK1-accross-orders","page":"Stiff ODE: Van der Pol","title":"EK1 accross orders","text":"","category":"section"},{"location":"benchmarks/vanderpol/","page":"Stiff ODE: Van der Pol","title":"Stiff ODE: Van der Pol","text":"DENSE = false;\nSAVE_EVERYSTEP = false;\n\n_setups = [\n \"EK1($order)\" => Dict(:alg => EK1(order=order, smooth=DENSE))\n for order in 2:6\n]\n\nlabels = first.(_setups)\nsetups = last.(_setups)\n\nabstols = 1.0 ./ 10.0 .^ (6:13)\nreltols = 1.0 ./ 10.0 .^ (3:10)\n\nwp = WorkPrecisionSet(\n prob, abstols, reltols, setups;\n names = labels,\n #print_names = true,\n appxsol = test_sol,\n dense = DENSE,\n save_everystep = SAVE_EVERYSTEP,\n numruns = 10,\n maxiters = Int(1e7),\n timeseries_errors = false,\n verbose = false,\n)\n\nplot(wp, palette=Plots.palette([:blue, :red], length(_setups)), xticks = 10.0 .^ (-16:1:5))","category":"page"},{"location":"benchmarks/vanderpol/","page":"Stiff ODE: Van der Pol","title":"Stiff ODE: Van der Pol","text":"(Image: )","category":"page"},{"location":"benchmarks/vanderpol/#Solving-the-first-vs-second-order-ODE","page":"Stiff ODE: Van der Pol","title":"Solving the first- vs second-order ODE","text":"","category":"section"},{"location":"benchmarks/vanderpol/","page":"Stiff ODE: Van der Pol","title":"Stiff ODE: Van der Pol","text":"function vanderpol2!(ddu, du, u, p, t)\n ddu[1] = p[1] * ((1 - u[1]^2) * du[1] - u[1])\nend\np = [1e5]\ntspan = (0.0, 6.3)\nu0 = [2.0]\ndu0 = [0.0]\nprob2 = SecondOrderODEProblem(vanderpol2!, du0, u0, tspan, p)\n\ntest_sol2 = solve(prob2, RadauIIA5(), abstol=1/10^14, reltol=1/10^14, dense=false)\nplot(test_sol2, title=\"Van der Pol Solution (2nd order)\", legend=false, ylims=(-2.5, 2.5))","category":"page"},{"location":"benchmarks/vanderpol/","page":"Stiff ODE: Van der Pol","title":"Stiff ODE: Van der Pol","text":"(Image: )","category":"page"},{"location":"benchmarks/vanderpol/","page":"Stiff ODE: Van der Pol","title":"Stiff ODE: Van der Pol","text":"DENSE = false;\nSAVE_EVERYSTEP = false;\n\n_setups = [\n \"EK1(2) 1st order\" => Dict(:alg => EK1(order=2, smooth=DENSE))\n \"EK1(3) 1st order\" => Dict(:alg => EK1(order=3, smooth=DENSE))\n \"EK1(4) 1st order\" => Dict(:alg => EK1(order=4, smooth=DENSE))\n \"EK1(5) 1st order\" => Dict(:alg => EK1(order=5, smooth=DENSE))\n \"EK1(3) 2nd order\" => Dict(:prob_choice => 2, :alg => EK1(order=3, smooth=DENSE))\n \"EK1(4) 2nd order\" => Dict(:prob_choice => 2, :alg => EK1(order=4, smooth=DENSE))\n \"EK1(5) 2nd order\" => Dict(:prob_choice => 2, :alg => EK1(order=5, smooth=DENSE))\n \"EK1(5) 2nd order\" => Dict(:prob_choice => 2, :alg => EK1(order=6, smooth=DENSE))\n]\n\nlabels = first.(_setups)\nsetups = last.(_setups)\n\nabstols = 1.0 ./ 10.0 .^ (6:13)\nreltols = 1.0 ./ 10.0 .^ (3:10)\n\nwp = WorkPrecisionSet(\n [prob, prob2], abstols, reltols, setups;\n names = labels,\n #print_names = true,\n appxsol = [test_sol, test_sol2],\n dense = DENSE,\n save_everystep = SAVE_EVERYSTEP,\n numruns = 10,\n maxiters = Int(1e7),\n timeseries_errors = false,\n verbose = false,\n)\n\nplot(wp, color=[1 1 1 1 2 2 2 2], xticks = 10.0 .^ (-16:1:5))","category":"page"},{"location":"benchmarks/vanderpol/","page":"Stiff ODE: Van der Pol","title":"Stiff ODE: Van der Pol","text":"(Image: )","category":"page"},{"location":"benchmarks/vanderpol/#Conclusion","page":"Stiff ODE: Van der Pol","title":"Conclusion","text":"","category":"section"},{"location":"benchmarks/vanderpol/","page":"Stiff ODE: Van der Pol","title":"Stiff ODE: Van der Pol","text":"Use the EK1 to solve stiff problems, with orders leq 6 depending on the error tolerance.\nWhen the problem is actually a second-order ODE, as is the case for the Van der Pol system here, solve it as a second-order ODE.","category":"page"},{"location":"benchmarks/vanderpol/#Appendix","page":"Stiff ODE: Van der Pol","title":"Appendix","text":"","category":"section"},{"location":"benchmarks/vanderpol/","page":"Stiff ODE: Van der Pol","title":"Stiff ODE: Van der Pol","text":"Computer information:","category":"page"},{"location":"benchmarks/vanderpol/","page":"Stiff ODE: Van der Pol","title":"Stiff ODE: Van der Pol","text":"InteractiveUtils.versioninfo()","category":"page"},{"location":"benchmarks/vanderpol/","page":"Stiff ODE: Van der Pol","title":"Stiff ODE: Van der Pol","text":"Julia Version 1.8.5\nCommit 17cfb8e65ea (2023-01-08 06:45 UTC)\nPlatform Info:\n OS: Linux (x86_64-linux-gnu)\n CPU: 12 × Intel(R) Core(TM) i7-6800K CPU @ 3.40GHz\n WORD_SIZE: 64\n LIBM: libopenlibm\n LLVM: libLLVM-13.0.1 (ORCJIT, broadwell)\n Threads: 12 on 12 virtual cores\nEnvironment:\n JULIA_NUM_THREADS = auto\n JULIA_STACKTRACE_MINIMAL = true","category":"page"},{"location":"benchmarks/rober/#ROBER-benchmark","page":"DAE: ROBER","title":"ROBER benchmark","text":"","category":"section"},{"location":"benchmarks/rober/","page":"DAE: ROBER","title":"DAE: ROBER","text":"Adapted from SciMLBenchmarks.jl.","category":"page"},{"location":"benchmarks/rober/","page":"DAE: ROBER","title":"DAE: ROBER","text":"using LinearAlgebra, Statistics, InteractiveUtils\nusing DiffEqDevTools, ParameterizedFunctions, SciMLBase, OrdinaryDiffEq, Sundials, Plots\nusing ModelingToolkit\nusing ProbNumDiffEq\n\n# Plotting theme\ntheme(:dao;\n markerstrokewidth=0.5,\n legend=:outertopright,\n bottom_margin=5Plots.mm,\n size = (1000, 400),\n)","category":"page"},{"location":"benchmarks/rober/#ROBER-problem-definition","page":"DAE: ROBER","title":"ROBER problem definition","text":"","category":"section"},{"location":"benchmarks/rober/","page":"DAE: ROBER","title":"DAE: ROBER","text":"@variables t y₁(t)=1.0 y₂(t)=0.0 y₃(t)=0.0\n@parameters k₁=0.04 k₂=3e7 k₃=1e4\nD = Differential(t)\neqs = [\n D(y₁) ~ -k₁*y₁ + k₃*y₂*y₃\n D(y₂) ~ k₁*y₁ - k₃*y₂*y₃ - k₂*y₂^2\n 0 ~ y₁ + y₂ + y₃ - 1\n]\n@named sys = ODESystem(eqs)\nmmprob = ODEProblem(sys,[],(0.0,1e5))\ndaeprob = DAEProblem(sys,[D(y₁)=>-0.04, D(y₂)=>0.04, D(y₃)=>0.0],[],(0.0,1e5)) # can't handle this yet\nodaeprob = ODAEProblem(structural_simplify(sys),[],(0.0,1e5)) # can't handle this yet\n\nref_sol = solve(daeprob,IDA(),abstol=1/10^14,reltol=1/10^14,dense=false)\nplot(ref_sol, vars=[y₁,y₂,y₃], title=\"ROBER Solution\", legend=false, ylims=(0, 1))","category":"page"},{"location":"benchmarks/rober/","page":"DAE: ROBER","title":"DAE: ROBER","text":"(Image: )","category":"page"},{"location":"benchmarks/rober/#EK1-accross-orders","page":"DAE: ROBER","title":"EK1 accross orders","text":"","category":"section"},{"location":"benchmarks/rober/","page":"DAE: ROBER","title":"DAE: ROBER","text":"DENSE = false;\nSAVE_EVERYSTEP = false;\n\n_setups = [\n \"EK1($order)\" => Dict(:alg => EK1(order=order, smooth=DENSE))\n for order in 2:6\n]\n\nlabels = first.(_setups)\nsetups = last.(_setups)\n\n# works:\n# abstols = 1.0 ./ 10.0 .^ (4:10)\n# reltols = 1.0 ./ 10.0 .^ (1:7)\n# test:\nabstols = 1.0 ./ 10.0 .^ (4:9)\nreltols = 1.0 ./ 10.0 .^ (1:6)\n\nwp = WorkPrecisionSet(\n mmprob, abstols, reltols, setups;\n names = labels,\n #print_names = true,\n appxsol = ref_sol,\n dense = DENSE,\n save_everystep = SAVE_EVERYSTEP,\n numruns = 10,\n maxiters = Int(1e7),\n timeseries_errors = false,\n verbose = false,\n)\n\nplot(wp, palette=Plots.palette([:blue, :red], length(_setups)), xticks = 10.0 .^ (-16:1:5),\n xlims = (2e-15, 3e-7), ylims = (1e-2, 6e-1))","category":"page"},{"location":"benchmarks/rober/","page":"DAE: ROBER","title":"DAE: ROBER","text":"(Image: )","category":"page"},{"location":"benchmarks/rober/#Conclusion","page":"DAE: ROBER","title":"Conclusion","text":"","category":"section"},{"location":"benchmarks/rober/","page":"DAE: ROBER","title":"DAE: ROBER","text":"The EK1 can solve mass-matrix DAEs! But it only really works well for low errors.\nOrder 3 seems to work well here. But the order-to-error-tolerance heuristic should in principle still hold: lower tolerance level rightarrow higher order.","category":"page"},{"location":"benchmarks/rober/#Appendix","page":"DAE: ROBER","title":"Appendix","text":"","category":"section"},{"location":"benchmarks/rober/","page":"DAE: ROBER","title":"DAE: ROBER","text":"Computer information:","category":"page"},{"location":"benchmarks/rober/","page":"DAE: ROBER","title":"DAE: ROBER","text":"InteractiveUtils.versioninfo()","category":"page"},{"location":"benchmarks/rober/","page":"DAE: ROBER","title":"DAE: ROBER","text":"Julia Version 1.8.5\nCommit 17cfb8e65ea (2023-01-08 06:45 UTC)\nPlatform Info:\n OS: Linux (x86_64-linux-gnu)\n CPU: 12 × Intel(R) Core(TM) i7-6800K CPU @ 3.40GHz\n WORD_SIZE: 64\n LIBM: libopenlibm\n LLVM: libLLVM-13.0.1 (ORCJIT, broadwell)\n Threads: 12 on 12 virtual cores\nEnvironment:\n JULIA_NUM_THREADS = auto\n JULIA_STACKTRACE_MINIMAL = true","category":"page"},{"location":"filtering/#Gaussian-Filtering-and-Smoothing","page":"Filtering and Smoothing","title":"Gaussian Filtering and Smoothing","text":"","category":"section"},{"location":"filtering/#Predict","page":"Filtering and Smoothing","title":"Predict","text":"","category":"section"},{"location":"filtering/","page":"Filtering and Smoothing","title":"Filtering and Smoothing","text":"ProbNumDiffEq.predict\nProbNumDiffEq.predict!","category":"page"},{"location":"filtering/#ProbNumDiffEq.predict","page":"Filtering and Smoothing","title":"ProbNumDiffEq.predict","text":"predict(x::Gaussian, A::AbstractMatrix, Q::AbstractMatrix)\n\nPrediction step in Kalman filtering for linear dynamics models.\n\nGiven a Gaussian x = mathcalN(μ Σ), compute and return mathcalN(A μ A Σ A^T + Q).\n\nSee also the non-allocating square-root version predict!.\n\n\n\n\n\n","category":"function"},{"location":"filtering/#ProbNumDiffEq.predict!","page":"Filtering and Smoothing","title":"ProbNumDiffEq.predict!","text":"predict!(x_out, x_curr, Ah, Qh, cachemat)\n\nIn-place and square-root implementation of predict which saves the result into x_out.\n\nOnly works with PSDMatrices.PSDMatrix types as Ah, Qh, and in the covariances of x_curr and x_out (both of type Gaussian). To prevent allocations, a cache matrix cachemat of size D times 2D (where D times D is the size of Ah and Qh) needs to be passed.\n\nSee also: predict.\n\n\n\n\n\n","category":"function"},{"location":"filtering/#Update","page":"Filtering and Smoothing","title":"Update","text":"","category":"section"},{"location":"filtering/","page":"Filtering and Smoothing","title":"Filtering and Smoothing","text":"ProbNumDiffEq.update\nProbNumDiffEq.update!","category":"page"},{"location":"filtering/#ProbNumDiffEq.update","page":"Filtering and Smoothing","title":"ProbNumDiffEq.update","text":"update(x, measurement, H)\n\nUpdate step in Kalman filtering for linear dynamics models.\n\nGiven a Gaussian x = mathcalN(μ Σ) and a measurement z = mathcalN(hatz S), with S = H Σ H^T, compute\n\nbeginaligned\nK = Σ^P H^T S^-1 \nμ^F = μ + K (0 - hatz) \nΣ^F = Σ - K S K^T\nendaligned\n\nand return an updated state \\mathcal{N}(μ^F, Σ^F). Note that this assumes zero-measurements. When called with ProbNumDiffEq.SquarerootMatrix type arguments it performs the update in Joseph / square-root form.\n\nFor better performance, we recommend to use the non-allocating update!.\n\n\n\n\n\n","category":"function"},{"location":"filtering/#ProbNumDiffEq.update!","page":"Filtering and Smoothing","title":"ProbNumDiffEq.update!","text":"update!(x_out, x_pred, measurement, H, K_cache, M_cache, S_cache)\n\nIn-place and square-root implementation of update which saves the result into x_out.\n\nImplemented in Joseph Form to retain the PSDMatrix covariances:\n\nbeginaligned\nK = Σ^P H^T S^-1 \nμ^F = μ + K (0 - hatz) \nsqrtΣ^F = (I - KH) sqrt(Σ)\nendaligned\n\nwhere sqrtM denotes the left square-root of a matrix M, i.e. M = sqrtM sqrtM^T.\n\nTo prevent allocations, write into caches K_cache and M_cache, both of size D × D, and S_cache of same type as measurement.Σ.\n\nSee also: update.\n\n\n\n\n\n","category":"function"},{"location":"filtering/#Smooth","page":"Filtering and Smoothing","title":"Smooth","text":"","category":"section"},{"location":"filtering/","page":"Filtering and Smoothing","title":"Filtering and Smoothing","text":"ProbNumDiffEq.smooth\nProbNumDiffEq.smooth!","category":"page"},{"location":"filtering/#ProbNumDiffEq.smooth","page":"Filtering and Smoothing","title":"ProbNumDiffEq.smooth","text":"smooth(x_curr, x_next_smoothed, A, Q)\n\nUpdate step of the Kalman smoother, aka. Rauch-Tung-Striebel smoother, for linear dynamics models.\n\nGiven Gaussians x_n = mathcalN(μ_n Σ_n) and x_n+1 = mathcalN(μ_n+1^S Σ_n+1^S), compute\n\nbeginaligned\nμ_n+1^P = A μ_n^F \nP_n+1^P = A Σ_n^F A + Q \nG = Σ_n^S A^T (Σ_n+1^P)^-1 \nμ_n^S = μ_n^F + G (μ_n+1^S - μ_n+1^P) \nΣ_n^S = (I - G A) Σ_n^F (I - G A)^T + G Q G^T + G Σ_n+1^S G^T\nendaligned\n\nand return a smoothed state \\mathcal{N}(μ_n^S, Σ_n^S). When called with ProbNumDiffEq.SquarerootMatrix type arguments it performs the update in Joseph / square-root form.\n\nFor better performance, we recommend to use the non-allocating smooth!.\n\n\n\n\n\n","category":"function"},{"location":"filtering/#ProbNumDiffEq.smooth!","page":"Filtering and Smoothing","title":"ProbNumDiffEq.smooth!","text":"smooth!(x_curr, x_next, Ah, Qh, cache, diffusion=1)\n\nIn-place and square-root implementation of smooth which overwrites x_curr.\n\nImplemented in Joseph form to preserve square-root structure. It requires access to the solvers cache to prevent allocations.\n\nSee also: smooth.\n\n\n\n\n\n","category":"function"},{"location":"solvers/#Solvers","page":"Solvers","title":"Solvers","text":"","category":"section"},{"location":"solvers/","page":"Solvers","title":"Solvers","text":"ProbNumDiffEq.jl provides two solvers: the EK1 and the EK0. Both based on extended Kalman filtering and smoothing, but the latter relies on evaluating the Jacobian of the vector field. For the best results, use the EK1.","category":"page"},{"location":"solvers/","page":"Solvers","title":"Solvers","text":"All solvers are compatible with DAEs in mass-matrix ODE form. They also specialize on second-order ODEs: If the problem is of type SecondOrderODEProblem, it solves the second-order problem directly; this is more efficient than solving the transformed first-order problem and provides more meaningful posteriors [1].","category":"page"},{"location":"solvers/#API","page":"Solvers","title":"API","text":"","category":"section"},{"location":"solvers/","page":"Solvers","title":"Solvers","text":"EK1\nEK0","category":"page"},{"location":"solvers/#ProbNumDiffEq.EK1","page":"Solvers","title":"ProbNumDiffEq.EK1","text":"EK1(; order=3,\n smooth=true,\n prior=IWP(order),\n diffusionmodel=DynamicDiffusion(),\n initialization=TaylorModeInit(),\n kwargs...)\n\nGaussian ODE filter with first-order vector field linearization.\n\nArguments\n\norder::Integer: Order of the integrated Wiener process (IWP) prior.\nsmooth::Bool: Turn smoothing on/off; smoothing is required for dense output.\nprior::AbstractODEFilterPrior: Prior to be used by the ODE filter. By default, uses a 3-times integrated Wiener process prior IWP(3). See also: Priors.\ndiffusionmodel::ProbNumDiffEq.AbstractDiffusion: See Diffusion models and calibration.\ninitialization::ProbNumDiffEq.InitializationScheme: See Initialization.\n\nSome additional kwargs relating to implicit solvers are supported; check out DifferentialEquations.jl's Extra Options page. Right now, we support autodiff, chunk_size, and diff_type. In particular, autodiff=false can come in handy to use finite differences instead of ForwardDiff.jl to compute Jacobians.\n\nExamples\n\njulia> solve(prob, EK1())\n\nReferences\n\n\n\n\n\n","category":"type"},{"location":"solvers/#ProbNumDiffEq.EK0","page":"Solvers","title":"ProbNumDiffEq.EK0","text":"EK0(; order=3,\n smooth=true,\n prior=IWP(order),\n diffusionmodel=DynamicDiffusion(),\n initialization=TaylorModeInit())\n\nGaussian ODE filter with zeroth-order vector field linearization.\n\nArguments\n\norder::Integer: Order of the integrated Wiener process (IWP) prior.\nsmooth::Bool: Turn smoothing on/off; smoothing is required for dense output.\nprior::AbstractODEFilterPrior: Prior to be used by the ODE filter. By default, uses a 3-times integrated Wiener process prior IWP(3). See also: Priors.\ndiffusionmodel::ProbNumDiffEq.AbstractDiffusion: See Diffusion models and calibration.\ninitialization::ProbNumDiffEq.InitializationScheme: See Initialization.\n\nExamples\n\njulia> solve(prob, EK0())\n\nReferences\n\n\n\n\n\n","category":"type"},{"location":"solvers/#Probabilistic-Exponential-Integrators","page":"Solvers","title":"Probabilistic Exponential Integrators","text":"","category":"section"},{"location":"solvers/","page":"Solvers","title":"Solvers","text":"ExpEK\nRosenbrockExpEK","category":"page"},{"location":"solvers/#ProbNumDiffEq.ExpEK","page":"Solvers","title":"ProbNumDiffEq.ExpEK","text":"ExpEK(; L, order=3, kwargs...)\n\nProbabilistic exponential integrator\n\nProbabilistic exponential integrators are a class of integrators for semi-linear stiff ODEs that provide improved stability by essentially solving the linear part of the ODE exactly. In probabilistic numerics, this amounts to including the linear part into the prior model of the solver.\n\nExpEK is therefore just a short-hand for EK0 with IOUP prior:\n\nExpEK(; order=3, L, kwargs...) = EK0(; prior=IOUP(order, L), kwargs...)\n\nSee also RosenbrockExpEK, EK0, EK1.\n\nArguments\n\nSee EK0 for available keyword arguments.\n\nExamples\n\njulia> prob = ODEProblem((du, u, p, t) -> (@. du = - u + sin(u)), [1.0], (0.0, 10.0))\njulia> solve(prob, ExpEK(L=-1))\n\nReference\n\n[2] Bosch et al, \"Probabilistic Exponential Integrators\", arXiv (2021)\n\n\n\n\n\n","category":"function"},{"location":"solvers/#ProbNumDiffEq.RosenbrockExpEK","page":"Solvers","title":"ProbNumDiffEq.RosenbrockExpEK","text":"RosenbrockExpEK(; order=3, kwargs...)\n\nProbabilistic Rosenbrock-type exponential integrator\n\nA probabilistic exponential integrator similar to ExpEK, but with automatic linearization along the mean numerical solution. This brings the advantage that the linearity does not need to be specified manually, and the more accurate local linearization can sometimes also improve stability; but since the \"prior\" is adjusted at each step the probabilistic interpretation becomes more complicated.\n\nRosenbrockExpEK is just a short-hand for EK1 with appropriete IOUP prior:\n\nRosenbrockExpEK(; order=3, kwargs...) = EK1(; prior=IOUP(order, update_rate_parameter=true), kwargs...)\n\nSee also ExpEK, EK0, EK1.\n\nArguments\n\nSee EK1 for available keyword arguments.\n\nExamples\n\njulia> prob = ODEProblem((du, u, p, t) -> (@. du = - u + sin(u)), [1.0], (0.0, 10.0))\njulia> solve(prob, RosenbrockExpEK())\n\nReference\n\n[2] Bosch et al, \"Probabilistic Exponential Integrators\", arXiv (2021)\n\n\n\n\n\n","category":"function"},{"location":"solvers/#solversrefs","page":"Solvers","title":"References","text":"","category":"section"},{"location":"solvers/","page":"Solvers","title":"Solvers","text":"Pages = []\nCanonical = false\n\nbosch23expint","category":"page"},{"location":"tutorials/exponential_integrators/#Probabilistic-Exponential-Integrators","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"","category":"section"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"Exponential integrators are a class of numerical methods for solving semi-linear ordinary differential equations (ODEs) of the form","category":"page"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"beginaligned\ndoty(t) = L y(t) + f(y(t) t) quad y(0) = y_0\nendaligned","category":"page"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"where L is a linear operator and f is a nonlinear function. In a nutshell, exponential integrators solve the linear part of the ODE exactly, and only approximate the nonlinear part. Probabilistic exponential integrators [2] are the probabilistic numerics approach to exponential integrators.","category":"page"},{"location":"tutorials/exponential_integrators/#Example","page":"Probabilistic Exponential Integrators","title":"Example","text":"","category":"section"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"Let's consider a simple semi-linear ODE","category":"page"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"beginaligned\ndoty(t) = - y(t) + sin(y(t)) quad y(0) = 10\nendaligned","category":"page"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"We can solve this ODE reasonably well with the standard EK1 and adaptive steps (the default):","category":"page"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"using ProbNumDiffEq, Plots, LinearAlgebra\ntheme(:default; palette=[\"#4063D8\", \"#389826\", \"#9558B2\", \"#CB3C33\"])\n\nf(du, u, p, t) = (@. du = -u + sin(u))\nu0 = [1.0]\ntspan = (0.0, 20.0)\nprob = ODEProblem(f, u0, tspan)\n\nref = solve(prob, EK1(), abstol=1e-10, reltol=1e-10)\nplot(ref, color=:black, linestyle=:dash, label=\"Reference\")","category":"page"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"But for fixed (large) step sizes this ODE is more challenging: The explicit EK0 method oscillates and diverges due to the stiffness of the ODE, and the semi-implicit EK1 method is stable but the solution is not very accurate.","category":"page"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"STEPSIZE = 4\nDM = FixedDiffusion() # recommended for fixed steps\n\n# we don't smooth the EK0 here to show the oscillations more clearly\nsol0 = solve(prob, EK0(smooth=false, diffusionmodel=DM), adaptive=false, dt=STEPSIZE, dense=false)\nsol1 = solve(prob, EK1(diffusionmodel=DM), adaptive=false, dt=STEPSIZE)\n\nplot(ylims=(0.3, 1.05))\nplot!(ref, color=:black, linestyle=:dash, label=\"Reference\")\nplot!(sol0, denseplot=false, marker=:o, markersize=2, label=\"EK0\", color=1)\nplot!(sol1, denseplot=false, marker=:o, markersize=2, label=\"EK1\", color=2)","category":"page"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"Probabilistic exponential integrators leverage the semi-linearity of the ODE to compute more accurate solutions for the same fixed step size. You can use either the ExpEK method and provide the linear part (with the keyword argument L), or the RosenbrockExpEK to automatically linearize along the mean of the numerical solution:","category":"page"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"sol_exp = solve(prob, ExpEK(L=-1, diffusionmodel=DM), adaptive=false, dt=STEPSIZE)\nsol_ros = solve(prob, RosenbrockExpEK(diffusionmodel=DM), adaptive=false, dt=STEPSIZE)\n\nplot(ylims=(0.3, 1.05))\nplot!(ref, color=:black, linestyle=:dash, label=\"Reference\")\nplot!(sol_exp, denseplot=false, marker=:o, markersize=2, label=\"ExpEK\", color=3)\nplot!(sol_ros, denseplot=false, marker=:o, markersize=2, label=\"RosenbrockExpEK\", color=4)","category":"page"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"The solutions are indeed much more accurate than those of the standard EK1, for the same fixed step size!","category":"page"},{"location":"tutorials/exponential_integrators/#Background:-Integrated-Ornstein-Uhlenbeck-priors","page":"Probabilistic Exponential Integrators","title":"Background: Integrated Ornstein-Uhlenbeck priors","text":"","category":"section"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"Probabilistic exponential integrators \"solve the linear part exactly\" by including it into the prior model of the solver. Namely, the solver chooses a (q-times) integrated Ornstein-Uhlenbeck prior with rate parameter equal to the linearity. The ExpEK solver is just a short-hand for an EK0 with appropriate prior:","category":"page"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"ExpEK(order=3, L=-1) == EK0(prior=IOUP(3, -1))","category":"page"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"Similarly, the RosenbrockExpEK solver is also just a short-hand:","category":"page"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"RosenbrockExpEK(order=3) == EK1(prior=IOUP(3, update_rate_parameter=true))","category":"page"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"This means that you can also construct other probabilistic exponential integrators by hand! In this example the EK1 with IOUP prior with rate parameter -1 performs extremely well:","category":"page"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"sol_expek1 = solve(prob, EK1(prior=IOUP(3, -1), diffusionmodel=DM), adaptive=false, dt=STEPSIZE)\n\nplot(ylims=(0.3, 1.05))\nplot!(ref, color=:black, linestyle=:dash, label=\"Reference\")\nplot!(sol_expek1, denseplot=false, marker=:o, markersize=2, label=\"EK1 + IOUP\")","category":"page"},{"location":"tutorials/exponential_integrators/#References","page":"Probabilistic Exponential Integrators","title":"References","text":"","category":"section"},{"location":"tutorials/exponential_integrators/","page":"Probabilistic Exponential Integrators","title":"Probabilistic Exponential Integrators","text":"Pages = []\nCanonical = false\n\nbosch23expint","category":"page"},{"location":"benchmarks/lotkavolterra/#Lotka-Volterra-benchmark","page":"Non-stiff ODE: Lotka-Volterra","title":"Lotka-Volterra benchmark","text":"","category":"section"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"Adapted from SciMLBenchmarks.jl.","category":"page"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"using LinearAlgebra, Statistics, InteractiveUtils\nusing DiffEqDevTools, ParameterizedFunctions, SciMLBase, OrdinaryDiffEq, Plots\nusing ProbNumDiffEq\n\n# Plotting theme\ntheme(:dao;\n markerstrokewidth=0.5,\n legend=:outertopright,\n bottom_margin=5Plots.mm,\n size = (1000, 400),\n)","category":"page"},{"location":"benchmarks/lotkavolterra/#Lotka-Volterra-problem-definition","page":"Non-stiff ODE: Lotka-Volterra","title":"Lotka-Volterra problem definition","text":"","category":"section"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"f = @ode_def LotkaVolterra begin\n dx = a*x - b*x*y\n dy = -c*y + d*x*y\nend a b c d\np = [1.5, 1, 3, 1]\ntspan = (0.0, 10.0)\nu0 = [1.0, 1.0]\nprob = ODEProblem{true, SciMLBase.FullSpecialize}(f, u0, tspan, p)\n\ntest_sol = solve(prob, Vern7(), abstol=1/10^14, reltol=1/10^14, dense=false)\nplot(test_sol, title=\"Lotka-Volterra Solution\", legend=false)","category":"page"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"(Image: )","category":"page"},{"location":"benchmarks/lotkavolterra/#EK0-accross-orders","page":"Non-stiff ODE: Lotka-Volterra","title":"EK0 accross orders","text":"","category":"section"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"DENSE = false;\nSAVE_EVERYSTEP = false;\n\n_setups = [\n \"EK0(order=$order)\" => Dict(:alg => EK0(order=order, smooth=DENSE))\n for order in 1:7\n]\n\nlabels = first.(_setups)\nsetups = last.(_setups)\n\nabstols = 1.0 ./ 10.0 .^ (4:13)\nreltols = 1.0 ./ 10.0 .^ (1:10)\n\nwp = WorkPrecisionSet(\n prob, abstols, reltols, setups;\n names = labels,\n #print_names = true,\n appxsol = test_sol,\n dense = DENSE,\n save_everystep = SAVE_EVERYSTEP,\n numruns = 10,\n maxiters = Int(1e7),\n timeseries_errors = false,\n verbose = false,\n)\n\nplot(wp, palette=Plots.palette([:blue, :red], length(_setups)), xticks = 10.0 .^ (-16:1:5))","category":"page"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"(Image: )","category":"page"},{"location":"benchmarks/lotkavolterra/#EK1-accross-orders","page":"Non-stiff ODE: Lotka-Volterra","title":"EK1 accross orders","text":"","category":"section"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"DENSE = false;\nSAVE_EVERYSTEP = false;\n\n_setups = [\n \"EK1(order=$order)\" => Dict(:alg => EK1(order=order, smooth=DENSE))\n for order in 1:7\n]\n\nlabels = first.(_setups)\nsetups = last.(_setups)\n\nabstols = 1.0 ./ 10.0 .^ (4:13)\nreltols = 1.0 ./ 10.0 .^ (1:10)\n\nwp = WorkPrecisionSet(\n prob, abstols, reltols, setups;\n names = labels,\n #print_names = true,\n appxsol = test_sol,\n dense = DENSE,\n save_everystep = SAVE_EVERYSTEP,\n numruns = 10,\n maxiters = Int(1e7),\n timeseries_errors = false,\n verbose = false,\n)\n\nplot(wp, palette=Plots.palette([:blue, :red], length(_setups)), xticks = 10.0 .^ (-16:1:5))","category":"page"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"(Image: )","category":"page"},{"location":"benchmarks/lotkavolterra/#EK0-vs.-EK1","page":"Non-stiff ODE: Lotka-Volterra","title":"EK0 vs. EK1","text":"","category":"section"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"DENSE = false;\nSAVE_EVERYSTEP = false;\n\n_setups = [\n \"EK0(order=2)\" => Dict(:alg => EK0(order=2, smooth=DENSE))\n \"EK0(order=3)\" => Dict(:alg => EK0(order=3, smooth=DENSE))\n \"EK0(order=4)\" => Dict(:alg => EK0(order=4, smooth=DENSE))\n \"EK0(order=5)\" => Dict(:alg => EK0(order=5, smooth=DENSE))\n \"EK1(order=2)\" => Dict(:alg => EK1(order=2, smooth=DENSE))\n \"EK1(order=3)\" => Dict(:alg => EK1(order=3, smooth=DENSE))\n \"EK1(order=4)\" => Dict(:alg => EK1(order=4, smooth=DENSE))\n \"EK1(order=5)\" => Dict(:alg => EK1(order=5, smooth=DENSE))\n]\n\nlabels = first.(_setups)\nsetups = last.(_setups)\n\nabstols = 1.0 ./ 10.0 .^ (4:14)\nreltols = 1.0 ./ 10.0 .^ (1:11)\n\nwp = WorkPrecisionSet(\n prob, abstols, reltols, setups;\n names = labels,\n #print_names = true,\n appxsol = test_sol,\n dense = DENSE,\n save_everystep = SAVE_EVERYSTEP,\n numruns = 10,\n maxiters = Int(1e7),\n timeseries_errors = false,\n verbose = false,\n)\n\nplot(wp, color=[1 1 1 1 2 2 2 2], xticks = 10.0 .^ (-16:1:5))","category":"page"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"(Image: )","category":"page"},{"location":"benchmarks/lotkavolterra/#DynamicDiffusion-vs-FixedDiffusion","page":"Non-stiff ODE: Lotka-Volterra","title":"DynamicDiffusion vs FixedDiffusion","text":"","category":"section"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"DENSE = false;\nSAVE_EVERYSTEP = false;\n\n_setups = [\n \"EK1(2) dynamic\" => Dict(:alg => EK1(order=2, smooth=DENSE, diffusionmodel=DynamicDiffusion()))\n \"EK1(3) dynamic\" => Dict(:alg => EK1(order=3, smooth=DENSE, diffusionmodel=DynamicDiffusion()))\n \"EK1(5) dynamic\" => Dict(:alg => EK1(order=5, smooth=DENSE, diffusionmodel=DynamicDiffusion()))\n \"EK1(2) fixed\" => Dict(:alg => EK1(order=2, smooth=DENSE, diffusionmodel=FixedDiffusion()))\n \"EK1(3) fixed\" => Dict(:alg => EK1(order=3, smooth=DENSE, diffusionmodel=FixedDiffusion()))\n \"EK1(5) fixed\" => Dict(:alg => EK1(order=5, smooth=DENSE, diffusionmodel=FixedDiffusion()))\n]\n\nlabels = first.(_setups)\nsetups = last.(_setups)\n\nabstols = 1.0 ./ 10.0 .^ (4:15)\nreltols = 1.0 ./ 10.0 .^ (1:12)\n\nwp = WorkPrecisionSet(\n prob, abstols, reltols, setups;\n names = labels,\n #print_names = true,\n appxsol = test_sol,\n dense = DENSE,\n save_everystep = SAVE_EVERYSTEP,\n numruns = 10,\n maxiters = Int(1e7),\n timeseries_errors = false,\n verbose = false,\n)\n\nplot(wp, color=[2 2 2 3 3 3], xticks = 10.0 .^ (-16:1:5))","category":"page"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"(Image: )","category":"page"},{"location":"benchmarks/lotkavolterra/#TaylorModeInit-vs-ClassicSolverInit","page":"Non-stiff ODE: Lotka-Volterra","title":"TaylorModeInit vs ClassicSolverInit","text":"","category":"section"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"DENSE = false;\nSAVE_EVERYSTEP = false;\n\n_setups = [\n \"EK1(2) TaylorInit\" => Dict(:alg => EK1(order=2, smooth=DENSE, initialization=TaylorModeInit()))\n \"EK1(3) TaylorInit\" => Dict(:alg => EK1(order=3, smooth=DENSE, initialization=TaylorModeInit()))\n \"EK1(5) TaylorInit\" => Dict(:alg => EK1(order=5, smooth=DENSE, initialization=TaylorModeInit()))\n \"EK1(2) Tsit5Init\" => Dict(:alg => EK1(order=2, smooth=DENSE, initialization=ClassicSolverInit()))\n \"EK1(3) Tsit5Init\" => Dict(:alg => EK1(order=3, smooth=DENSE, initialization=ClassicSolverInit()))\n \"EK1(5) Tsit5Init\" => Dict(:alg => EK1(order=5, smooth=DENSE, initialization=ClassicSolverInit()))\n \"EK1(2) Tsit5Init+ddu\" => Dict(:alg => EK1(order=2, smooth=DENSE, initialization=ClassicSolverInit(init_on_ddu=true)))\n \"EK1(3) Tsit5Init+ddu\" => Dict(:alg => EK1(order=3, smooth=DENSE, initialization=ClassicSolverInit(init_on_ddu=true)))\n \"EK1(5) Tsit5Init+ddu\" => Dict(:alg => EK1(order=5, smooth=DENSE, initialization=ClassicSolverInit(init_on_ddu=true)))\n]\n\nlabels = first.(_setups)\nsetups = last.(_setups)\n\nabstols = 1.0 ./ 10.0 .^ (4:15)\nreltols = 1.0 ./ 10.0 .^ (1:12)\n\nwp = WorkPrecisionSet(\n prob, abstols, reltols, setups;\n names = labels,\n #print_names = true,\n appxsol = test_sol,\n dense = DENSE,\n save_everystep = SAVE_EVERYSTEP,\n numruns = 10,\n maxiters = Int(1e7),\n timeseries_errors = false,\n verbose = false,\n)\n\nplot(wp, color=[2 2 2 4 4 4 5 5 5], xticks = 10.0 .^ (-16:1:5))","category":"page"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"(Image: )","category":"page"},{"location":"benchmarks/lotkavolterra/#Conclusion","page":"Non-stiff ODE: Lotka-Volterra","title":"Conclusion","text":"","category":"section"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"Use the EK1! It seems to be strictly better than the EK0 here. Though note that by using ParameterizedFunctions.jl, the Jacobian of the vector field is available analytically.\nOrders behave as in classic solvers: Use low order for low accuracy, medium order for medium accuracy, high order for high accuracy.\nMost likely, the default choice of diffusionmodel=DynamicDiffusion and initialization=TaylorModeInit are fine.","category":"page"},{"location":"benchmarks/lotkavolterra/#Appendix","page":"Non-stiff ODE: Lotka-Volterra","title":"Appendix","text":"","category":"section"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"Computer information:","category":"page"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"InteractiveUtils.versioninfo()","category":"page"},{"location":"benchmarks/lotkavolterra/","page":"Non-stiff ODE: Lotka-Volterra","title":"Non-stiff ODE: Lotka-Volterra","text":"Julia Version 1.8.5\nCommit 17cfb8e65ea (2023-01-08 06:45 UTC)\nPlatform Info:\n OS: Linux (x86_64-linux-gnu)\n CPU: 12 × Intel(R) Core(TM) i7-6800K CPU @ 3.40GHz\n WORD_SIZE: 64\n LIBM: libopenlibm\n LLVM: libLLVM-13.0.1 (ORCJIT, broadwell)\n Threads: 12 on 12 virtual cores\nEnvironment:\n JULIA_NUM_THREADS = auto\n JULIA_STACKTRACE_MINIMAL = true","category":"page"},{"location":"initialization/#Initialization","page":"Initialization","title":"Initialization","text":"","category":"section"},{"location":"initialization/","page":"Initialization","title":"Initialization","text":"The notion of \"initialization\" relates to the prior part of the model.","category":"page"},{"location":"initialization/#Background:-The-prior","page":"Initialization","title":"Background: The prior","text":"","category":"section"},{"location":"initialization/","page":"Initialization","title":"Initialization","text":"We model the ODE solution y(t) with a Gauss–Markov prior. More precisely, let","category":"page"},{"location":"initialization/","page":"Initialization","title":"Initialization","text":"beginaligned\nY(t) = left Y^(0)(t) Y^(1)(t) dots Y^(q)(t) right\nendaligned","category":"page"},{"location":"initialization/","page":"Initialization","title":"Initialization","text":"be the solution to the SDE","category":"page"},{"location":"initialization/","page":"Initialization","title":"Initialization","text":"beginaligned\ntextd Y^(i)(t) = Y^(i+1)(t) textdt qquad i = 0 dots q-1 \ntextd Y^(q)(t) = textcolor389826A Y(t) textdt + textcolor4063D8Gamma textdW(t) \nY(0) sim textcolor9558B2 mathcalN left( mu_0 Sigma_0 right) \nendaligned","category":"page"},{"location":"initialization/","page":"Initialization","title":"Initialization","text":"Then Y^(i)(t) models the i-th derivative of y(t). In this section, we consider the initial distribution textcolorpurple mathcalN left( mu_0 Sigma_0 right) . If you're more interested in the drift matrix textcolor389826A check out the Priors section, and for more info on the diffusion textcolor4063D8Gamma check out the Diffusion models and calibration section.","category":"page"},{"location":"initialization/#Setting-the-initial-distribution","page":"Initialization","title":"Setting the initial distribution","text":"","category":"section"},{"location":"initialization/","page":"Initialization","title":"Initialization","text":"Let's assume an initial value problem of the form","category":"page"},{"location":"initialization/","page":"Initialization","title":"Initialization","text":"beginaligned\ndoty(t) = f(y(t) t) qquad 0 T \ny(0) = y_0\nendaligned","category":"page"},{"location":"initialization/","page":"Initialization","title":"Initialization","text":"It is clear that this contains quite some information for Y(0): The initial value y_0 and the vector field f imply","category":"page"},{"location":"initialization/","page":"Initialization","title":"Initialization","text":"beginaligned\nY^(0)(0) = y_0 \nY^(1)(0) = f(y_0 0)\nendaligned","category":"page"},{"location":"initialization/","page":"Initialization","title":"Initialization","text":"It turns out that we can also compute higher-order derivatives of y with the chain rule, and then use these to better initialize Y^(i)(0). This, done efficiently with Taylor-mode autodiff by using TaylorIntegration.jl, is what TaylorModeInit does. See also [1].","category":"page"},{"location":"initialization/","page":"Initialization","title":"Initialization","text":"In the vast majority of cases, just stick to the exact Taylor-mode initialization TaylorModeInit!","category":"page"},{"location":"initialization/#API","page":"Initialization","title":"API","text":"","category":"section"},{"location":"initialization/","page":"Initialization","title":"Initialization","text":"TaylorModeInit\nClassicSolverInit","category":"page"},{"location":"initialization/#ProbNumDiffEq.TaylorModeInit","page":"Initialization","title":"ProbNumDiffEq.TaylorModeInit","text":"TaylorModeInit()\n\nExact initialization via Taylor-mode automatic differentiation.\n\nThis is the recommended initialization method!\n\nIt uses TaylorIntegration.jl to efficiently compute the higher-order derivatives of the solution at the initial value, via Taylor-mode automatic differentiation.\n\nIn some special cases it can happen that TaylorIntegration.jl is incompatible with the given problem (typically because the problem definition does not allow for elements of type Taylor). If this happens, try ClassicSolverInit.\n\nReferences\n\n[4] Krämer et al, \"Stable Implementation of Probabilistic ODE Solvers\" (2020)\n\n\n\n\n\n","category":"type"},{"location":"initialization/#ProbNumDiffEq.ClassicSolverInit","page":"Initialization","title":"ProbNumDiffEq.ClassicSolverInit","text":"ClassicSolverInit(; alg=OrdinaryDiffEq.Tsit5(), init_on_ddu=false)\n\nInitialization via regression on a few steps of a classic ODE solver.\n\nIn a nutshell, instead of specifying mu_0 exactly and setting Sigma_0=0 (which is what TaylorModeInit does), use a classic ODE solver to compute a few steps of the solution, and then regress on the computed values (by running a smoother) to compute mu_0 and Sigma_0 as the mean and covariance of the smoothing posterior at time 0. See also [2].\n\nThe initial value and derivative are set directly from the given initial value problem; optionally the second derivative can also be set via automatic differentiation by setting init_on_ddu=true.\n\nArguments\n\nalg: The solver to be used. Can be any solver from OrdinaryDiffEq.jl.\ninit_on_ddu: If true, the second derivative is also initialized exactly via automatic differentiation with ForwardDiff.jl.\n\nReferences\n\n[4] Krämer et al, \"Stable Implementation of Probabilistic ODE Solvers\" (2020)\n[5] Schober et al, \"A probabilistic model for the numerical solution of initial value problems\", Statistics and Computing (2019)\n\n\n\n\n\n","category":"type"},{"location":"initialization/#initrefs","page":"Initialization","title":"References","text":"","category":"section"},{"location":"initialization/","page":"Initialization","title":"Initialization","text":"Pages = []\nCanonical = false\n\nkraemer20stableimplementation\nschober16probivp","category":"page"},{"location":"tutorials/dae/#Solving-DAEs-with-Probabilistic-Numerics","page":"Differential Algebraic Equations","title":"Solving DAEs with Probabilistic Numerics","text":"","category":"section"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"ProbNumDiffEq.jl provides probabilistic numerical solvers for differential algebraic equations (DAEs). Currently, we recommend using the semi-implicit EK1 algorithm.","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"note: Note\nFor a more general tutorial on DAEs check out the DifferentialEquations.jl DAE tutorial.","category":"page"},{"location":"tutorials/dae/#Solving-mass-matrix-DAEs-with-the-EK1","page":"Differential Algebraic Equations","title":"Solving mass-matrix DAEs with the EK1","text":"","category":"section"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"First, define the DAE (here the ROBER problem) as an ODE problem with singular mass matrix:","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"using ProbNumDiffEq, Plots, LinearAlgebra, OrdinaryDiffEq, ModelingToolkit\n\nfunction rober(du, u, p, t)\n y₁, y₂, y₃ = u\n k₁, k₂, k₃ = p\n du[1] = -k₁ * y₁ + k₃ * y₂ * y₃\n du[2] = k₁ * y₁ - k₃ * y₂ * y₃ - k₂ * y₂^2\n du[3] = y₁ + y₂ + y₃ - 1\n nothing\nend\nM = [1 0 0\n 0 1 0\n 0 0 0]\nf = ODEFunction(rober, mass_matrix=M)\nprob_mm = ODEProblem(f, [1.0, 0.0, 0.0], (0.0, 1e5), (0.04, 3e7, 1e4))","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"We can solve this problem directly with the EK1:","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"sol = solve(prob_mm, EK1(), reltol=1e-8, abstol=1e-8)\nplot(\n sol,\n xscale=:log10,\n tspan=(1e-6, 1e5),\n layout=(3, 1),\n legend=false,\n ylabel=[\"u₁(t)\" \"u₂(t)\" \"u₃(t)\"],\n xlabel=[\"\" \"\" \"t\"],\n denseplot=false,\n)","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"Looks good!","category":"page"},{"location":"tutorials/dae/#Solving-an-Index-3-DAE-directly","page":"Differential Algebraic Equations","title":"Solving an Index-3 DAE directly","text":"","category":"section"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"The following is based on the \"Automatic Index Reduction of DAEs\" tutorial by ModelingToolkit.jl, which demonstrates how the classic Rodas4 solver fails to solve a DAE due to the fact that it is of index 3; which is why ModelingToolkit's automatic index reduction is so useful.","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"It turns out that our probabilistic numerical solvers can directly solve the index-3 DAE!","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"First, define the pendulum problem as in the tutorial:","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"function pendulum!(du, u, p, t)\n x, dx, y, dy, T = u\n g, L = p\n du[1] = dx\n du[2] = T * x\n du[3] = dy\n du[4] = T * y - g\n du[5] = x^2 + y^2 - L^2\nend\npendulum_fun! = ODEFunction(pendulum!, mass_matrix=Diagonal([1, 1, 1, 1, 0]))\nu0 = [1.0, 0, 0, 0, 0];\np = [9.8, 1];\ntspan = (0, 5.0);\npendulum_prob = ODEProblem(pendulum_fun!, u0, tspan, p)","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"We can try to solve it directly with one of the classic mass-matrix DAE solvers from OrdinaryDiffEq.jl:","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"solve(pendulum_prob, Rodas4())","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"It does not work! This is because of the index of the DAE; see e.g. this explenation from the tutorial.","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"Does this also hold for the EK1 solver? Let's find out:","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"sol = solve(pendulum_prob, EK1())","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"Nope! The EK1 is able to solve the index-3 DAE directly. Pretty cool!","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"plot(sol)","category":"page"},{"location":"tutorials/dae/#Is-index-reduction-still-worth-it?","page":"Differential Algebraic Equations","title":"Is index-reduction still worth it?","text":"","category":"section"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"The point of the \"Automatic Index Reduction of DAEs\" tutorial is to demonstrate ModelingToolkit's utility for automatic index reduction, which enables the classic implicit Runge-Kutta solvers such as Rodas5 to solve this DAE. Let's see if that still helps in this context here.","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"First, modelingtoolkitize the problem:","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"traced_sys = modelingtoolkitize(pendulum_prob)","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"(how cool is this latex output ?!?)","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"Next, lower the DAE index and simplify it with MTK's dae_index_lowering and structural_simplify:","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"simplified_sys = structural_simplify(dae_index_lowering(traced_sys))","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"Let's build two different ODE problems, and check how well we can solve each:","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"prob_index3 = ODEProblem(traced_sys, Pair[], tspan)\nprob_index1 = ODEProblem(simplified_sys, Pair[], tspan)\n\nsol3 = solve(prob_index3, EK1())\nsol1 = solve(prob_index1, EK1())\n\ntruesol = solve(prob_index1, Rodas4(), abstol=1e-10, reltol=1e-10)\n\nsol1_final_error = norm(sol1.u[end] - truesol.u[end])\nsol1_f_evals = sol1.stats.nf\nsol3_final_error = norm(sol3.u[end] - truesol.u[end])\nsol3_f_evals = sol3.stats.nf\n@info \"Results\" sol1_final_error sol1_f_evals sol3_final_error sol3_f_evals","category":"page"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"The error for the index-1 DAE solve is much lower. So it seems that, even if the index-3 DAE could also be solved directly, index lowering might still be beneficial when solving DAEs with the EK1!","category":"page"},{"location":"tutorials/dae/#References","page":"Differential Algebraic Equations","title":"References","text":"","category":"section"},{"location":"tutorials/dae/","page":"Differential Algebraic Equations","title":"Differential Algebraic Equations","text":"Pages = []\nCanonical = false\n\nbosch22pickandmix","category":"page"},{"location":"diffusions/#Diffusion-models-and-calibration","page":"Diffusion models and calibration","title":"Diffusion models and calibration","text":"","category":"section"},{"location":"diffusions/","page":"Diffusion models and calibration","title":"Diffusion models and calibration","text":"The notion of \"diffusion\" and \"calibration\" relates to the prior part of the model.","category":"page"},{"location":"diffusions/#Background:-The-prior","page":"Diffusion models and calibration","title":"Background: The prior","text":"","category":"section"},{"location":"diffusions/","page":"Diffusion models and calibration","title":"Diffusion models and calibration","text":"We model the ODE solution y(t) with a Gauss–Markov prior. More precisely, let","category":"page"},{"location":"diffusions/","page":"Diffusion models and calibration","title":"Diffusion models and calibration","text":"beginaligned\nY(t) = left Y^(0)(t) Y^(1)(t) dots Y^(q)(t) right\nendaligned","category":"page"},{"location":"diffusions/","page":"Diffusion models and calibration","title":"Diffusion models and calibration","text":"be the solution to the SDE","category":"page"},{"location":"diffusions/","page":"Diffusion models and calibration","title":"Diffusion models and calibration","text":"beginaligned\ntextd Y^(i)(t) = Y^(i+1)(t) textdt qquad i = 0 dots q-1 \ntextd Y^(q)(t) = textcolor389826A Y(t) textdt + textcolor4063D8Gamma textdW(t) \nY(0) sim textcolorpurple mathcalN left( mu_0 Sigma_0 right) \nendaligned","category":"page"},{"location":"diffusions/","page":"Diffusion models and calibration","title":"Diffusion models and calibration","text":"Then Y^(i)(t) models the i-th derivative of y(t). In this section, we consider choices relating to the \"diffusion\" textcolor4063D8Gamma. If you're more interested in the drift matrix textcolor389826A check out the Priors section, and for info on the initial distribution textcolorpurple mathcalN left( mu_0 Sigma_0 right) check out the Initialization section.","category":"page"},{"location":"diffusions/#Diffusion-and-calibration","page":"Diffusion models and calibration","title":"Diffusion and calibration","text":"","category":"section"},{"location":"diffusions/","page":"Diffusion models and calibration","title":"Diffusion models and calibration","text":"We call textcolor4063D8Gamma the \"diffusion\" parameter. Since it is typically not known we need to estimate it; this is called \"calibration\".","category":"page"},{"location":"diffusions/","page":"Diffusion models and calibration","title":"Diffusion models and calibration","text":"There are a few different choices for how to model and estimate textcolor4063D8Gamma:","category":"page"},{"location":"diffusions/","page":"Diffusion models and calibration","title":"Diffusion models and calibration","text":"FixedDiffusion assumes an isotropic, time-fixed textcolor4063D8Gamma = sigma cdot I_d,\nDynamicDiffusion assumes an isotropic, time-varying textcolor4063D8Gamma(t) = sigma(t) cdot I_d (recommended),\nFixedMVDiffusion assumes a diagonal, time-fixed textcolor4063D8Gamma = operatornamediag(sigma_1 dots sigma_d),\nDynamicMVDiffusion assumes a diagonal, time-varying textcolor4063D8Gamma(t) = operatornamediag(sigma_1(t) dots sigma_d(t)).","category":"page"},{"location":"diffusions/","page":"Diffusion models and calibration","title":"Diffusion models and calibration","text":"Or more compactly:","category":"page"},{"location":"diffusions/","page":"Diffusion models and calibration","title":"Diffusion models and calibration","text":" Isotropic: Diagonal (only for the EK0)\nTime-varying DynamicDiffusion DynamicMVDiffusion\nTime-fixed FixedDiffusion FixedMVDiffusion","category":"page"},{"location":"diffusions/","page":"Diffusion models and calibration","title":"Diffusion models and calibration","text":"For more details on diffusions and calibration, check out this paper [6].","category":"page"},{"location":"diffusions/#API","page":"Diffusion models and calibration","title":"API","text":"","category":"section"},{"location":"diffusions/","page":"Diffusion models and calibration","title":"Diffusion models and calibration","text":"DynamicDiffusion\nFixedDiffusion\nDynamicMVDiffusion\nFixedMVDiffusion","category":"page"},{"location":"diffusions/#ProbNumDiffEq.DynamicDiffusion","page":"Diffusion models and calibration","title":"ProbNumDiffEq.DynamicDiffusion","text":"DynamicDiffusion()\n\nTime-varying, isotropic diffusion, which is quasi-maximum-likelihood-estimated at each step.\n\nThis is the recommended diffusion when using adaptive step-size selection, and in particular also when solving stiff systems.\n\n\n\n\n\n","category":"type"},{"location":"diffusions/#ProbNumDiffEq.FixedDiffusion","page":"Diffusion models and calibration","title":"ProbNumDiffEq.FixedDiffusion","text":"FixedDiffusion(; initial_diffusion=1.0, calibrate=true)\n\nTime-fixed, isotropic diffusion, which is (optionally) quasi-maximum-likelihood-estimated.\n\nThis is the recommended diffusion when using fixed steps.\n\nBy default with calibrate=true, all covariances are re-scaled at the end of the solve with the MLE diffusion. Set calibrate=false to skip this step, e.g. when setting the initial_diffusion and then estimating the diffusion outside of the solver (e.g. with Fenrir.jl).\n\n\n\n\n\n","category":"type"},{"location":"diffusions/#ProbNumDiffEq.DynamicMVDiffusion","page":"Diffusion models and calibration","title":"ProbNumDiffEq.DynamicMVDiffusion","text":"DynamicMVDiffusion()\n\nTime-varying, diagonal diffusion, which is quasi-maximum-likelihood-estimated at each step.\n\nOnly works with the EK0!\n\nA multi-variate version of DynamicDiffusion, where instead of an isotropic matrix, a diagonal matrix is estimated. This can be helpful to get more expressive posterior covariances when using the EK0, since the individual dimensions can be adjusted separately.\n\nReferences\n\n[6] Bosch et al, \"Calibrated Adaptive Probabilistic ODE Solvers\", AISTATS (2021)\n\n\n\n\n\n","category":"type"},{"location":"diffusions/#ProbNumDiffEq.FixedMVDiffusion","page":"Diffusion models and calibration","title":"ProbNumDiffEq.FixedMVDiffusion","text":"FixedMVDiffusion(; initial_diffusion=1.0, calibrate=true)\n\nTime-fixed, diagonal diffusion, which is quasi-maximum-likelihood-estimated at each step.\n\nOnly works with the EK0!\n\nA multi-variate version of FixedDiffusion, where instead of an isotropic matrix, a diagonal matrix is estimated. This can be helpful to get more expressive posterior covariances when using the EK0, since the individual dimensions can be adjusted separately.\n\nReferences\n\n[6] Bosch et al, \"Calibrated Adaptive Probabilistic ODE Solvers\", AISTATS (2021)\n\n\n\n\n\n","category":"type"},{"location":"diffusions/#diffusionrefs","page":"Diffusion models and calibration","title":"References","text":"","category":"section"},{"location":"diffusions/","page":"Diffusion models and calibration","title":"Diffusion models and calibration","text":"Pages = []\nCanonical = false\n\nbosch20capos","category":"page"},{"location":"priors/#Priors","page":"Priors","title":"Priors","text":"","category":"section"},{"location":"priors/","page":"Priors","title":"Priors","text":"We model the ODE solution y(t) with a Gauss–Markov prior. More precisely, let","category":"page"},{"location":"priors/","page":"Priors","title":"Priors","text":"beginaligned\nY(t) = left Y^(0)(t) Y^(1)(t) dots Y^(q)(t) right\nendaligned","category":"page"},{"location":"priors/","page":"Priors","title":"Priors","text":"be the solution to the SDE","category":"page"},{"location":"priors/","page":"Priors","title":"Priors","text":"beginaligned\ntextd Y^(i)(t) = Y^(i+1)(t) textdt qquad i = 0 dots q-1 \ntextd Y^(q)(t) = textcolor389826A Y(t) textdt + textcolor4063D8Gamma textdW(t) \nY(0) sim textcolorpurple mathcalN left( mu_0 Sigma_0 right) \nendaligned","category":"page"},{"location":"priors/","page":"Priors","title":"Priors","text":"Then Y^(i)(t) models the i-th derivative of y(t). In this section, we consider choices relating to the drift matrix textcolor389826A. If you're more interested in the diffusion textcolor4063D8Gamma check out the Diffusion models and calibration section, and for info on the initial distribution textcolorpurple mathcalN left( mu_0 Sigma_0 right) check out the Initialization section.","category":"page"},{"location":"priors/","page":"Priors","title":"Priors","text":"If you're unsure which prior to use, just stick to the integrated Wiener process prior IWP! This is also the default choice for all solvers. The other priors are rather experimental / niche at the time of writing.","category":"page"},{"location":"priors/#API","page":"Priors","title":"API","text":"","category":"section"},{"location":"priors/","page":"Priors","title":"Priors","text":"IWP\nIOUP\nMatern","category":"page"},{"location":"priors/#ProbNumDiffEq.IWP","page":"Priors","title":"ProbNumDiffEq.IWP","text":"IWP([wiener_process_dimension::Integer,] num_derivatives::Integer)\n\nIntegrated Wiener process.\n\nThis is the recommended prior! It is the most well-tested prior, both in this package and in the probabilistic numerics literature in general (see the references). It is also the prior that has the most efficient implementation.\n\nThe IWP can be created without specifying the dimension of the Wiener process, in which case it will be inferred from the dimension of the ODE during the solve. This is typically the preferred usage.\n\nIn math\n\nbeginaligned\ntextd Y^(i)(t) = Y^(i+1)(t) textdt qquad i = 0 dots q-1 \ntextd Y^(q)(t) = Gamma textdW(t)\nendaligned\n\nExamples\n\njulia> solve(prob, EK1(prior=IWP(2)))\n\n\n\n\n\n","category":"type"},{"location":"priors/#ProbNumDiffEq.IOUP","page":"Priors","title":"ProbNumDiffEq.IOUP","text":"IOUP([wiener_process_dimension::Integer,]\n num_derivatives::Integer,\n rate_parameter::Union{Number,AbstractVector,AbstractMatrix})\n\nIntegrated Ornstein–Uhlenbeck process.\n\nAs with the IWP, the IOUP can be created without specifying its dimension, in which case it will be inferred from the dimension of the ODE during the solve. This is typically the preferred usage. The rate parameter however always needs to be specified.\n\nIn math\n\nbeginaligned\ntextd Y^(i)(t) = Y^(i+1)(t) textdt qquad i = 0 dots q-1 \ntextd Y^(q)(t) = L Y^(q)(t) textdt + Gamma textdW(t)\nendaligned\n\nwhere L is the rate_parameter.\n\nExamples\n\njulia> solve(prob, EK1(prior=IOUP(2, -1)))\n\n\n\n\n\n","category":"type"},{"location":"priors/#ProbNumDiffEq.Matern","page":"Priors","title":"ProbNumDiffEq.Matern","text":"Matern([wiener_process_dimension::Integer,]\n num_derivatives::Integer,\n lengthscale::Number)\n\nMatern process.\n\nAs with the IWP, the Matern can be created without specifying its dimension, in which case it will be inferred from the dimension of the ODE during the solve. This is typically the preferred usage. The lengthscale parameter however always needs to be specified.\n\nIn math\n\nbeginaligned\ntextd Y^(i)(t) = Y^(i+1)(t) textdt qquad i = 0 dots q-1 \ntextd Y^(q)(t) = - sum_j=0^q left(\n beginpmatrix q+1 j endpmatrix\n left( fracsqrt2q - 1l right)^q-j\n Y^(j)(t) right) textdt + Gamma textdW(t)\nendaligned\n\nwhere l is the lengthscale.\n\nExamples\n\njulia> solve(prob, EK1(prior=Matern(2, 1)))\n\n\n\n\n\n","category":"type"},{"location":"tutorials/dynamical_odes/#Second-Order-ODEs-and-Energy-Preservation","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"","category":"section"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"In this tutorial we consider an energy-preserving, physical dynamical system, given by a second-order ODE.","category":"page"},{"location":"tutorials/dynamical_odes/#TL;DR:","page":"Second Order ODEs and Energy Preservation","title":"TL;DR:","text":"","category":"section"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"To efficiently solve second-order ODEs, just define the problem as a SecondOrderODEProblem.\nTo preserve constant quantities, use the ManifoldUpdate callback; same syntax as DiffEqCallback.jl's ManifoldProjection.","category":"page"},{"location":"tutorials/dynamical_odes/#Simulating-the-Hénon-Heiles-system","page":"Second Order ODEs and Energy Preservation","title":"Simulating the Hénon-Heiles system","text":"","category":"section"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"The Hénon-Heiles model describes the motion of a star around a galactic center, restricted to a plane. It is given by a second-order ODE","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"beginaligned\nddotx = - x - 2 x y \nddoty = y^2 - y - x^2\nendaligned","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"Our goal is to numerically simulate this system on a time span t in 0 T, starting with initial values x(0)=0, y(0) = 01, dotx(0) = 05, doty(0) = 0.","category":"page"},{"location":"tutorials/dynamical_odes/#Transforming-the-problem-into-a-first-order-ODE","page":"Second Order ODEs and Energy Preservation","title":"Transforming the problem into a first-order ODE","text":"","category":"section"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"A very common approach is to first transform the problem into a first-order ODE by introducing a new variable","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"u = dxdyxy","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"to obtain","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"beginaligned\ndotu_1(t) = - u_3 - 2 u_3 u_4 \ndotu_2(t) = u_4^2 - u_4 - u_4^2 \ndotu_3(t) = u_1 \ndotu_4(t) = u_2\nendaligned","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"This first-order ODE can then be solved using any conventional ODE solver - including our EK1:","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"using ProbNumDiffEq, Plots\n\nfunction Hénon_Heiles(du, u, p, t)\n du[1] = -u[3] - 2 * u[3] * u[4]\n du[2] = u[4]^2 - u[4] - u[3]^2\n du[3] = u[1]\n du[4] = u[2]\nend\nu0, du0 = [0.0, 0.1], [0.5, 0.0]\ntspan = (0.0, 100.0)\nprob = ODEProblem(Hénon_Heiles, [du0; u0], tspan)\nsol = solve(prob, EK1());\nplot(sol, vars=(3, 4)) # where `vars=(3,4)` is used to plot x agains y","category":"page"},{"location":"tutorials/dynamical_odes/#Solving-the-second-order-ODE-directly","page":"Second Order ODEs and Energy Preservation","title":"Solving the second-order ODE directly","text":"","category":"section"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"Instead of first transforming the problem, we can also solve it directly as a second-order ODE, by defining it as a SecondOrderODEProblem.","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"note: Note\nThe SecondOrderODEProblem type is not defined in ProbNumDiffEq.jl but is provided by SciMLBase.jl. For more information, check out the DifferentialEquations.jl documentation on Dynamical, Hamiltonian and 2nd Order ODE Problems.","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"function Hénon_Heiles2(ddu, du, u, p, t)\n ddu[1] = -u[1] - 2 * u[1] * u[2]\n ddu[2] = u[2]^2 - u[2] - u[1]^2\nend\nprob2 = SecondOrderODEProblem(Hénon_Heiles2, du0, u0, tspan)\nsol2 = solve(prob2, EK1());\nplot(sol2, vars=(3, 4))","category":"page"},{"location":"tutorials/dynamical_odes/#Benchmark:-Solving-second-order-ODEs-is-*faster*","page":"Second Order ODEs and Energy Preservation","title":"Benchmark: Solving second order ODEs is faster","text":"","category":"section"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"Solving second-order ODEs is not just a matter of convenience - in fact, SciMLBase's SecondOrderODEProblem is neatly designed in such a way that all the classic solvers from OrdinaryDiffEq.jl can handle it by solving the corresponding first-order ODE. But, transforming the ODE to first order increases the dimensionality of the problem, and comes therefore at increased computational cost; this also motivates classic specialized solvers for second-order ODEs.","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"The probabilistic numerical solvers from ProbNumDiffEq.jl have the same internal state representation for first and second order ODEs; all that changes is the measurement model [1]. As a result, we can use the EK1 both for first and second order ODEs, but it automatically specializes on the latter to provide a 2x performance boost:","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"julia> @btime solve(prob, EK1(order=3), adaptive=false, dt=1e-2);\n 766.312 ms (400362 allocations: 173.38 MiB)\n\njulia> @btime solve(prob2, EK1(order=4), adaptive=false, dt=1e-2);\n 388.301 ms (510676 allocations: 102.78 MiB)","category":"page"},{"location":"tutorials/dynamical_odes/#Energy-preservation","page":"Second Order ODEs and Energy Preservation","title":"Energy preservation","text":"","category":"section"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"In addition to the ODE given above, we know that the solution of the Hénon-Heiles model has to preserve energy over time. The total energy can be expressed as the sum of the potential and kinetic energies, given by","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"beginaligned\noperatornamePotentialEnergy(xy) = frac12 left( x^2 + y^2 + 2 x^2 y - frac2y^33 right) \noperatornameKineticEnergy(dotx doty) = frac12 left( dotx^2 + doty^2 right)\nendaligned","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"In code:","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"PotentialEnergy(x, y) = 1 // 2 * (x^2 + y^2 + 2x^2 * y - 2 // 3 * y^3)\nKineticEnergy(dx, dy) = 1 // 2 * (dx^2 + dy^2)\nE(dx, dy, x, y) = PotentialEnergy(x, y) + KineticEnergy(dx, dy)\nE(u) = E(u...); # convenient shorthand","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"So, let's have a look at how the total energy changes over time when we numerically simulate the Hénon-Heiles model over a long period of time: Standard solve","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"longprob = remake(prob2, tspan=(0.0, 1e3))\nlongsol = solve(longprob, EK1(smooth=false), dense=false)\nplot(longsol.t, E.(longsol.u))","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"It visibly loses energy over time, from an initial 0.12967 to a final 0.12899. Let's fix this to get a physically more meaningful solution.","category":"page"},{"location":"tutorials/dynamical_odes/#Energy-preservation-with-the-ManifoldUpdate-callback","page":"Second Order ODEs and Energy Preservation","title":"Energy preservation with the ManifoldUpdate callback","text":"","category":"section"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"In the language of ODE filters, preserving energy over time amounts to just another measurement model [1]. The most convenient way of updating on this additional zero measurement with ProbNumDiffEq.jl is with the ManifoldUpdate callback.","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"note: Note\nThe ManifoldUpdate callback can be thought of a probabilistic counterpart to the ManifoldProjection callback provided by DiffEqCallbacks.jl.","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"To do so, first define a (vector-valued) residual function, here chosen to be the difference between the current energy and the initial energy, and build a ManifoldUpdate callback","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"residual(u) = [E(u) - E(du0..., u0...)]\ncb = ManifoldUpdate(residual)","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"Then, solve the ODE with this callback","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"longsol_preserving = solve(longprob, EK1(smooth=false), dense=false, callback=cb)\nplot(longsol.t, E.(longsol.u))\nplot!(longsol_preserving.t, E.(longsol_preserving.u))","category":"page"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"Voilà! With the ManifoldUpdate callback we could preserve the energy over time and obtain a more truthful probabilistic numerical long-term simulation of the Hénon-Heiles model.","category":"page"},{"location":"tutorials/dynamical_odes/#References","page":"Second Order ODEs and Energy Preservation","title":"References","text":"","category":"section"},{"location":"tutorials/dynamical_odes/","page":"Second Order ODEs and Energy Preservation","title":"Second Order ODEs and Energy Preservation","text":"Pages = []\nCanonical = false\n\nbosch22pickandmix","category":"page"},{"location":"benchmarks/multi-language-wrappers/#ProbNumDiffEq.jl-vs.-various-solver-packages","page":"Multi-Language Wrapper Benchmark","title":"ProbNumDiffEq.jl vs. various solver packages","text":"","category":"section"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"Adapted from SciMLBenchmarks.jl multi-language wrapper benchmark.","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"# Imports\nusing LinearAlgebra, Statistics, InteractiveUtils\nusing StaticArrays, DiffEqDevTools, ParameterizedFunctions, Plots, SciMLBase, OrdinaryDiffEq\nusing ODEInterface, ODEInterfaceDiffEq, Sundials, SciPyDiffEq, deSolveDiffEq, MATLABDiffEq, LSODA\nusing LoggingExtras\n\nusing ProbNumDiffEq","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"# Plotting theme\ntheme(:dao;\n markerstrokewidth=0.5,\n legend=:outertopright,\n bottom_margin=5Plots.mm,\n size = (1000, 400),\n)","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"# Constants used throughout this benchmark\nconst DENSE = false # used to decide if we smooth or not\nconst SAVE_EVERYSTEP = false;","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"_COLORS = Dict(\n \"Julia\" => :LightGreen,\n \"Julia (static)\" => :DarkGreen,\n \"Hairer\" => :Red,\n \"MATLAB\" => :Orange,\n \"SciPy\" => :Yellow,\n \"deSolve\" => :Blue,\n \"Sundials\" => :Purple,\n \"liblsoda\" => :Purple,\n \"ProbNumDiffEq\" => :Darkgray,\n)\ntocolor(n) = _COLORS[split(n, ':')[1]];","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"deprecated_filter(log_args) = !contains(log_args.message, \"deprecated\")\nfiltered_logger = ActiveFilteredLogger(deprecated_filter, global_logger());","category":"page"},{"location":"benchmarks/multi-language-wrappers/#Non-Stiff-Problem-1:-Lotka-Volterra","page":"Multi-Language Wrapper Benchmark","title":"Non-Stiff Problem 1: Lotka-Volterra","text":"","category":"section"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"f = @ode_def LotkaVolterra begin\n dx = a*x - b*x*y\n dy = -c*y + d*x*y\nend a b c d\np = [1.5, 1, 3, 1]\ntspan = (0.0, 10.0)\nu0 = [1.0, 1.0]\nprob = ODEProblem{true,SciMLBase.FullSpecialize()}(f,u0,tspan,p)\nstaticprob = ODEProblem{false,SciMLBase.FullSpecialize()}(f,SVector{2}(u0),tspan,SVector{4}(p))\n\nsol = solve(prob,Vern7(),abstol=1/10^14,reltol=1/10^14,dense=false)\ntest_sol = sol\nplot(sol, title=\"Lotka-Volterra Solution\", legend=false)","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"(Image: )","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"_setups = [\n \"Julia: DP5\" => Dict(:alg=>DP5())\n \"Julia: Tsit5\" => Dict(:alg=>Tsit5())\n \"Julia: Vern7\" => Dict(:alg=>Vern7())\n \"Hairer: dopri5\" => Dict(:alg=>ODEInterfaceDiffEq.dopri5())\n \"MATLAB: ode45\" => Dict(:alg=>MATLABDiffEq.ode45())\n \"MATLAB: ode113\" => Dict(:alg=>MATLABDiffEq.ode113())\n \"SciPy: RK45\" => Dict(:alg=>SciPyDiffEq.RK45())\n \"SciPy: LSODA\" => Dict(:alg=>SciPyDiffEq.LSODA())\n \"SciPy: odeint\" => Dict(:alg=>SciPyDiffEq.odeint())\n \"deSolve: lsoda\" => Dict(:alg=>deSolveDiffEq.lsoda())\n \"deSolve: ode45\" => Dict(:alg=>deSolveDiffEq.ode45())\n \"Sundials: Adams\" => Dict(:alg=>Sundials.CVODE_Adams())\n \"ProbNumDiffEq: EK0(2)\" => Dict(:alg=>EK0(order=2, smooth=DENSE))\n \"ProbNumDiffEq: EK0(3)\" => Dict(:alg=>EK0(order=3, smooth=DENSE))\n \"ProbNumDiffEq: EK1(3)\" => Dict(:alg=>EK1(order=3, smooth=DENSE))\n \"ProbNumDiffEq: EK1(5)\" => Dict(:alg=>EK1(order=5, smooth=DENSE))\n]\n\nlabels = first.(_setups)\nsetups = last.(_setups)\ncolors = tocolor.(labels) |> permutedims\n\nabstols = 1.0 ./ 10.0 .^ (6:13)\nreltols = 1.0 ./ 10.0 .^ (3:10)\n\nwp = with_logger(filtered_logger) do\n WorkPrecisionSet(\n [prob, staticprob], abstols, reltols, setups;\n names = labels,\n #print_names = true,\n appxsol = [test_sol, test_sol],\n dense = DENSE,\n save_everystep = SAVE_EVERYSTEP,\n numruns = 10,\n maxiters = Int(1e7),\n timeseries_errors = false,\n verbose = false,\n )\nend\n\nplot(\n wp,\n title = \"Non-stiff 1: Lotka-Volterra\",\n color = colors,\n xticks = 10.0 .^ (-16:1:5),\n yticks = 10.0 .^ (-6:1:5),\n)","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"(Image: )","category":"page"},{"location":"benchmarks/multi-language-wrappers/#Non-Stiff-Problem-2:-Rigid-Body","page":"Multi-Language Wrapper Benchmark","title":"Non-Stiff Problem 2: Rigid Body","text":"","category":"section"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"f = @ode_def RigidBodyBench begin\n dy1 = -2*y2*y3\n dy2 = 1.25*y1*y3\n dy3 = -0.5*y1*y2 + 0.25*sin(t)^2\nend\nu0 = [1.0;0.0;0.9]\ntspan = (0.0, 10.0)\nprob = ODEProblem{true,SciMLBase.FullSpecialize()}(f,u0,tspan)\nstaticprob = ODEProblem{false,SciMLBase.FullSpecialize()}(f,SVector{3}(u0),tspan)\nsol = solve(prob,Vern7(),abstol=1/10^14,reltol=1/10^14,dense=false)\ntest_sol = sol\nplot(sol, title=\"Rigid Body Solution\", legend=false)","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"(Image: )","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"_setups = [\n \"Julia: DP5\" => Dict(:alg=>DP5())\n \"Julia: Tsit5\" => Dict(:alg=>Tsit5())\n \"Julia: Vern7\" => Dict(:alg=>Vern7())\n \"Hairer: dopri5\" => Dict(:alg=>dopri5())\n \"MATLAB: ode45\" => Dict(:alg=>MATLABDiffEq.ode45())\n \"MATLAB: ode113\" => Dict(:alg=>MATLABDiffEq.ode113())\n \"SciPy: RK45\" => Dict(:alg=>SciPyDiffEq.RK45())\n \"SciPy: LSODA\" => Dict(:alg=>SciPyDiffEq.LSODA())\n \"SciPy: odeint\" => Dict(:alg=>SciPyDiffEq.odeint())\n \"deSolve: lsoda\" => Dict(:alg=>deSolveDiffEq.lsoda())\n \"deSolve: ode45\" => Dict(:alg=>deSolveDiffEq.ode45())\n \"Sundials: Adams\" => Dict(:alg=>CVODE_Adams())\n \"ProbNumDiffEq: EK0(2)\" => Dict(:alg=>EK0(order=2, smooth=DENSE))\n \"ProbNumDiffEq: EK0(3)\" => Dict(:alg=>EK0(order=3, smooth=DENSE))\n \"ProbNumDiffEq: EK1(3)\" => Dict(:alg=>EK1(order=3, smooth=DENSE))\n \"ProbNumDiffEq: EK1(5)\" => Dict(:alg=>EK1(order=5, smooth=DENSE))\n]\n\nlabels = first.(_setups)\nsetups = last.(_setups)\ncolors = tocolor.(labels) |> permutedims\n\nabstols = 1.0 ./ 10.0 .^ (6:13)\nreltols = 1.0 ./ 10.0 .^ (3:10)\n\nwp = with_logger(filtered_logger) do\n WorkPrecisionSet(\n [prob,staticprob], abstols, reltols, setups;\n names = labels,\n #print_names = true,\n appxsol = [test_sol, test_sol],\n dense = DENSE,\n save_everystep = SAVE_EVERYSTEP,\n numruns = 10,\n maxiters = Int(1e7),\n timeseries_errors = false,\n verbose = false\n )\nend\n\nplot(\n wp,\n title = \"Non-stiff 2: Rigid-Body\",\n color = colors,\n xticks = 10.0 .^ (-12:1:5),\n yticks = 10.0 .^ (-6:1:5),\n)","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"(Image: )","category":"page"},{"location":"benchmarks/multi-language-wrappers/#Stiff-Problem-1:-ROBER","page":"Multi-Language Wrapper Benchmark","title":"Stiff Problem 1: ROBER","text":"","category":"section"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"rober = @ode_def begin\n dy₁ = -k₁*y₁+k₃*y₂*y₃\n dy₂ = k₁*y₁-k₂*y₂^2-k₃*y₂*y₃\n dy₃ = k₂*y₂^2\nend k₁ k₂ k₃\nu0 = [1.0,0.0,0.0]\np = [0.04,3e7,1e4]\nprob = ODEProblem{true,SciMLBase.FullSpecialize()}(rober,u0,(0.0,1e5),p)\nstaticprob = ODEProblem{false,SciMLBase.FullSpecialize()}(rober,SVector{3}(u0),(0.0,1e5),SVector{3}(p))\nsol = solve(prob,CVODE_BDF(),abstol=1/10^14,reltol=1/10^14,dense=false)\ntest_sol = sol\nplot(sol, title=\"ROBER Solution\", legend=false, xlims=(1e0, 1e5))","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"(Image: )","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"_setups = [\n \"Julia: Rosenbrock23\" => Dict(:alg=>Rosenbrock23())\n \"Julia: Rodas4\" => Dict(:alg=>Rodas4())\n \"Julia: Rodas5\" => Dict(:alg=>Rodas5())\n \"Hairer: rodas\" => Dict(:alg=>rodas())\n \"Hairer: radau\" => Dict(:alg=>radau())\n \"MATLAB: ode23s\" => Dict(:alg=>MATLABDiffEq.ode23s())\n \"MATLAB: ode15s\" => Dict(:alg=>MATLABDiffEq.ode15s())\n \"SciPy: LSODA\" => Dict(:alg=>SciPyDiffEq.LSODA())\n \"SciPy: BDF\" => Dict(:alg=>SciPyDiffEq.BDF())\n \"SciPy: odeint\" => Dict(:alg=>SciPyDiffEq.odeint())\n \"deSolve: lsoda\" => Dict(:alg=>deSolveDiffEq.lsoda())\n \"Sundials: CVODE\" => Dict(:alg=>CVODE_BDF())\n \"ProbNumDiffEq: EK1(2)\" => Dict(:alg=>EK1(order=2, smooth=DENSE))\n \"ProbNumDiffEq: EK1(3)\" => Dict(:alg=>EK1(order=3, smooth=DENSE))\n \"ProbNumDiffEq: EK1(5)\" => Dict(:alg=>EK1(order=5, smooth=DENSE))\n]\n\nlabels = first.(_setups)\nsetups = last.(_setups)\ncolors = tocolor.(labels) |> permutedims\n\nabstols = 1.0 ./ 10.0 .^ (5:12)\nreltols = 1.0 ./ 10.0 .^ (2:9)\n\nwp = with_logger(filtered_logger) do\n WorkPrecisionSet(\n [prob, staticprob], abstols, reltols, setups;\n names = labels,\n #print_names = true,\n dense = DENSE,\n verbose = false,\n save_everystep = SAVE_EVERYSTEP,\n appxsol = [test_sol, test_sol],\n maxiters=Int(1e5)\n )\nend\n\nplot(\n wp,\n title = \"Stiff 1: ROBER\",\n color = colors,\n xticks = 10.0 .^ (-16:1:4),\n yticks = 10.0 .^ (-6:1:5),\n)","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"(Image: )","category":"page"},{"location":"benchmarks/multi-language-wrappers/#Stiff-Problem-2:-HIRES","page":"Multi-Language Wrapper Benchmark","title":"Stiff Problem 2: HIRES","text":"","category":"section"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"f = @ode_def Hires begin\n dy1 = -1.71*y1 + 0.43*y2 + 8.32*y3 + 0.0007\n dy2 = 1.71*y1 - 8.75*y2\n dy3 = -10.03*y3 + 0.43*y4 + 0.035*y5\n dy4 = 8.32*y2 + 1.71*y3 - 1.12*y4\n dy5 = -1.745*y5 + 0.43*y6 + 0.43*y7\n dy6 = -280.0*y6*y8 + 0.69*y4 + 1.71*y5 -\n 0.43*y6 + 0.69*y7\n dy7 = 280.0*y6*y8 - 1.81*y7\n dy8 = -280.0*y6*y8 + 1.81*y7\nend\n\nu0 = zeros(8)\nu0[1] = 1\nu0[8] = 0.0057\nprob = ODEProblem{true,SciMLBase.FullSpecialize()}(f,u0,(0.0,321.8122))\nstaticprob = ODEProblem{false,SciMLBase.FullSpecialize()}(f,SVector{8}(u0),(0.0,321.8122))\n\nsol = solve(prob,Rodas5(),abstol=1/10^14,reltol=1/10^14, dense=false)\ntest_sol = sol\nplot(sol, title=\"ROBER Solution\", legend=false)","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"(Image: )","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"_setups = [\n \"Julia: Rosenbrock23\" => Dict(:alg=>Rosenbrock23())\n \"Julia: Rodas4\" => Dict(:alg=>Rodas4())\n \"Julia: radau\" => Dict(:alg=>RadauIIA5())\n \"Hairer: rodas\" => Dict(:alg=>rodas())\n \"Hairer: radau\" => Dict(:alg=>radau())\n \"MATLAB: ode23s\" => Dict(:alg=>MATLABDiffEq.ode23s())\n \"MATLAB: ode15s\" => Dict(:alg=>MATLABDiffEq.ode15s())\n \"SciPy: LSODA\" => Dict(:alg=>SciPyDiffEq.LSODA())\n \"SciPy: BDF\" => Dict(:alg=>SciPyDiffEq.BDF())\n \"SciPy: odeint\" => Dict(:alg=>SciPyDiffEq.odeint())\n \"deSolve: lsoda\" => Dict(:alg=>deSolveDiffEq.lsoda())\n \"Sundials: CVODE\" => Dict(:alg=>CVODE_BDF())\n \"ProbNumDiffEq: EK1(2)\" => Dict(:alg=>EK1(order=2, smooth=DENSE))\n \"ProbNumDiffEq: EK1(3)\" => Dict(:alg=>EK1(order=3, smooth=DENSE))\n \"ProbNumDiffEq: EK1(5)\" => Dict(:alg=>EK1(order=5, smooth=DENSE))\n]\n\nlabels = first.(_setups)\nsetups = last.(_setups)\ncolors = tocolor.(labels) |> permutedims\n\nabstols = 1.0 ./ 10.0 .^ (5:10)\nreltols = 1.0 ./ 10.0 .^ (1:6)\n\nwp = with_logger(filtered_logger) do\n WorkPrecisionSet(\n [prob, staticprob], abstols, reltols, setups;\n names = labels,\n #print_names = true,\n dense = false,\n verbose = false,\n save_everystep = false,\n appxsol = [test_sol, test_sol],\n maxiters = Int(1e5),\n numruns=100\n )\nend\n\nplot(\n wp,\n title = \"Stiff 2: Hires\",\n color=colors,\n xticks = 10.0 .^ (-16:1:4),\n yticks = 10.0 .^ (-6:1:5),\n)","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"(Image: )","category":"page"},{"location":"benchmarks/multi-language-wrappers/#Appendix","page":"Multi-Language Wrapper Benchmark","title":"Appendix","text":"","category":"section"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"Computer information:","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"InteractiveUtils.versioninfo()","category":"page"},{"location":"benchmarks/multi-language-wrappers/","page":"Multi-Language Wrapper Benchmark","title":"Multi-Language Wrapper Benchmark","text":"Julia Version 1.8.5\nCommit 17cfb8e65ea (2023-01-08 06:45 UTC)\nPlatform Info:\n OS: Linux (x86_64-linux-gnu)\n CPU: 12 × Intel(R) Core(TM) i7-6800K CPU @ 3.40GHz\n WORD_SIZE: 64\n LIBM: libopenlibm\n LLVM: libLLVM-13.0.1 (ORCJIT, broadwell)\n Threads: 12 on 12 virtual cores\nEnvironment:\n JULIA_NUM_THREADS = auto\n JULIA_STACKTRACE_MINIMAL = true","category":"page"},{"location":"tutorials/getting_started/#Solving-ODEs-with-Probabilistic-Numerics","page":"Getting Started","title":"Solving ODEs with Probabilistic Numerics","text":"","category":"section"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"In this tutorial we solve a simple non-linear ordinary differential equation (ODE) with the probabilistic numerical ODE solvers implemented in this package.","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"note: Note\nIf you never used DifferentialEquations.jl, check out their \"Getting Started with Differential Equations in Julia\" tutorial. It explains how to define and solve ODE problems and how to analyze the solution, so it's a great starting point. Most of ProbNumDiffEq.jl works exaclty as you would expect from DifferentialEquations.jl – just with some added uncertainties and related functionality on top!","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"In this tutorial, we consider a Fitzhugh-Nagumo model described by an ODE of the form","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"beginaligned\ndoty_1 = c (y_1 - fracy_1^33 + y_2) \ndoty_2 = -frac1c (y_1 - a - b y_2)\nendaligned","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"on a time span t in 0 T, with initial value y(0) = y_0. In the following, we","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"define the problem with explicit choices of initial values, integration domains, and parameters,\nsolve the problem with our ODE filters, and\nvisualize the results and the corresponding uncertainties.","category":"page"},{"location":"tutorials/getting_started/#TL;DR:-Just-use-DifferentialEquations.jl-with-the-EK1-algorithm","page":"Getting Started","title":"TL;DR: Just use DifferentialEquations.jl with the EK1 algorithm","text":"","category":"section"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"using ProbNumDiffEq, Plots\n\nfunction fitz(du, u, p, t)\n a, b, c = p\n du[1] = c * (u[1] - u[1]^3 / 3 + u[2])\n du[2] = -(1 / c) * (u[1] - a - b * u[2])\nend\nu0 = [-1.0; 1.0]\ntspan = (0.0, 20.0)\np = (0.2, 0.2, 3.0)\nprob = ODEProblem(fitz, u0, tspan, p)\n\nusing Logging; Logging.disable_logging(Logging.Warn); # hide\nsol = solve(prob, EK1())\nLogging.disable_logging(Logging.Debug) # hide\nplot(sol)","category":"page"},{"location":"tutorials/getting_started/#Step-1:-Define-the-problem","page":"Getting Started","title":"Step 1: Define the problem","text":"","category":"section"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"First, import ProbNumDiffEq.jl","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"using ProbNumDiffEq","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"Then, set up the ODEProblem exactly as you would in DifferentialEquations.jl. Define the vector field","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"function fitz(du, u, p, t)\n a, b, c = p\n du[1] = c * (u[1] - u[1]^3 / 3 + u[2])\n du[2] = -(1 / c) * (u[1] - a - b * u[2])\nend\nnothing # hide","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"and then the ODEProblem, with initial value u0, time span tspan, and parameters p","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"u0 = [-1.0; 1.0]\ntspan = (0.0, 20.0)\np = (0.2, 0.2, 3.0)\nprob = ODEProblem(fitz, u0, tspan, p)\nnothing # hide","category":"page"},{"location":"tutorials/getting_started/#Step-2:-Solve-the-problem","page":"Getting Started","title":"Step 2: Solve the problem","text":"","category":"section"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"To solve the ODE we just use DifferentialEquations.jl's solve interface, together with one of the algorithms implemented in this package. For now, let's use EK1:","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"sol = solve(prob, EK1())\n# nothing # hide","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"That's it! we just computed a probabilistic numerical ODE solution!","category":"page"},{"location":"tutorials/getting_started/#Step-3:-Analyze-the-solution","page":"Getting Started","title":"Step 3: Analyze the solution","text":"","category":"section"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"Let's plot the result with Plots.jl.","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"using Plots\nplot(sol)","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"Looks good! Looks like the EK1 managed to solve the Fitzhugh-Nagumo problem quite well.","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"tip: Tip\nTo learn more about plotting ODE solutions, check out the plotting tutorial for DifferentialEquations.jl + Plots.jl provided here. Most of that works exactly as expected with ProbNumDiffEq.jl.","category":"page"},{"location":"tutorials/getting_started/#Plot-the-probabilistic-error-estimates","page":"Getting Started","title":"Plot the probabilistic error estimates","text":"","category":"section"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"The plot above looks like a standard ODE solution – but it's not! The numerical errors are just so small that we can't see them in the plot, and the probabilistic error estimates are too. We can visualize them by plotting the errors and error estimates directly:","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"using OrdinaryDiffEq, Statistics\nreference = solve(prob, Vern9(), abstol=1e-9, reltol=1e-9, saveat=sol.t)\nerrors = reduce(hcat, mean.(sol.pu) .- reference.u)'\nerror_estimates = reduce(hcat, std.(sol.pu))'\nplot(sol.t, errors, label=\"error\", color=[1 2], xlabel=\"t\", ylabel=\"err\")\nplot!(sol.t, zero(errors), ribbon=3error_estimates, label=\"error estimate\",\n color=[1 2], alpha=0.2)","category":"page"},{"location":"tutorials/getting_started/#More-about-the-ProbabilisticODESolution","page":"Getting Started","title":"More about the ProbabilisticODESolution","text":"","category":"section"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"The solution object returned by ProbNumDiffEq.jl mostly behaves just like any other ODESolution in DifferentialEquations.jl – with some added uncertainties and related functionality on top. So, sol can be indexed","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"sol[1]\nsol[end]","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"and has fields sol.t and sol.u which store the time points and mean estimates:","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"sol.t[end]\nsol.u[end]","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"But since sol is a probabilistic numerical ODE solution, it contains a Gaussian distributions over solution values. The marginals of this posterior are stored in sol.pu:","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"sol.pu[end]","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"You can compute means, covariances, and standard deviations via Statistics.jl:","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"using Statistics\nmean(sol.pu[5])\ncov(sol.pu[5])\nstd(sol.pu[5])","category":"page"},{"location":"tutorials/getting_started/#Dense-output","page":"Getting Started","title":"Dense output","text":"","category":"section"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"Probabilistic numerical ODE solvers approximate the posterior distribution","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"p Big( y(t) big y(0) = y_0 doty(t_i) = f_theta(y(t_i) t_i) Big)","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"which describes a posterior not just for the discrete steps but for any t in the continuous space t in 0 T; in classic ODE solvers, this is also known as \"interpolation\" or \"dense output\". The probabilistic solutions returned by our solvers can be interpolated as usual by treating them as functions, but they return Gaussian distributions","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"sol(0.45)\nmean(sol(0.45))","category":"page"},{"location":"tutorials/getting_started/#Next-steps","page":"Getting Started","title":"Next steps","text":"","category":"section"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"Check out one of the other tutorials:","category":"page"},{"location":"tutorials/getting_started/","page":"Getting Started","title":"Getting Started","text":"\"Second Order ODEs and Energy Preservation\" explains how to solve second-order ODEs more efficiently while also better perserving energy or other conserved quantities;\n\"Solving DAEs with Probabilistic Numerics\" demonstrates how to solve differential algebraic equatios in a probabilistic numerical way.","category":"page"},{"location":"tutorials/fenrir/#Parameter-Inference-with-ProbNumDiffEq.jl-and-Fenrir.jl","page":"Parameter Inference","title":"Parameter Inference with ProbNumDiffEq.jl and Fenrir.jl","text":"","category":"section"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"note: Note\nThis is mostly just a copy from the tutorial included in the Fenrir.jl documentation, so have a look there too!","category":"page"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"using LinearAlgebra\nusing OrdinaryDiffEq, ProbNumDiffEq, Plots\nusing Fenrir\nusing Optimization, OptimizationOptimJL\nstack(x) = copy(reduce(hcat, x)') # convenient\nnothing # hide","category":"page"},{"location":"tutorials/fenrir/#The-parameter-inference-problem-in-general","page":"Parameter Inference","title":"The parameter inference problem in general","text":"","category":"section"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"Let's assume we have an initial value problem (IVP)","category":"page"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"beginaligned\ndoty = f_theta(y t) qquad y(t_0) = y_0\nendaligned","category":"page"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"which we observe through a set mathcalD = u(t_n)_n=1^N of noisy data points","category":"page"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"beginaligned\nu(t_n) = H y(t_n) + v_n qquad v_n sim mathcalN(0 R)\nendaligned","category":"page"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"The question of interest is: How can we compute the marginal likelihood p(mathcalD mid theta)? Short answer: We can't. It's intractable, because computing the true IVP solution exactly y(t) is intractable. What we can do however is compute an approximate marginal likelihood. This is what Fenrir.jl provides. For details, check out the paper.","category":"page"},{"location":"tutorials/fenrir/#The-specific-problem,-in-code","page":"Parameter Inference","title":"The specific problem, in code","text":"","category":"section"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"Let's assume that the true underlying dynamics are given by a FitzHugh-Nagumo model","category":"page"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"function f(du, u, p, t)\n a, b, c = p\n du[1] = c*(u[1] - u[1]^3/3 + u[2])\n du[2] = -(1/c)*(u[1] - a - b*u[2])\nend\nu0 = [-1.0, 1.0]\ntspan = (0.0, 20.0)\np = (0.2, 0.2, 3.0)\ntrue_prob = ODEProblem(f, u0, tspan, p)","category":"page"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"from which we generate some artificial noisy data","category":"page"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"true_sol = solve(true_prob, Vern9(), abstol=1e-10, reltol=1e-10)\n\ntimes = 1:0.5:20\nobservation_noise_var = 1e-1\nodedata = [true_sol(t) .+ sqrt(observation_noise_var) * randn(length(u0)) for t in times]\n\nplot(true_sol, color=:black, linestyle=:dash, label=[\"True Solution\" \"\"])\nscatter!(times, stack(odedata), markersize=2, markerstrokewidth=0.1, color=1, label=[\"Noisy Data\" \"\"])","category":"page"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"Our goal is then to recover the true parameter p (and thus also the true trajectoy plotted above) the noisy data.","category":"page"},{"location":"tutorials/fenrir/#Computing-the-negative-log-likelihood","page":"Parameter Inference","title":"Computing the negative log-likelihood","text":"","category":"section"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"To do parameter inference - be it maximum-likelihod, maximum a posteriori, or full Bayesian inference with MCMC - we need to evaluate the likelihood of given a parameter estimate theta_textest. This is exactly what Fenrir.jl's fenrir_nll provides:","category":"page"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"p_est = (0.1, 0.1, 2.0)\nprob = remake(true_prob, p=p_est)\ndata = (t=times, u=odedata)\nκ² = 1e10\nnll, _, _ = fenrir_nll(prob, data, observation_noise_var, κ²; dt=1e-1)\nnll","category":"page"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"This is the negative marginal log-likelihood of the parameter p_est. You can use it as any other NLL: Optimize it to compute maximum-likelihood estimates or MAPs, or plug it into MCMC to sample from the posterior. In our paper [3] we compute MLEs by pairing Fenrir with Optimization.jl and ForwardDiff.jl. Let's quickly explore how to do this next.","category":"page"},{"location":"tutorials/fenrir/#Maximum-likelihood-parameter-inference","page":"Parameter Inference","title":"Maximum-likelihood parameter inference","text":"","category":"section"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"To compute a maximum-likelihood estimate (MLE), we just need to maximize theta to p(mathcalD mid theta) - that is, minimize the nll from above. We use Optimization.jl for this. First, define a loss function and create an OptimizationProblem","category":"page"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"function loss(x, _)\n ode_params = x[begin:end-1]\n prob = remake(true_prob, p=ode_params)\n κ² = exp(x[end]) # the diffusion parameter of the EK1\n return fenrir_nll(prob, data, observation_noise_var, κ²; dt=1e-1)\nend\n\nfun = OptimizationFunction(loss, Optimization.AutoForwardDiff())\noptprob = OptimizationProblem(\n fun, [p_est..., 1e0];\n lb=[0.0, 0.0, 0.0, -10], ub=[1.0, 1.0, 5.0, 20] # lower and upper bounds\n)","category":"page"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"Then, just solve it! Here we use LBFGS:","category":"page"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"optsol = solve(optprob, LBFGS())\np_mle = optsol.u[1:3]\np_mle # hide","category":"page"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"Success! The computed MLE is quite close to the true parameter which we used to generate the data. As a final step, let's plot the true solution, the data, and the result of the MLE:","category":"page"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"plot(true_sol, color=:black, linestyle=:dash, label=[\"True Solution\" \"\"])\nscatter!(times, stack(odedata), markersize=2, markerstrokewidth=0.1, color=1, label=[\"Noisy Data\" \"\"])\nmle_sol = solve(remake(true_prob, p=p_mle), EK1())\nplot!(mle_sol, color=3, label=[\"MLE-parameter Solution\" \"\"])","category":"page"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"Looks good!","category":"page"},{"location":"tutorials/fenrir/#Reference","page":"Parameter Inference","title":"Reference","text":"","category":"section"},{"location":"tutorials/fenrir/","page":"Parameter Inference","title":"Parameter Inference","text":"Pages = []\nCanonical = false\n\ntronarp22fenrir","category":"page"},{"location":"#Probabilistic-Numerical-Differential-Equation-Solvers","page":"Home","title":"Probabilistic Numerical Differential Equation Solvers","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"(Image: Banner)","category":"page"},{"location":"","page":"Home","title":"Home","text":"ProbNumDiffEq.jl provides probabilistic numerical solvers to the DifferentialEquations.jl ecosystem. The implemented ODE filters solve differential equations via Bayesian filtering and smoothing and compute not just a single point estimate of the true solution, but a posterior distribution that contains an estimate of its numerical approximation error.","category":"page"},{"location":"","page":"Home","title":"Home","text":"For a short intro video, check out our poster presentation at JuliaCon2021.","category":"page"},{"location":"#Installation","page":"Home","title":"Installation","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"Run Julia, enter ] to bring up Julia's package manager, and add the ProbNumDiffEq.jl package:","category":"page"},{"location":"","page":"Home","title":"Home","text":"julia> ]\n(v1.9) pkg> add ProbNumDiffEq","category":"page"},{"location":"#Getting-Started","page":"Home","title":"Getting Started","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"For a quick introduction check out the \"Solving ODEs with Probabilistic Numerics\" tutorial.","category":"page"},{"location":"#Features","page":"Home","title":"Features","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"Two extended Kalman filtering-based probabilistic solvers: the explicit EK0 and semi-implicit EK1.\nAdaptive step-size selection with PI control; fully compatible with DifferentialEquations.jl's timestepping options\nOnline uncertainty calibration for multiple different diffusion models (see \"Diffusion models and calibration\")\nDense output\nSampling from the solution\nCallback support\nConvenient plotting through a Plots.jl recipe\nAutomatic differentiation via ForwardDiff.jl\nArbitrary precision via Julia's built-in arbitrary precision arithmetic\nSpecialized solvers for second-order ODEs (see Second Order ODEs and Energy Preservation)\nCompatible with DAEs in mass-matrix ODE form (see Solving DAEs with Probabilistic Numerics)","category":"page"},{"location":"#Related-packages","page":"Home","title":"Related packages","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"probdiffeq: Fast and feature-rich filtering-based probabilistic ODE solvers in JAX.\nProbNum: Probabilistic numerics in Python. It has not only probabilistic ODE solvers, but also probabilistic linear solvers, Bayesian quadrature, and many filtering and smoothing implementations.\nFenrir.jl: Parameter-inference in ODEs with probabilistic ODE solvers. This package builds on ProbNumDiffEq.jl to provide a negative marginal log-likelihood function, which can then be used with an optimizer or with MCMC for parameter inference.","category":"page"},{"location":"implementation/#Solver-Implementation-via-OrdinaryDiffEq.jl","page":"Implementation via OrdinaryDiffEq.jl","title":"Solver Implementation via OrdinaryDiffEq.jl","text":"","category":"section"},{"location":"implementation/","page":"Implementation via OrdinaryDiffEq.jl","title":"Implementation via OrdinaryDiffEq.jl","text":"ProbNumDiffEq.jl builds directly on OrdinaryDiffEq.jl to benefit from its iterator interface, flexible step-size control, and efficient Jacobian calculations. But, this requires extending non-public APIs. This page is meant to provide an overview on which parts exactly ProbNumDiffEq.jl builds on.","category":"page"},{"location":"implementation/","page":"Implementation via OrdinaryDiffEq.jl","title":"Implementation via OrdinaryDiffEq.jl","text":"For more discussion on the pros and cons of building on OrdinaryDiffEq.jl, see this thread on discourse.","category":"page"},{"location":"implementation/#Building-on-OrdinaryDiffEq.jl","page":"Implementation via OrdinaryDiffEq.jl","title":"Building on OrdinaryDiffEq.jl","text":"","category":"section"},{"location":"implementation/","page":"Implementation via OrdinaryDiffEq.jl","title":"Implementation via OrdinaryDiffEq.jl","text":"ProbNumDiffEq.jl shares most of OrdinaryDiffEq.jl's implementation. In particular:","category":"page"},{"location":"implementation/","page":"Implementation via OrdinaryDiffEq.jl","title":"Implementation via OrdinaryDiffEq.jl","text":"OrdinaryDiffEq.__init builds the cache and the integrator, and calls OrdinaryDiffEq.initialize!\nOrdinaryDiffEq.solve! implements the actual iterator structure, with\nOrdinaryDiffEq.loopheader!\nOrdinaryDiffEq.perform_step!\nOrdinaryDiffEq.loopfooter!\nOrdinaryDiffEq.postamble!","category":"page"},{"location":"implementation/","page":"Implementation via OrdinaryDiffEq.jl","title":"Implementation via OrdinaryDiffEq.jl","text":"ProbNumDiffEq.jl builds around this structure and overloads some of the parts:","category":"page"},{"location":"implementation/","page":"Implementation via OrdinaryDiffEq.jl","title":"Implementation via OrdinaryDiffEq.jl","text":"Algorithms: EK0/EK1 <: AbstractEK <: OrdinaryDiffEq.OrdinaryDiffEqAdaptiveAlgorithm\n./src/algorithms.jl provides the algorithms themselves\n./src/alg_utils.jl implements many traits (e.g. relating to autodiff, implicitness, step-size control)\nCache: EKCache <: AbstractODEFilterCache <: OrdinaryDiffEq.OrdinaryDiffEqCache\n./src/caches.jl implements the cache and its main constructor: OrdinaryDiffEq.alg_cache\nInitialization and perform_step!: via OrdinaryDiffEq.initialize! and OrdinaryDiffEq.perform_step!. Implemented in ./src/perform_step.jl.\nCustom postamble by overloading OrdinaryDiffEq.postamble! (which should always call OrdinaryDiffEq._postamble!). This is where we do the \"smoothing\" of the solution. Implemented in ./src/integrator_utils.jl. \nCustom saving by overloading OrdinaryDiffEq.savevalues! (which should always call OrdinaryDiffEq._savevalues!). Implemented in ./src/integrator_utils.jl.","category":"page"},{"location":"implementation/#Building-on-DiffEqBase.jl","page":"Implementation via OrdinaryDiffEq.jl","title":"Building on DiffEqBase.jl","text":"","category":"section"},{"location":"implementation/","page":"Implementation via OrdinaryDiffEq.jl","title":"Implementation via OrdinaryDiffEq.jl","text":"DiffEqBase.__init is currently overloaded to transform OOP problems into IIP problems (in ./src/solve.jl).\nThe solution object: ProbODESolution <: AbstractProbODESolution <: DiffEqBase.AbstractODESolution\n./src/solution.jl implements the main parts. Note that the main constructor DiffEqBase.build_solution is called by OrdinaryDiffEq.__init - so OrdinaryDiffEq.jl has control over its inputs.\nThere is also MeanProbODESolution <: DiffEqBase.AbstractODESolution: It allows handling the mean of a probabilistic ODE solution the same way one would handle any \"standard\" ODE solution - e.g. it is compatible with DiffEqDevTools.appxtrue.\nAbstractODEFilterPosterior <: DiffEqBase.AbstractDiffEqInterpolation is the current interpolant, but it does not actually fully handle the interpolation right now. This part might be subject to change soon.\nPlot recipe in ./src/solution_plotting.jl\nSampling in ./src/solution_sampling.jl\nDiffEqBase.prepare_alg(::EK1{0}); closely follows a similar function implemented in OrdinaryDiffEq.jl ./src/alg_utils.jl\nthis also required DiffEqBase.remake(::EK1)","category":"page"},{"location":"implementation/#Other-packages","page":"Implementation via OrdinaryDiffEq.jl","title":"Other packages","text":"","category":"section"},{"location":"implementation/","page":"Implementation via OrdinaryDiffEq.jl","title":"Implementation via OrdinaryDiffEq.jl","text":"DiffEqDevTools.appxtrue is overloaded to work with ProbODESolution (by just doing mean(sol)). This also enables DiffEqDevTools.WorkPrecision to work out of th box.","category":"page"}]
}
diff --git a/dev/solvers/index.html b/dev/solvers/index.html
index 8a82ed7a1..45e8991d0 100644
--- a/dev/solvers/index.html
+++ b/dev/solvers/index.html
@@ -1,13 +1,17 @@
-Solvers · ProbNumDiffEq.jl
ProbNumDiffEq.jl provides two solvers: the EK1 and the EK0. Both based on extended Kalman filtering and smoothing, but the latter relies on evaluating the Jacobian of the vector field. For the best results, use the EK1.
All solvers are compatible with DAEs in mass-matrix ODE form. They also specialize on second-order ODEs: If the problem is of type SecondOrderODEProblem, it solves the second-order problem directly; this is more efficient than solving the transformed first-order problem and provides more meaningful posteriors [1].
ProbNumDiffEq.jl provides two solvers: the EK1 and the EK0. Both based on extended Kalman filtering and smoothing, but the latter relies on evaluating the Jacobian of the vector field. For the best results, use the EK1.
All solvers are compatible with DAEs in mass-matrix ODE form. They also specialize on second-order ODEs: If the problem is of type SecondOrderODEProblem, it solves the second-order problem directly; this is more efficient than solving the transformed first-order problem and provides more meaningful posteriors [1].
Gaussian ODE filter with first-order vector field linearization.
Arguments
order::Integer: Order of the integrated Wiener process (IWP) prior.
smooth::Bool: Turn smoothing on/off; smoothing is required for dense output.
prior::AbstractODEFilterPrior: Prior to be used by the ODE filter. By default, uses a 3-times integrated Wiener process prior IWP(3). See also: Priors.
initialization::ProbNumDiffEq.InitializationScheme: See Initialization.
Some additional kwargs relating to implicit solvers are supported; check out DifferentialEquations.jl's Extra Options page. Right now, we support autodiff, chunk_size, and diff_type. In particular, autodiff=false can come in handy to use finite differences instead of ForwardDiff.jl to compute Jacobians.
Gaussian ODE filter with first-order vector field linearization.
Arguments
order::Integer: Order of the integrated Wiener process (IWP) prior.
smooth::Bool: Turn smoothing on/off; smoothing is required for dense output.
prior::AbstractODEFilterPrior: Prior to be used by the ODE filter. By default, uses a 3-times integrated Wiener process prior IWP(3). See also: Priors.
initialization::ProbNumDiffEq.InitializationScheme: See Initialization.
Some additional kwargs relating to implicit solvers are supported; check out DifferentialEquations.jl's Extra Options page. Right now, we support autodiff, chunk_size, and diff_type. In particular, autodiff=false can come in handy to use finite differences instead of ForwardDiff.jl to compute Jacobians.
Gaussian ODE filter with zeroth-order vector field linearization.
Arguments
order::Integer: Order of the integrated Wiener process (IWP) prior.
smooth::Bool: Turn smoothing on/off; smoothing is required for dense output.
prior::AbstractODEFilterPrior: Prior to be used by the ODE filter. By default, uses a 3-times integrated Wiener process prior IWP(3). See also: Priors.
Probabilistic exponential integrators are a class of integrators for semi-linear stiff ODEs that provide improved stability by essentially solving the linear part of the ODE exactly. In probabilistic numerics, this amounts to including the linear part into the prior model of the solver.
ExpEK is therefore just a short-hand for EK0 with IOUP prior:
A probabilistic exponential integrator similar to ExpEK, but with automatic linearization along the mean numerical solution. This brings the advantage that the linearity does not need to be specified manually, and the more accurate local linearization can sometimes also improve stability; but since the "prior" is adjusted at each step the probabilistic interpretation becomes more complicated.
RosenbrockExpEK is just a short-hand for EK1 with appropriete IOUP prior:
[1] N. Bosch, F. Tronarp, P. Hennig: Pick-and-Mix Information Operators for Probabilistic ODE Solvers (2022) (link)
Settings
This document was generated with Documenter.jl version 0.27.25 on Tuesday 4 July 2023. Using Julia version 1.9.1.
+ initialization=TaylorModeInit())
Gaussian ODE filter with zeroth-order vector field linearization.
Arguments
order::Integer: Order of the integrated Wiener process (IWP) prior.
smooth::Bool: Turn smoothing on/off; smoothing is required for dense output.
prior::AbstractODEFilterPrior: Prior to be used by the ODE filter. By default, uses a 3-times integrated Wiener process prior IWP(3). See also: Priors.
Probabilistic exponential integrators are a class of integrators for semi-linear stiff ODEs that provide improved stability by essentially solving the linear part of the ODE exactly. In probabilistic numerics, this amounts to including the linear part into the prior model of the solver.
ExpEK is therefore just a short-hand for EK0 with IOUP prior:
A probabilistic exponential integrator similar to ExpEK, but with automatic linearization along the mean numerical solution. This brings the advantage that the linearity does not need to be specified manually, and the more accurate local linearization can sometimes also improve stability; but since the "prior" is adjusted at each step the probabilistic interpretation becomes more complicated.
RosenbrockExpEK is just a short-hand for EK1 with appropriete IOUP prior:
ProbNumDiffEq.jl provides probabilistic numerical solvers for differential algebraic equations (DAEs). Currently, we recommend using the semi-implicit EK1 algorithm.
ProbNumDiffEq.jl provides probabilistic numerical solvers for differential algebraic equations (DAEs). Currently, we recommend using the semi-implicit EK1 algorithm.
The following is based on the "Automatic Index Reduction of DAEs" tutorial by ModelingToolkit.jl, which demonstrates how the classic Rodas4 solver fails to solve a DAE due to the fact that it is of index 3; which is why ModelingToolkit's automatic index reduction is so useful.
It turns out that our probabilistic numerical solvers can directly solve the index-3 DAE!
First, define the pendulum problem as in the tutorial:
function pendulum!(du, u, p, t)
x, dx, y, dy, T = u
@@ -197,62 +197,62 @@
[0.9445849315579167, -0.8347208413797078, -0.32826712908215605, -2.3960584356227934, -9.733564093090328]
Nope! The EK1 is able to solve the index-3 DAE directly. Pretty cool!
The point of the "Automatic Index Reduction of DAEs" tutorial is to demonstrate ModelingToolkit's utility for automatic index reduction, which enables the classic implicit Runge-Kutta solvers such as Rodas5 to solve this DAE. Let's see if that still helps in this context here.
First, modelingtoolkitize the problem:
traced_sys = modelingtoolkitize(pendulum_prob)
\[ \begin{align}
\frac{\mathrm{d} x_1\left( t \right)}{\mathrm{d}t} =& x_2\left( t \right) \\
\frac{\mathrm{d} x_2\left( t \right)}{\mathrm{d}t} =& x_1\left( t \right) x_5\left( t \right) \\
@@ -283,4 +283,10 @@
[1] N. Bosch, F. Tronarp, P. Hennig: Pick-and-Mix Information Operators for Probabilistic ODE Solvers (2022) (link)
Settings
This document was generated with Documenter.jl version 0.27.25 on Tuesday 4 July 2023. Using Julia version 1.9.1.
+└ sol3_f_evals = 1389
The error for the index-1 DAE solve is much lower. So it seems that, even if the index-3 DAE could also be solved directly, index lowering might still be beneficial when solving DAEs with the EK1!
The Hénon-Heiles model describes the motion of a star around a galactic center, restricted to a plane. It is given by a second-order ODE
\[\begin{aligned}
\ddot{x} &= - x - 2 x y \\
\ddot{y} &= y^2 - y - x^2.
\end{aligned}\]
Our goal is to numerically simulate this system on a time span $t \in [0, T]$, starting with initial values $x(0)=0$, $y(0) = 0.1$, $\dot{x}(0) = 0.5$, $\dot{y}(0) = 0$.
Instead of first transforming the problem, we can also solve it directly as a second-order ODE, by defining it as a SecondOrderODEProblem.
Note
The SecondOrderODEProblem type is not defined in ProbNumDiffEq.jl but is provided by SciMLBase.jl. For more information, check out the DifferentialEquations.jl documentation on Dynamical, Hamiltonian and 2nd Order ODE Problems.
function Hénon_Heiles2(ddu, du, u, p, t)
ddu[1] = -u[1] - 2 * u[1] * u[2]
ddu[2] = u[2]^2 - u[2] - u[1]^2
@@ -74,50 +74,50 @@
Solving second-order ODEs is not just a matter of convenience - in fact, SciMLBase's SecondOrderODEProblem is neatly designed in such a way that all the classic solvers from OrdinaryDiffEq.jl can handle it by solving the corresponding first-order ODE. But, transforming the ODE to first order increases the dimensionality of the problem, and comes therefore at increased computational cost; this also motivates classic specialized solvers for second-order ODEs.
The probabilistic numerical solvers from ProbNumDiffEq.jl have the same internal state representation for first and second order ODEs; all that changes is the measurement model [1]. As a result, we can use the EK1 both for first and second order ODEs, but it automatically specializes on the latter to provide a 2x performance boost:
Solving second-order ODEs is not just a matter of convenience - in fact, SciMLBase's SecondOrderODEProblem is neatly designed in such a way that all the classic solvers from OrdinaryDiffEq.jl can handle it by solving the corresponding first-order ODE. But, transforming the ODE to first order increases the dimensionality of the problem, and comes therefore at increased computational cost; this also motivates classic specialized solvers for second-order ODEs.
The probabilistic numerical solvers from ProbNumDiffEq.jl have the same internal state representation for first and second order ODEs; all that changes is the measurement model[1]. As a result, we can use the EK1 both for first and second order ODEs, but it automatically specializes on the latter to provide a 2x performance boost:
In the language of ODE filters, preserving energy over time amounts to just another measurement model [1]. The most convenient way of updating on this additional zero measurement with ProbNumDiffEq.jl is with the ManifoldUpdate callback.
Note
The ManifoldUpdate callback can be thought of a probabilistic counterpart to the ManifoldProjection callback provided by DiffEqCallbacks.jl.
To do so, first define a (vector-valued) residual function, here chosen to be the difference between the current energy and the initial energy, and build a ManifoldUpdate callback
In the language of ODE filters, preserving energy over time amounts to just another measurement model [1]. The most convenient way of updating on this additional zero measurement with ProbNumDiffEq.jl is with the ManifoldUpdate callback.
Note
The ManifoldUpdate callback can be thought of a probabilistic counterpart to the ManifoldProjection callback provided by DiffEqCallbacks.jl.
To do so, first define a (vector-valued) residual function, here chosen to be the difference between the current energy and the initial energy, and build a ManifoldUpdate callback
Voilà! With the ManifoldUpdate callback we could preserve the energy over time and obtain a more truthful probabilistic numerical long-term simulation of the Hénon-Heiles model.
Voilà! With the ManifoldUpdate callback we could preserve the energy over time and obtain a more truthful probabilistic numerical long-term simulation of the Hénon-Heiles model.
where $L$ is a linear operator and $f$ is a nonlinear function. In a nutshell, exponential integrators solve the linear part of the ODE exactly, and only approximate the nonlinear part. Probabilistic exponential integrators are the probabilistic numerics approach to exponential integrators.
where $L$ is a linear operator and $f$ is a nonlinear function. In a nutshell, exponential integrators solve the linear part of the ODE exactly, and only approximate the nonlinear part. Probabilistic exponential integrators[2] are the probabilistic numerics approach to exponential integrators.
But for fixed (large) step sizes this ODE is more challenging: The explicit EK0 method oscillates and diverges due to the stiffness of the ODE, and the semi-implicit EK1 method is stable but the solution is not very accurate.
Probabilistic exponential integrators leverage the semi-linearity of the ODE to compute more accurate solutions for the same fixed step size. You can use either the ExpEK method and provide the linear part (with the keyword argument L), or the RosenbrockExpEK to automatically linearize along the mean of the numerical solution:
Probabilistic exponential integrators "solve the linear part exactly" by including it into the prior model of the solver. Namely, the solver chooses a (q-times) integrated Ornstein-Uhlenbeck prior with rate parameter equal to the linearity. The ExpEK solver is just a short-hand for an EK0 with appropriate prior:
This means that you can also construct other probabilistic exponential integrators by hand! In this example the EK1 with IOUP prior with rate parameter -1 performs extremely well:
To do parameter inference - be it maximum-likelihod, maximum a posteriori, or full Bayesian inference with MCMC - we need to evaluate the likelihood of given a parameter estimate $\theta_\text{est}$. This is exactly what Fenrir.jl's fenrir_nll provides:
This is the negative marginal log-likelihood of the parameter p_est. You can use it as any other NLL: Optimize it to compute maximum-likelihood estimates or MAPs, or plug it into MCMC to sample from the posterior. In our paper we compute MLEs by pairing Fenrir with Optimization.jl and ForwardDiff.jl. Let's quickly explore how to do this next.
To compute a maximum-likelihood estimate (MLE), we just need to maximize $\theta \to p(\mathcal{D} \mid \theta)$ - that is, minimize the nll from above. We use Optimization.jl for this. First, define a loss function and create an OptimizationProblem
function loss(x, _)
+nll
273.85306939704054
This is the negative marginal log-likelihood of the parameter p_est. You can use it as any other NLL: Optimize it to compute maximum-likelihood estimates or MAPs, or plug it into MCMC to sample from the posterior. In our paper [3] we compute MLEs by pairing Fenrir with Optimization.jl and ForwardDiff.jl. Let's quickly explore how to do this next.
To compute a maximum-likelihood estimate (MLE), we just need to maximize $\theta \to p(\mathcal{D} \mid \theta)$ - that is, minimize the nll from above. We use Optimization.jl for this. First, define a loss function and create an OptimizationProblem
function loss(x, _)
ode_params = x[begin:end-1]
prob = remake(true_prob, p=ode_params)
κ² = exp(x[end]) # the diffusion parameter of the EK1
@@ -175,140 +175,146 @@
2.0
1.0
Success! The computed MLE is quite close to the true parameter which we used to generate the data. As a final step, let's plot the true solution, the data, and the result of the MLE:
Success! The computed MLE is quite close to the true parameter which we used to generate the data. As a final step, let's plot the true solution, the data, and the result of the MLE:
In this tutorial we solve a simple non-linear ordinary differential equation (ODE) with the probabilistic numerical ODE solvers implemented in this package.
Note
If you never used DifferentialEquations.jl, check out their "Getting Started with Differential Equations in Julia" tutorial. It explains how to define and solve ODE problems and how to analyze the solution, so it's a great starting point. Most of ProbNumDiffEq.jl works exaclty as you would expect from DifferentialEquations.jl – just with some added uncertainties and related functionality on top!
In this tutorial, we consider a Fitzhugh-Nagumo model described by an ODE of the form
In this tutorial we solve a simple non-linear ordinary differential equation (ODE) with the probabilistic numerical ODE solvers implemented in this package.
Note
If you never used DifferentialEquations.jl, check out their "Getting Started with Differential Equations in Julia" tutorial. It explains how to define and solve ODE problems and how to analyze the solution, so it's a great starting point. Most of ProbNumDiffEq.jl works exaclty as you would expect from DifferentialEquations.jl – just with some added uncertainties and related functionality on top!
In this tutorial, we consider a Fitzhugh-Nagumo model described by an ODE of the form
\[\begin{aligned}
\dot{y}_1 &= c (y_1 - \frac{y_1^3}{3} + y_2) \\
\dot{y}_2 &= -\frac{1}{c} (y_1 - a - b y_2)
\end{aligned}\]
on a time span $t \in [0, T]$, with initial value $y(0) = y_0$. In the following, we
define the problem with explicit choices of initial values, integration domains, and parameters,
solve the problem with our ODE filters, and
visualize the results and the corresponding uncertainties.
The plot above looks like a standard ODE solution – but it's not! The numerical errors are just so small that we can't see them in the plot, and the probabilistic error estimates are too. We can visualize them by plotting the errors and error estimates directly:
The solution object returned by ProbNumDiffEq.jl mostly behaves just like any other ODESolution in DifferentialEquations.jl – with some added uncertainties and related functionality on top. So, sol can be indexed
which describes a posterior not just for the discrete steps but for any $t$ in the continuous space $t \in [0, T]$; in classic ODE solvers, this is also known as "interpolation" or "dense output". The probabilistic solutions returned by our solvers can be interpolated as usual by treating them as functions, but they return Gaussian distributions
"Second Order ODEs and Energy Preservation" explains how to solve second-order ODEs more efficiently while also better perserving energy or other conserved quantities;
"Second Order ODEs and Energy Preservation" explains how to solve second-order ODEs more efficiently while also better perserving energy or other conserved quantities;
