diff --git a/README.md b/README.md index 0aaab38..f6394fc 100644 --- a/README.md +++ b/README.md @@ -4,8 +4,8 @@ [![Dev Docs](https://img.shields.io/badge/docs-dev-blue.svg)](https://samuelsonric.github.io/AlgebraicInference.jl/dev/) [![Code Coverage](https://codecov.io/gh/samuelsonric/AlgebraicInference.jl/branch/master/graph/badge.svg?token=FJJQQCTUCF)](https://codecov.io/gh/samuelsonric/AlgebraicInference.jl) -AlgebraicInference.jl is a library for performing Bayesian inference on wiring diagrams. It -builds on [Catlab.jl](https://algebraicjulia.github.io/Catlab.jl/dev/). See the +AlgebraicInference.jl is a library for performing Bayesian inference on wiring diagrams, +building on [Catlab.jl](https://algebraicjulia.github.io/Catlab.jl/dev/). See the [documentation](https://samuelsonric.github.io/AlgebraicInference.jl/dev/) for example notebooks and an API. @@ -48,7 +48,8 @@ marginalized out during composition. Hence, a wiring diagram can be thought of a ## Message Passing Bayesian inference problem on large graphs are often solved using -[message passing](https://en.wikipedia.org/wiki/Belief_propagation). AlgebraicInference.jl -can compose large numbers of Gaussian systems by translating undirected wiring diagrams into -inference problems over a [valuation algebra](https://en.wikipedia.org/wiki/Information_algebra). -These problems can then be solved using generic inference algorithms. +[message passing](https://en.wikipedia.org/wiki/Belief_propagation). +With AlgebraicInference.jl you can compose large numbers of Gaussian systems by translating +undirected wiring diagrams into inference problems over a +[valuation algebra](https://en.wikipedia.org/wiki/Information_algebra). These problems can +be solved using generic inference algorithms like the Shenoy-Shafer architecture. diff --git a/docs/src/index.md b/docs/src/index.md index 1f6b61b..b083993 100644 --- a/docs/src/index.md +++ b/docs/src/index.md @@ -1,3 +1,49 @@ # AlgebraicInference.jl -AlgebraicInference.jl is a library for compositional Bayesian inference. It builds on [Catlab.jl](https://algebraicjulia.github.io/Catlab.jl/dev/). +AlgebraicInference.jl is a library for performing Bayesian inference on wiring diagrams, +building on [Catlab.jl](https://algebraicjulia.github.io/Catlab.jl/dev/). + +## Gaussian Systems + +Gaussian systems were introduced by Jan Willems in his 2013 article *Open Stochastic +Systems*. A probability space $\Sigma = (\mathbb{R}^n, \mathcal{E}, P)$ is called an +$n$-variate Gaussian system with fiber $\mathbb{L} \subseteq \mathbb{R}^n$ if it is +isomorphic to a Gaussian measure on the quotient space $\mathbb{R}^n / \mathbb{L}$. + +If $`\mathbb{L} = \{0\}`$, then $\Sigma$ is an $n$-variate normal distribution. + +Every $n$-variate Gaussian system $\Sigma$ corresponds to a convex *energy function* +$E: \mathbb{R}^n \to (0, \infty]$ of the form +```math + E(x) = \begin{cases} + \frac{1}{2} x^\mathsf{T} P x - x^\mathsf{T} p & Sx = s \\ + \infty & \text{else}, + \end{cases} +``` +where $P$ and $S$ are positive semidefinite matrices, $p \in \mathtt{image}(P)$, and +$s \in \mathtt{image}(S)$. + +If $\Sigma$ is an $n$-variate normal distribution, then $E$ is its negative +log-density. + +## Hypergraph Categories + +There exists a hypergraph PROP whose morphisms $m \to n$ are $m + n$-variate Gaussian +systems. Hence, Gaussian systems can be composed using undirected wiring diagrams. + +![inference](./inference.svg) + +These wiring diagrams look a lot like +[undirected graphical models](https://en.wikipedia.org/wiki/Graphical_model). One difference +is that wiring diagrams can contain half-edges, which specify which variables are +marginalized out during composition. Hence, a wiring diagram can be thought of as an +*inference problem*: a graphical model paired with a query. + +## Message Passing + +Bayesian inference problem on large graphs are often solved using +[message passing](https://en.wikipedia.org/wiki/Belief_propagation). +With AlgebraicInference.jl you can compose large numbers of Gaussian systems by translating +undirected wiring diagrams into inference problems over a +[valuation algebra](https://en.wikipedia.org/wiki/Information_algebra). These problems can +be solved using generic inference algorithms like the Shenoy-Shafer architecture. diff --git a/docs/src/inference.svg b/docs/src/inference.svg new file mode 100644 index 0000000..500681b --- /dev/null +++ b/docs/src/inference.svg @@ -0,0 +1 @@ +Gn1priorn5n1--n5n2likelihoodn2--n5n6n2--n6n3evidencen3--n6n4--n5 \ No newline at end of file diff --git a/src/architectures.jl b/src/architectures.jl index ef5c412..f3ffdf2 100644 --- a/src/architectures.jl +++ b/src/architectures.jl @@ -2,10 +2,7 @@ Architecture{T₁, T₂} <: AbstractNode{T₁} A join tree ``(V, E, \\lambda, D)``, along with a set of factors -```math - \\left\\{ \\phi_i \\right\\}_{i \\in V} -``` -and mailboxes +``\\left\\{ \\phi_i \\right\\}_{i \\in V}`` and mailboxes ```math \\left\\{ \\left( \\mu_{ i \\to \\mathtt{pa}(i)}, \\mu_{\\mathtt{pa}(i) \\to i} \\right) \\right\\}_{i \\in V}. ``` @@ -95,16 +92,9 @@ function parent(node::Architecture) end """ - answer_query(jt::Architecture{T₁, T₂}, query::Set{T₂}) where {T₁, T₂} + answer_query(jt::Architecture, query) Answer a query. - -Let ``(V, E, \\lambda, D)`` be a join tree with factors ``\\{\\phi_i\\}_{i \\in V}`` and -``x`` a query covered by ``(V, E, \\lambda, D)``. Then `answer_query(jt, query)` solves the -inference problem -```math -\\left( \\bigotimes_{i \\in V} \\psi_i \\right)^{\\downarrow x}. -``` """ function answer_query(jt::Architecture, query) for node in PreOrderDFS(jt) @@ -123,16 +113,9 @@ function answer_query(jt::Architecture, query) end """ - answer_query!(jt::Architecture{T₁, T₂}, query::Set{T₂}) where {T₁, T₂} - -Answer a query, caching intermediate computations. + answer_query!(jt::Architecture, query) -Let ``(V, E, \\lambda, D)`` be a join tree with factors ``\\{\\phi_i\\}_{i \\in V}`` and -``x`` be a query covered by ``(V, E, \\lambda, D)``. Then `answer_query!(jt, query)` solves -the inference problem -```math -\\left( \\bigotimes_{i \\in V} \\psi_i \\right)^{\\downarrow x}. -``` +Answer a query, caching intermediate computations in the mailboxes of `jt`. """ function answer_query!(jt::Architecture, query) for node in PreOrderDFS(jt) diff --git a/src/systems.jl b/src/systems.jl index 8808d0a..dca69cc 100644 --- a/src/systems.jl +++ b/src/systems.jl @@ -29,8 +29,9 @@ struct GaussianSystem{ s::AbstractVector, σ) - Construct a Gaussian system by specifying its energy function. You should set - ``\\sigma = s^\\mathsf{T} S^+ s``, where ``S^+`` is the Moore-Penrose + Construct a Gaussian system by specifying its energy function. + + Set ``\\sigma = s^\\mathsf{T} S^+ s``, where ``S^+`` is the Moore-Penrose psuedoinverse of ``S``. """ function GaussianSystem(P::T₁, S::T₂, p::T₃, s::T₄, σ::T₅) where { @@ -198,7 +199,8 @@ end """ *(Σ::GaussianSystem, M::AbstractMatrix) -Construct a Gaussian system by pulling the energy function of `Σ` back along `M`. +Construct a Gaussian system with energy function ``E'(x) = E(Mx),`` where ``E`` is the +energy function of `Σ`. """ function *(Σ::GaussianSystem, M::AbstractMatrix) @assert size(M, 1) == length(Σ) @@ -242,7 +244,7 @@ end """ pushfwd(M::AbstractMatrix, Σ::GaussianSystem) -Compute the pushforward of `Σ` along `M`. +Compute the pushforward ``M_*\\Sigma``. """ function pushfwd(M::AbstractMatrix, Σ::GaussianSystem) @assert size(M, 2) == length(Σ) diff --git a/src/valuations.jl b/src/valuations.jl index f650007..c858dff 100644 --- a/src/valuations.jl +++ b/src/valuations.jl @@ -10,10 +10,6 @@ Subtypes should specialize the following methods: Valuations are parametrized by the type of the variables in their variable system. If `isa(ϕ, Valuation{T})`, then `domain(ϕ)` should return a container with element type `T`. - -References: -- Pouly, M.; Kohlas, J. *Generic Inference. A Unified Theory for Automated Reasoning*; - Wiley: Hoboken, NJ, USA, 2011. """ abstract type Valuation{T} end