NeuralEstimators.DeepSetExpert — TypeDeepSetExpert(ψ, ϕ, S, a)
+θ̂((Z₃, x₂))NeuralEstimators.DeepSetExpert — TypeDeepSetExpert(ψ, ϕ, S, a)
+θ̂((Z₃, x₂))NeuralEstimators.DeepSetExpert — TypeDeepSetExpert(ψ, ϕ, S, a)
DeepSetExpert(ψ, ϕ, S; a::String = "mean")
DeepSetExpert(deepset::DeepSet, ϕ, S)Identical to DeepSet, but with additional expert summary statistics,
\[θ̂(𝐙) = ϕ((𝐓(𝐙)', 𝐒(𝐙)')'), 𝐓(𝐙) = 𝐚(\{ψ(𝐙ᵢ) : i = 1, …, m\}),\]
where S is a function that returns a vector of expert summary statistics.
The constructor DeepSetExpert(deepset::DeepSet, ϕ, S) inherits ψ and a from deepset.
Similarly to DeepSet, set-level information can be incorporated by passing a Tuple, in which case we have
\[θ̂(𝐙) = ϕ((𝐓(𝐙)', 𝐒(𝐙)', 𝐱')'), 𝐓(𝐙) = 𝐚(\{ψ(𝐙ᵢ) : i = 1, …, m\}).\]
Examples
using NeuralEstimators
using Flux
@@ -58,7 +58,7 @@
x₁ = rand(qₓ)
x₂ = [rand(qₓ) for _ ∈ eachindex(Z₂)]
θ̂((Z₁, x₁))
-θ̂((Z₂, x₂))NeuralEstimators.GNN — TypeGNN(propagation, readout, ϕ, a)
+θ̂((Z₂, x₂))NeuralEstimators.GNN — TypeGNN(propagation, readout, ϕ, a)
GNN(propagation, readout, ϕ, a::String = "mean")A graph neural network (GNN) designed for parameter point estimation.
The propagation module transforms graphical input data into a set of hidden-feature graphs; the readout module aggregates these feature graphs into a single hidden feature vector of fixed length; the function a(⋅) is a permutation-invariant aggregation function, and ϕ is a neural network.
The data should be stored as a GNNGraph or AbstractVector{GNNGraph}, where each graph is associated with a single parameter vector. The graphs may contain sub-graphs corresponding to independent replicates from the model.
Examples
using NeuralEstimators
using Flux
using Flux: batch
@@ -95,7 +95,7 @@
g₃ = batch([g₁, g₂])
θ̂(g₁)
θ̂(g₃)
-θ̂([g₁, g₂, g₃])NeuralEstimators.WeightedGraphConv — TypeWeightedGraphConv(in => out, σ=identity; aggr=mean, bias=true, init=glorot_uniform)Same as regular GraphConv layer, but where the neighbours of a node are weighted by their spatial distance to that node.
Arguments
in: The dimension of input features.out: The dimension of output features.σ: Activation function.aggr: Aggregation operator for the incoming messages (e.g. +, *, max, min, and mean).bias: Add learnable bias.init: Weights' initializer.Examples
using NeuralEstimators
+θ̂([g₁, g₂, g₃])NeuralEstimators.WeightedGraphConv — TypeWeightedGraphConv(in => out, σ=identity; aggr=mean, bias=true, init=glorot_uniform)Same as regular GraphConv layer, but where the neighbours of a node are weighted by their spatial distance to that node.
Arguments
in: The dimension of input features.out: The dimension of output features.σ: Activation function.aggr: Aggregation operator for the incoming messages (e.g. +, *, max, min, and mean).bias: Add learnable bias.init: Weights' initializer.Examples
using NeuralEstimators
using GraphNeuralNetworks
# Construct a spatially-weighted adjacency matrix based on k-nearest neighbours
@@ -108,7 +108,7 @@
# Construct the layer and apply it to the data to generate convolved features
layer = WeightedGraphConv(d => 16)
-layer(Z)NeuralEstimators.UniversalPool — TypeUniversalPool(ψ, ϕ)Pooling layer (i.e., readout layer) from the paper 'Universal Readout for Graph Convolutional Neural Networks'. It takes the form,
\[\mathbf{V} = ϕ(|G|⁻¹ \sum_{s\in G} ψ(\mathbf{h}_s)),\]
where $\mathbf{V}$ denotes the summary vector for graph $G$, $\mathbf{h}_s$ denotes the vector of hidden features for node $s \in G$, and ψ and ϕ are dense neural networks.
See also the pooling layers available from GraphNeuralNetworks.jl.
Examples
using NeuralEstimators
+layer(Z)NeuralEstimators.UniversalPool — TypeUniversalPool(ψ, ϕ)Pooling layer (i.e., readout layer) from the paper 'Universal Readout for Graph Convolutional Neural Networks'. It takes the form,
\[\mathbf{V} = ϕ(|G|⁻¹ \sum_{s\in G} ψ(\mathbf{h}_s)),\]
where $\mathbf{V}$ denotes the summary vector for graph $G$, $\mathbf{h}_s$ denotes the vector of hidden features for node $s \in G$, and ψ and ϕ are dense neural networks.
See also the pooling layers available from GraphNeuralNetworks.jl.
Examples
using NeuralEstimators
using Flux
using GraphNeuralNetworks
using Graphs: random_regular_graph
@@ -127,7 +127,7 @@
pool = UniversalPool(ψ, ϕ)
# Apply the pooling layer
-pool(G)These layers can be used at the end of an architecture to ensure that the neural estimator provides valid parameters.
NeuralEstimators.Compress — TypeCompress(a, b, k = 1)Layer that compresses its input to be within the range a and b, where each element of a is less than the corresponding element of b.
The layer uses a logistic function,
\[l(θ) = a + \frac{b - a}{1 + e^{-kθ}},\]
where the arguments a and b together combine to shift and scale the logistic function to the range (a, b), and the growth rate k controls the steepness of the curve.
The logistic function given here contains an additional parameter, θ₀, which is the input value corresponding to the functions midpoint. In Compress, we fix θ₀ = 0, since the output of a randomly initialised neural network is typically around zero.
Examples
using NeuralEstimators
+pool(G)These layers can be used at the end of an architecture to ensure that the neural estimator provides valid parameters.
NeuralEstimators.Compress — TypeCompress(a, b, k = 1)Layer that compresses its input to be within the range a and b, where each element of a is less than the corresponding element of b.
The layer uses a logistic function,
\[l(θ) = a + \frac{b - a}{1 + e^{-kθ}},\]
where the arguments a and b together combine to shift and scale the logistic function to the range (a, b), and the growth rate k controls the steepness of the curve.
The logistic function given here contains an additional parameter, θ₀, which is the input value corresponding to the functions midpoint. In Compress, we fix θ₀ = 0, since the output of a randomly initialised neural network is typically around zero.
Examples
using NeuralEstimators
using Flux
a = [25, 0.5, -pi/2]
@@ -141,7 +141,7 @@
n = 20
θ̂ = Chain(Dense(n, p), l)
Z = randn(n, K)
-θ̂(Z)NeuralEstimators.CholeskyCovariance — TypeCholeskyCovariance(d)Layer for constructing the parameters of the lower Cholesky factor associated with an unconstrained d×d covariance matrix.
The layer transforms a Matrix with d(d+1)÷2 rows into a Matrix of the same dimension, but with d rows constrained to be positive (corresponding to the diagonal elements of the Cholesky factor) and the remaining rows unconstrained.
The ordering of the transformed Matrix aligns with Julia's column-major ordering. For example, when modelling the Cholesky factor,
\[\begin{bmatrix} +θ̂(Z)
NeuralEstimators.CholeskyCovariance — TypeCholeskyCovariance(d)Layer for constructing the parameters of the lower Cholesky factor associated with an unconstrained d×d covariance matrix.
The layer transforms a Matrix with d(d+1)÷2 rows into a Matrix of the same dimension, but with d rows constrained to be positive (corresponding to the diagonal elements of the Cholesky factor) and the remaining rows unconstrained.
The ordering of the transformed Matrix aligns with Julia's column-major ordering. For example, when modelling the Cholesky factor,
\[\begin{bmatrix} L₁₁ & & \\ L₂₁ & L₂₂ & \\ L₃₁ & L₃₂ & L₃₃ \\ @@ -152,7 +152,7 @@ θ = randn(p, 50) l = CholeskyCovariance(d) θ = l(θ) # returns matrix (used for Flux networks) -L = [vectotril(y) for y ∈ eachcol(θ)] # convert matrix to Cholesky factors
NeuralEstimators.CovarianceMatrix — TypeCovarianceMatrix(d)Layer for constructing the parameters of an unconstrained d×d covariance matrix.
The layer transforms a Matrix with d(d+1)÷2 rows into a Matrix of the same dimension.
Internally, it uses a CholeskyCovariance layer to construct a valid Cholesky factor 𝐋, and then extracts the lower triangle from the positive-definite covariance matrix 𝚺 = 𝐋𝐋'. The lower triangle is extracted and vectorised in line with Julia's column-major ordering. For example, when modelling the covariance matrix,
\[\begin{bmatrix} +L = [vectotril(y) for y ∈ eachcol(θ)] # convert matrix to Cholesky factors
NeuralEstimators.CovarianceMatrix — TypeCovarianceMatrix(d)Layer for constructing the parameters of an unconstrained d×d covariance matrix.
The layer transforms a Matrix with d(d+1)÷2 rows into a Matrix of the same dimension.
Internally, it uses a CholeskyCovariance layer to construct a valid Cholesky factor 𝐋, and then extracts the lower triangle from the positive-definite covariance matrix 𝚺 = 𝐋𝐋'. The lower triangle is extracted and vectorised in line with Julia's column-major ordering. For example, when modelling the covariance matrix,
\[\begin{bmatrix} Σ₁₁ & Σ₁₂ & Σ₁₃ \\ Σ₂₁ & Σ₂₂ & Σ₂₃ \\ Σ₃₁ & Σ₃₂ & Σ₃₃ \\ @@ -165,7 +165,7 @@ l = CovarianceMatrix(d) θ = l(θ) -Σ = [Symmetric(cpu(vectotril(y)), :L) for y ∈ eachcol(θ)]
NeuralEstimators.CorrelationMatrix — TypeCorrelationMatrix(d)Layer for constructing the parameters of an unconstrained d×d correlation matrix.
The layer transforms a Matrix with d(d-1)÷2 rows into a Matrix with the same dimension.
Internally, the layers uses the algorithm described here and here to construct a valid Cholesky factor 𝐋, and then extracts the strict lower triangle from the positive-definite correlation matrix 𝐑 = 𝐋𝐋'. The strict lower triangle is extracted and vectorised in line with Julia's column-major ordering. For example, when modelling the correlation matrix,
\[\begin{bmatrix} +Σ = [Symmetric(cpu(vectotril(y)), :L) for y ∈ eachcol(θ)]
NeuralEstimators.CorrelationMatrix — TypeCorrelationMatrix(d)Layer for constructing the parameters of an unconstrained d×d correlation matrix.
The layer transforms a Matrix with d(d-1)÷2 rows into a Matrix with the same dimension.
Internally, the layers uses the algorithm described here and here to construct a valid Cholesky factor 𝐋, and then extracts the strict lower triangle from the positive-definite correlation matrix 𝐑 = 𝐋𝐋'. The strict lower triangle is extracted and vectorised in line with Julia's column-major ordering. For example, when modelling the correlation matrix,
\[\begin{bmatrix} 1 & R₁₂ & R₁₃ \\ R₂₁ & 1 & R₂₃\\ R₃₁ & R₃₂ & 1\\ @@ -185,7 +185,7 @@ R = Symmetric(cpu(vectotril(y, strict = true)), :L) R[diagind(R)] .= 1 R -end
NeuralEstimators.SplitApply — TypeSplitApply(layers, indices)Splits an array into multiple sub-arrays by subsetting the rows using the collection of indices, and then applies each layer in layers to the corresponding sub-array.
Specifically, for each i = 1, …, $n$, with $n$ the number of layers, SplitApply(x) performs layers[i](x[indices[i], :]), and then vertically concatenates the resulting transformed arrays.
Examples
using NeuralEstimators
+endNeuralEstimators.SplitApply — TypeSplitApply(layers, indices)Splits an array into multiple sub-arrays by subsetting the rows using the collection of indices, and then applies each layer in layers to the corresponding sub-array.
Specifically, for each i = 1, …, $n$, with $n$ the number of layers, SplitApply(x) performs layers[i](x[indices[i], :]), and then vertically concatenates the resulting transformed arrays.
Examples
using NeuralEstimators
d = 4
K = 50
@@ -200,4 +200,4 @@
l = SplitApply([l₁, l₂], [1:p₁, p₁+1:p])
θ = randn(p, K)
-l(θ)Settings
This document was generated with Documenter.jl version 1.1.2 on Saturday 4 November 2023. Using Julia version 1.7.3.