Kernel PCA Centered case analysis Let $\mathbf{S}$ be $D\times D$ sample covariance matrix, as discussed at conventional PCA: (assume $\mathbf{X}$ are Centered data set) Recall ...
Factor Analysis
Factor Analysis Factor analysis is a linear-Gaussian latent variable model that is closely related to probabilistic PCA. in probabilistic PCA: \[\begin{aligned} \boldsymbol{x} &= \ma...
Bayesian PCA
Bayesian PCA Given that we have a probabilistic formulation of PCA, it seems natural to seek a Bayesian approach to model selection. But to do this, we need to marginalize out the model parameter...
Bayesian Linear Regression (Single Output)
Bayesian Linear Regression (Single Output) The Linear Model of $f(\boldsymbol{x}, \boldsymbol{w}) = \boldsymbol{w}^T\phi(\boldsymbol{x})$, with Gaussian noise $p(\epsilon) = \mathcal{N}(\e...
Bayesian Linear Regression (Multiple Outputs)
Bayesian Linear Regression (Multiple Output) The Linear Model of $f(\boldsymbol{x}, W) = W\phi(\boldsymbol{x})$, with Gaussian noise $p(\boldsymbol{\epsilon}) = \mathcal{N}(\boldsymbol{\ep...
Bayesian Inference for Gaussian
Univariate Unknown Mean, Known Variance/Precision Gaussian Known Mean, Unknown Variance Gamma known Mean, Unknown Precision Inverse-Gamma Unknown Mean, Unknown Precision Gauss...
Gaussian-Gamma Distribution
Gaussian-Gamma distribution [\begin{aligned} \mathrm{GausGam}(\mu, \lambda \vert \mu_0, \tau , a, b) &= \mathcal{N}(\mu \vert \mu_0, (\tau\lambda)^{-1}) \mathrm{Gam}(\lambda \vert a,b) \ &am...
Gaussian Distribution (3)
Conditional Gaussian Distribution we partition $\boldsymbol{x}$ into disjoint subsets $\boldsymbol{x}_a, \boldsymbol{x}_b$ \(\begin{aligned} \boldsymbol{x} &= \binom{\boldsymbol{x...
Gaussian Distribution (2)
Maximum Likelihood for the Gaussian Given a data set $\mathbf{X} = {\mathbf{x}_1,\mathbf{x}_2,\cdots, \mathbf{x}_N}$ which are drawn independently from a multivariate Gaussian distribution...
Gaussian Distribution (1)
Gaussian Distribution Univariate [\begin{aligned} \mathcal{N}(x\vert \mu, \sigma^2) = \frac {1}{\sqrt{2\pi \sigma^2}} \exp\left(- \frac {1}{2\sigma^2} (x-\mu)^2 \right) \end{aligned}] Th...