Dual Representations of Linear Models Many linear models for regression and classification can be reformulated in terms of a dual representation in which the kernel function arises natural...
SVM for Regression
SVM for Regression Using $\epsilon$-insensitive error function $\epsilon$-insensitive can lead to sparse solutions [\begin{aligned} \min &\qquad C\sum_{n=1}^N (\xi_n + \widehat{\xi}_n) + \fr...
SVM for classification
SVM for classification Linearly Separable case The Largest Margin Principle In the two-classclassification problem using linear models of the form [\begin{aligned} y(\boldsymbol{x}) = \boldsymb...
Probabilistic Generative Model
Probabilistic Generative Model Model the class-conditional densities $p(x\vert C_k )$, as well as the class priors $p(C_k)$ The posterior probabilities: can obtained through Baye...
Probabilistic Discriminative Models
Probabilistic Discriminative Models Explicitly to use the functional form of the generalized linear model and to determine its parameters by using ML And in this direct approach, we ...
Discriminant Analysis
Discriminant Analysis Least Square for classification Each class has a linear model: \[\begin{aligned} y_k = \boldsymbol{w}_k^T\boldsymbol{x}_k + w_0 \end{aligned}\] ...
Graphical Models
Graphical Models Directed Graphical Model Also known as Bayesian Networks. Directed Acyclic Graphs (DAG): The directed graphs that we are considering are subject to an important restriction name...
Proximal Gradient Method
Proximal Gradient Method Proximal opeartor Proximal Opeartor \[{\displaystyle \operatorname {prox}_{f}(v)=\arg \min _{x\in {\mathcal {X}}}\left(f(x)+{\frac {1}{2}}\|x-v\|_{2}^{2}\right)....
Probabilistic PCA
Probabilistic PCA We now show that PCA can also be expressed as the maximum likelihood solution of a probabilistic latent variable model. Note: the probabilistic PCA model can be expressed as a d...
Principal Component Analysis
Principal Component Analysis Maximum Variance formulation PCA can be defined as the Orthogonal projection of the data into a lower dimensional linear space, known as the principal subspace, such ...