Computing the Support Vector Classifier
- The previous problem is quadratic with linear inequality constraints
- A convex optimization problem
- Quadratic programming solution using Lagrange multipliers
- We re-express the problem as:
- \(\min_{\beta,\beta_0} \frac{1}{2}\|\beta\|^2+C\sum\limits_{i=1}^{N}\xi_i\)
- subject to \(\xi_i \geq 0\), \(y_i(x_i^T\beta+\beta_0) \geq 1 - \xi_i\) \(\forall\) \(i\)
- We change the “constant” for the “cost” parameter \(C\)
- The separable case corresponds to \(C = \infty\)
- The Lagrange function is:
\(L_P=\frac{1}{2}\|\beta\|^2+C\sum\limits_{i=0}^{N}\xi_i\) \(-\sum\limits_{i=0}^{N} \alpha_i[y_i(x_i^T\beta+\beta_0)-(1-\xi_i)]\) \(-\sum\limits_{i=1}{N}\mu_i\xi_i\)
- We minimize w.r.t. \(\beta\), \(\beta_0\), and \(\xi_i\)