Support vector machine is a method for classification and regression that draws an optimal boundary in the space of covariates (p dimension) when the samples \((x_1, y_1), \ldots , (x_N, y_N)\) are given. This is a method to maximize the minimum value over \(i = 1, \ldots , N\) of the distance between \(x_i\) and the boundary. This notion is generalized even if the samples are not separated by a surface by softening the notion of a margin. Additionally, by using a general kernel that is not the inner product, even if the boundary is not a surface, we can mathematically formulate the problem and obtain the optimum solution. In this chapter, we consider only the two-class case and focus on the core part. Although omitted here, the theory of support vector machine also applies to regression and classification with more than two classes.