Bayes classifier

Bayes classifier

Bayes classifier

For classification, we use 0-1 loss function to choosing classifier, which can be denoted as L(y,y^)

If k-class is considered, our epxect prediction error is

EPE=E[L(G,G^(X))]=EXEY|X[L(G,G^(X))|X]

by minimizing pointwise,

G^(x)=argmin∑g=1kL(g,G^(X))p(g|X=x)=argming∈G[1−p(g|X=x)]=argmaxg∈Gp(g|X=x)

So Bayes classifier chooses the class with max posterior probability.