libsvm 多分类中model参数

一、先看REDEME中的介绍

struct svm_model

{

struct svm_parameter param;/* parameter */

int nr_class;
/* number of classes, = 2 in regression/one class svm */

int l;
/* total #SV */

struct svm_node **SV;/* SVs (SV[l]) */

double **sv_coef;/* coefficients for SVs in decision functions (sv_coef[k-1][l]) */

double *rho;
/* constants in decision functions (rho[k*(k-1)/2]) */

double *probA;/* pairwise probability information */

double *probB;

int *sv_indices; /* sv_indices[0,...,nSV-1] are values in [1,...,num_traning_data] to indicate SVs in the training set */

/* for classification only */

int *label;
/* label of each class (label[k]) */

int *nSV;
/* number of SVs for each class (nSV[k]) */

/* nSV[0] + nSV[1] + ... + nSV[k-1] = l */

/* XXX */

int free_sv;
/* 1 if svm_model is created by svm_load_model*/

/* 0 if svm_model is created by svm_train */

};

param describes the parameters used to obtain the model.

nr_class is the number of classes. It is 2 for regression and one-class SVM.

l is the number of support vectors. SV and sv_coef are support

vectors and the corresponding coefficients, respectively. Assume there are

k classes. For data in class j, the corresponding sv_coef includes (k-1) y*alpha vectors,

where alpha's are solutions of the following two class problems:

1 vs j, 2 vs j, ..., j-1 vs j, j vs j+1, j vs j+2, ..., j vs k

and y=1 for the first j-1 vectors, while y=-1 for the remaining k-j

vectors. For example, if there are 4 classes, sv_coef and SV are like:

+-+-+-+--------------------+

|1|1|1| |

|v|v|v| SVs from class 1 |

|2|3|4| |

+-+-+-+--------------------+

|1|2|2| |

|v|v|v| SVs from class 2 |

|2|3|4| |

+-+-+-+--------------------+

|1|2|3| |

|v|v|v| SVs from class 3 |

|3|3|4| |

+-+-+-+--------------------+

|1|2|3| |

|v|v|v| SVs from class 4 |

|4|4|4| |

+-+-+-+--------------------+

rho is the bias term (-b). probA and probB are parameters used in

probability outputs. If there are k classes, there are k*(k-1)/2

binary problems as well as rho, probA, and probB values. They are

aligned in the order of binary problems:

1 vs 2, 1 vs 3, ..., 1 vs k, 2 vs 3, ..., 2 vs k, ..., k-1 vs k.

sv_indices[0,...,nSV-1] are values in [1,...,num_traning_data] to

indicate support vectors in the training set.

label contains labels in the training data.

nSV is the number of support vectors in each class.

二、基本概念

什么是支持向量？ 存在y = +1 的支持向量 y=-1的支持向量

svm对多分类支持？用1vs1加上投票的方法来确定

三、二分类和多分类model参数

对于二分类

SV用于保持支持向量

sv_coef用于保存支持向量对应的系数yi*alpha

因为C-SVC 的决策函数如下：可以看到wi是标量，Xi是 支持向量



因此 SV 和 SV_coef的行数是相同的，假设为M，则SV 为M *（fetures个数） SV_coef 为 M * 1，注意这里的支持向量分正支持向量和 负支持向量，由nSV指出个数，这个概念在推广到多分类是很有用。

+-+-+-+--------------------+

|1| |

|v| SVs from class 1 |

|2| |

+-+-+-+--------------------+

对于多分类

SV用于保持支持向量

sv_coef用于保存支持向量对应的系数yi*alpha

因为C-SVC 的决策函数如下：可以看到wi是标量，Xi是 支持向量



k -1 fetures 个数

+-+-+-+ -------------------+

|1|1|1| |

|v|v|v| SVs from class 1 | 假设这里有 nSV[0]个支持向量

|2|3|4| |

| | | | | |

+-+-+ |+ --------------------+

|1|2|2| |

|v|v|v| S Vs from class 2 | nSV[1]个支持向量

|2|3|4| |

+-+-+-+-- ------------------+

|1|2|3| |

|v|v|v| SVs from class 3 | nSV[2]个支持向量

|3|3|4| |

| | | | | | | | | | | | | | | | | | | | | | | |

| | | | | | | | | | | | | | | | | | | | | |

+-+-+-+--- -----------------+

|1|2|3| |

|v|v|v| SVs from class 4 | nSV[3]个支持向量

|4|4|4| |

+-+-+-+---- ----------------+

For data in class j, the corresponding sv_coef includes (k-1) y*alpha vectors,

where alpha's are solutions of the following two class problems:

1 vs j, 2 vs j, ..., j-1 vs j, j vs j+1, j vs j+2, ..., j vs k

and y=1 for the first j-1 vectors, while y=-1 for the remaining k-j

vectors.

因此 SV 和 SV_coef的行数是相同的，注意由于使用1VS1 策略，需要分别计算

1 vs j, 2 vs j, ..., j-1 vs j, j vs j+1, j vs j+2, ..., j vs k 这么多分类器的支持向量和系数。

由于有k个分类器，因此对每个类需要（k-1）个 （y*alpha vectors）。

在这个k-1个系数向量中，有j-1个 是i vs j and i < j 的 ，此时，i中有正支持向量，有 k-j 个是 i vs j and i>j 的，此时i中有负支持向量。因此假如要计算 i vs j 分类器的w 系数，需要 class i中的 i vs j 系数向量，class i中的 支持向量 ，也需要
class j中的 j vs i系数向量，class j中的支持向量 ，注意这两个向量是互补的，一个在分类线左边，一个在右边，也就是上面说的正支持向量和负的支持向量。只要组合这些向量就能恢复出二分类中的中的系数向量和支持向量。

Q：libsvm 文档摘要

Q:How could i generate the primal variable w oflinear svm?

Let's start from the binary class and assume you have two labels -1 and +1. After obtaining the model from calling svmtrain, do the following to have w and b:

w = model.SVs' * model.sv_coef;
b = -model.rho;

if model.Label(1) == -1
w = -w;
b = -b;
end

If you do regression or one-class SVM, then the if statement is not needed.

For multi-class SVM, we illustrate the setting in the following example of running the iris data, which have 3 classes

> [y, x] = libsvmread('../../htdocs/libsvmtools/datasets/multiclass/iris.scale');
> m = svmtrain(y, x, '-t 0')

m =

Parameters: [5x1 double]
nr_class: 3
totalSV: 42
rho: [3x1 double]
Label: [3x1 double]
ProbA: []
ProbB: []
nSV: [3x1 double]
sv_coef: [42x2 double]
SVs: [42x4 double]

sv_coef is like:

+-+-+--------------------+
|1|1|                    |
|v|v|  SVs from class 1  |
|2|3|                    |
+-+-+--------------------+
|1|2|                    |
|v|v|  SVs from class 2  |
|2|3|                    |
+-+-+--------------------+
|1|2|                    |
|v|v|  SVs from class 3  |
|3|3|                    |
+-+-+--------------------+

so we need to see nSV of each classes.

> m.nSV

ans =

3
21
18

Suppose the goal is to find the vector w of classes 1 vs 3. Then y_i alpha_i of training 1 vs 3 are

> coef = [m.sv_coef(1:3,2); m.sv_coef(25:42,1)];

and SVs are:

> SVs = [m.SVs(1:3,:); m.SVs(25:42,:)];

Hence, w is

> w = SVs'*coef;

For rho,

> m.rho

ans =

1.1465
0.3682
-1.9969
> b = -m.rho(2);

because rho is arranged by 1vs2 1vs3 2vs3.