Python机器学习（四）：logistic回归

logistic回归

logistic回归虽名为回归但其实做的是分类问题，是一个典型的线性分类器。



如上图中所示：将一组数据特征X输入分类器，它会输出一个预测值y帽（也可以表示为a）。

logistic回归的模型参数为W和b，其中W为一（n，1）矩阵，n为数据特征的维数，也就是X的维数，图中n=3。

由于WTX+b 是一个处于正负无穷间的实数，由于是二分类问题，所以我们的输出（输出定义为预测为1类别的概率）的其实是一个概率要在[0,1]内。所以要引入sigmoid函数，将正负无穷之间的输出转换到[0,1]内。



损失函数L(a，y)用来衡量模型在单个样本上的表现（(i)表示第i个样本）。

L(a(i)，y(i))=−[y(i)loga(i)+(1−y(i))log(1−a(i))]

成本函数J(W,b)用来衡量模型在全体训练集上的表现（m为训练集样本个数）：

J(W,b)=1m∑i=1mL(a(i)，y(i))=−1m∑i=1m[y(i)loga(i)+(1−y(i))log(1−a(i))]

logistic回归中的梯度下降法

在反向传播中：

dLda=1−y1−a−ya

dLdz=dLdadadz=(1−y1−a−ya)(a(1−a))=a−y

当有m个训练样本时设：

da=dLda(i)

dz=(a(i)−y(i))

dw=dJdW=1mXdZT

dZ为dz的矩阵表示（大写字母表示矩阵）,上述dZ和X矩阵相乘是自带求和功能

dZ=A−Y

db=dJdb=1m∑i=1mdz

最终用dw和db配合学习率 α来更新模型参数W和b：

W−=αdw

b−=αdb

代码实现

# -*- coding: utf-8 -*-
"""
Created on Wed Sep 27 20:12:48 2017

@author: YangYuan
"""

import numpy as np

def sigmoid(inX):
return 1.0/(1+np.exp(-inX))

def plotBestFit(data,label,weight):
import matplotlib.pyplot as plt
x1 = []
y1 = []
x0 = []
y0 = []
for index,i in enumerate(label[0]):
if i == 1:
x1.append(data[1,index])
y1.append(data[2,index])
else:
x0.append(data[1,index])
y0.append(data[2,index])
fig = plt.figure()
ax = fig.add_subplot(111)
ax.scatter(x0,y0,s=30,c='red',marker='s')
ax.scatter(x1,y1,s=30,c='green')

x = np.arange(-4.0,4.0,0.1)
y = -(x*weight[1][0]+weight[0][0])/weight[2][0]
ax.plot(x,y)
plt.xlabel('X1')
plt.ylabel('X2')
plt.show()

def main():
data_raw = np.loadtxt('testSet.txt')
#分离数据与标签，并将标签转换为整形
data = data_raw[:,:2].T
label = data_raw[:,2:3].T.astype(int)
m,n = np.shape(data) #m,n = data.shape

temp = np.ones((1,n))
data = np.insert(data,0,values=temp,axis=0)

m,n = np.shape(data)
w = np.random.randn(m,1)*0.01

plotBestFit(data,label,w)

for i in range(1000):
dz = sigmoid(np.dot(w.T,data))-label
dw = np.dot(data,dz.T)/n
alpha = 0.1
w -= alpha*dw

plotBestFit(data,label,w)

if __name__ == '__main__':
main()

testSet.txt中的内容

-0.017612   14.053064   0
-1.395634   4.662541    1
-0.752157   6.538620    0
-1.322371   7.152853    0
0.423363    11.054677   0
0.406704    7.067335    1
0.667394    12.741452   0
-2.460150   6.866805    1
0.569411    9.548755    0
-0.026632   10.427743   0
0.850433    6.920334    1
1.347183    13.175500   0
1.176813    3.167020    1
-1.781871   9.097953    0
-0.566606   5.749003    1
0.931635    1.589505    1
-0.024205   6.151823    1
-0.036453   2.690988    1
-0.196949   0.444165    1
1.014459    5.754399    1
1.985298    3.230619    1
-1.693453   -0.557540   1
-0.576525   11.778922   0
-0.346811   -1.678730   1
-2.124484   2.672471    1
1.217916    9.597015    0
-0.733928   9.098687    0
-3.642001   -1.618087   1
0.315985    3.523953    1
1.416614    9.619232    0
-0.386323   3.989286    1
0.556921    8.294984    1
1.224863    11.587360   0
-1.347803   -2.406051   1
1.196604    4.951851    1
0.275221    9.543647    0
0.470575    9.332488    0
-1.889567   9.542662    0
-1.527893   12.150579   0
-1.185247   11.309318   0
-0.445678   3.297303    1
1.042222    6.105155    1
-0.618787   10.320986   0
1.152083    0.548467    1
0.828534    2.676045    1
-1.237728   10.549033   0
-0.683565   -2.166125   1
0.229456    5.921938    1
-0.959885   11.555336   0
0.492911    10.993324   0
0.184992    8.721488    0
-0.355715   10.325976   0
-0.397822   8.058397    0
0.824839    13.730343   0
1.507278    5.027866    1
0.099671    6.835839    1
-0.344008   10.717485   0
1.785928    7.718645    1
-0.918801   11.560217   0
-0.364009   4.747300    1
-0.841722   4.119083    1
0.490426    1.960539    1
-0.007194   9.075792    0
0.356107    12.447863   0
0.342578    12.281162   0
-0.810823   -1.466018   1
2.530777    6.476801    1
1.296683    11.607559   0
0.475487    12.040035   0
-0.783277   11.009725   0
0.074798    11.023650   0
-1.337472   0.468339    1
-0.102781   13.763651   0
-0.147324   2.874846    1
0.518389    9.887035    0
1.015399    7.571882    0
-1.658086   -0.027255   1
1.319944    2.171228    1
2.056216    5.019981    1
-0.851633   4.375691    1
-1.510047   6.061992    0
-1.076637   -3.181888   1
1.821096    10.283990   0
3.010150    8.401766    1
-1.099458   1.688274    1
-0.834872   -1.733869   1
-0.846637   3.849075    1
1.400102    12.628781   0
1.752842    5.468166    1
0.078557    0.059736    1
0.089392    -0.715300   1
1.825662    12.693808   0
0.197445    9.744638    0
0.126117    0.922311    1
-0.679797   1.220530    1
0.677983    2.556666    1
0.761349    10.693862   0
-2.168791   0.143632    1
1.388610    9.341997    0
0.317029    14.739025   0