逻辑回归模型的代价函数对参数的偏导数--推导过程

https://math.stackexchange.com/questions/477207/derivative-of-cost-function-for-logistic-regression/477261#477261 https://www.cnblogs.com/Belter/p/6653773.html

up vote down voteaccepted

The reason is the following. We use the notation θxi:=θ0+θ1xi1+⋯+θpxip.θxi:=θ0+θ1x1i+⋯+θpxpi. Then loghθ(xi)=log11+e−θxi=−log(1+e−θxi),log⁡hθ(xi)=log⁡11+e−θxi=−log⁡(1+e−θxi), log(1−hθ(xi))=log(1−11+e−θxi)=log(e−θxi)−log(1+e−θxi)=−θxi−log(1+e−θxi),log⁡(1−hθ(xi))=log⁡(1−11+e−θxi)=log⁡(e−θxi)−log⁡(1+e−θxi)=−θxi−log⁡(1+e−θxi), [ this used: 1=(1+e−θxi)(1+e−θxi),1=(1+e−θxi)(1+e−θxi), the 1's in numerator cancel, then we used: log(x/y)=log(x)−log(y)log⁡(x/y)=log⁡(x)−log⁡(y) ] Since our original cost function is the form of: J(θ)=−1m∑i=1myilog(hθ(xi))+(1−yi)log(1−hθ(xi))J(θ)=−1m∑i=1myilog⁡(hθ(xi))+(1−yi)log⁡(1−hθ(xi)) Plugging in the two simplified expressions above, we obtain J(θ)=−1m∑i=1m[−yi(log(1+e−θxi))+(1−yi)(−θxi−log(1+e−θxi))]J(θ)=−1m∑i=1m[−yi(log⁡(1+e−θxi))+(1−yi)(−θxi−log⁡(1+e−θxi))] , which can be simplified to: J(θ)=−1m∑i=1m[yiθxi−θxi−log(1+e−θxi)]=−1m∑i=1m[yiθxi−log(1+eθxi)], (∗)J(θ)=−1m∑i=1m[yiθxi−θxi−log⁡(1+e−θxi)]=−1m∑i=1m[yiθxi−log⁡(1+eθxi)], (∗) where the second equality follows from −θxi−log(1+e−θxi)=−[logeθxi+log(1+e−θxi)]=−log(1+eθxi).−θxi−log⁡(1+e−θxi)=−[log⁡eθxi+log⁡(1+e−θxi)]=−log⁡(1+eθxi). [ we used log(x)+log(y)=log(xy)log⁡(x)+log⁡(y)=log(xy) ] All you need now is to compute the partial derivatives of (∗)(∗) w.r.t. θjθj. As ∂∂θjyiθxi=yixij,∂∂θjyiθxi=yixji, ∂∂θjlog(1+eθxi)=xijeθxi1+eθxi=xijhθ(xi),∂∂θjlog⁡(1+eθxi)=xjieθxi1+eθxi=xjihθ(xi), the thesis follows.