Regression analysis
2016-03-04 16:20
274 查看
Source: http://wenku.baidu.com/link?url=9KrZhWmkIDHrqNHiXCGfkJVQWGFKOzaeiB7SslSdW_JnXCkVHsHsXJyvGbDva4V5A-uuOl84mg5zkTECichHX_AsN0mZalfI9BzDFOeNe-G###
❤ Simple linear regression
1. Y = β0 + β1*X + e
where:
Y - dependent variable (response)
X - independent variable (predictor/explanatory)
β0 - intercept
β1 - slope of the regression line
e - random error
2. Y' = b0 + b1*X
where: Y' - predicted value of Y
e = Y - Y'
3. Least squarea regression minizes the sum of the square of the errors and can be used to estimate b0 and b1.
4. Measuring the fit of the estimated model.
- The varibility of Y
SST (Sum of Squared Total): total variability about the mean, SST = sum((Y - mean(Y))^2);
SSE (Sum of Squared Error): variability about the regression line, SSE = sum(e^2) = sum((Y - mean(Y'))^2), SSE is unexplained varibility;
SSR (Sum of Squares due to Regression): variability that is explained, SSR = sum((Y' - mean(Y))^2), SSR is explained varibility.
Note that SST = SSE + SSR.
- Coefficient of determination
r^2: proportion of explained variability by the regression equation.
0 <= r^2 = 1 - SSE/SST = SSR/SST <= 1
- Correlation coefficient
r: strength of the relationship between X and Y.
-1 <= r <= 1
5. Assumptions in the regression model
Errors are independent, normally distributed, with the mean of zero, with a constant variance.
The assumptions can be tested by using residual analysis.
6. MSE (Mean Squared Error)
Estimation of error variance of the regression equation.
s^2 = MSE = SSE / (n - k - 1)
where:
n - number of observations in the sample
k - number of independent variables
Standard deviation of the regression: s = sqrt(MSE) is also frequently used.
❤ Test the model for significance: F-test
Used to statistically test the null hypothesis H0: there is no linear relationship between Y and X (i.e. β1 = 0).
If p value is low, then we regect H0 and conclude there is linear relationship:
F = MSR / MSE
where: MSR = SSR / k
Good regression model should have significant F value and high r^2 value.
Statistical test can be performed on the regression coefficients. H0: the βs are 0.
For a simple linear regression, the test for regression coefficient gives the same information as the ones given by F-test.
❤ ANOVA tables
The general form of the ANOVA table is helpful for understanding the interrelatedness of error terms.
❤ Multiple regression
Similar to the simple regression model, but there are more than one X in the multiple regression models.
Y' = b0 + b1*X1 + b2*X2 + ... + bn*Xn
Note that if indenpendent variables is correlate to each other, colinearity or multicolinearity will happen. This will cause problems when intepreate variables individually although the overall model estimation may still be good.
❤ Simple linear regression
1. Y = β0 + β1*X + e
where:
Y - dependent variable (response)
X - independent variable (predictor/explanatory)
β0 - intercept
β1 - slope of the regression line
e - random error
2. Y' = b0 + b1*X
where: Y' - predicted value of Y
e = Y - Y'
3. Least squarea regression minizes the sum of the square of the errors and can be used to estimate b0 and b1.
4. Measuring the fit of the estimated model.
- The varibility of Y
SST (Sum of Squared Total): total variability about the mean, SST = sum((Y - mean(Y))^2);
SSE (Sum of Squared Error): variability about the regression line, SSE = sum(e^2) = sum((Y - mean(Y'))^2), SSE is unexplained varibility;
SSR (Sum of Squares due to Regression): variability that is explained, SSR = sum((Y' - mean(Y))^2), SSR is explained varibility.
Note that SST = SSE + SSR.
- Coefficient of determination
r^2: proportion of explained variability by the regression equation.
0 <= r^2 = 1 - SSE/SST = SSR/SST <= 1
- Correlation coefficient
r: strength of the relationship between X and Y.
-1 <= r <= 1
5. Assumptions in the regression model
Errors are independent, normally distributed, with the mean of zero, with a constant variance.
The assumptions can be tested by using residual analysis.
6. MSE (Mean Squared Error)
Estimation of error variance of the regression equation.
s^2 = MSE = SSE / (n - k - 1)
where:
n - number of observations in the sample
k - number of independent variables
Standard deviation of the regression: s = sqrt(MSE) is also frequently used.
❤ Test the model for significance: F-test
Used to statistically test the null hypothesis H0: there is no linear relationship between Y and X (i.e. β1 = 0).
If p value is low, then we regect H0 and conclude there is linear relationship:
F = MSR / MSE
where: MSR = SSR / k
Good regression model should have significant F value and high r^2 value.
Statistical test can be performed on the regression coefficients. H0: the βs are 0.
For a simple linear regression, the test for regression coefficient gives the same information as the ones given by F-test.
❤ ANOVA tables
The general form of the ANOVA table is helpful for understanding the interrelatedness of error terms.
❤ Multiple regression
Similar to the simple regression model, but there are more than one X in the multiple regression models.
Y' = b0 + b1*X1 + b2*X2 + ... + bn*Xn
Note that if indenpendent variables is correlate to each other, colinearity or multicolinearity will happen. This will cause problems when intepreate variables individually although the overall model estimation may still be good.
相关文章推荐
- CSS---子div在父div中水平垂直居中
- ViewState 视图状态对象实例
- map转实体类(前提是实体类里的字段都是string)
- liunx 安装lnmp 环境配置多站点
- 第24课:Spark Shuffle内幕彻底解密
- 关于TextView的一些个人总结之二
- 2016 使用seajs时如何模块化jquery
- 重定向redirect与跳转forward区别
- 2016.03.04(struts前台取值)
- java调用百度定位api服务获取地理位置示例
- scala类的定义
- onConfigurationChanged,oncreate 等
- 简洁代码之道:用多态替代条件语句及避免全局可变状态
- C语言实现双向非循环链表(带头结点尾结点)的节点插入
- 将100~200之间的不能被3整除的数输出
- IEnumerable,ICollection,IList,List区别
- 学习&使用-libcurl
- 闭包--面试之必问
- HttpClient 发送post和get请求 调用第三方webservice
- Android客户端性能优化(魅族资深工程师毫无保留奉献)