Loadings vs eigenvectors in PCA 主成分分析(PCA)中的负荷和特征向量
2017-03-17 11:20
1191 查看
Source: Cross Validated
![](https://oscdn.geek-share.com/Uploads/Images/Content/202005/08/924031063ebb25a0fb125f6794d38ab6.jpg)
ttnphns
In PCA, you split covariance (or correlation) matrix into scale part (eigenvalues) and direction part (eigenvectors). You may then endow eigenvectors with the scale: loadings. So, loadings are thus become comparable by magnitude with the covariances/correlations observed between the variables, - because what had been drawn out from the variables' covariation now returns back - in the form of the covariation between the variables and the principal components. Actually, loadings are the covariances/correlations between the original variables and the unit-scaled components. This answer shows geometrically what loadings are and what are coefficients associating components with variables in PCA or factor analysis.
Loadings:
Help you interpret principal components or factors; Because they are the linear combination weights (coefficients) whereby unit-scaled components or factors define or "load" a variable.
(Eigenvector is just a coefficient of orthogonal transformation or projection, it is devoid of "load" within its value. "Load" is (information of the amount of) variance, magnitude. PCs are extracted to explain variance of the variables. Eigenvalues are the variances of (= explained by) PCs. When we multiply eigenvector by sq.root of the eivenvalue we "load" the bare coefficient by the amount of variance. By that virtue we make the coefficient to be the measure of association, co-variability.)
Loadings sometimes are "rotated" (e.g. varimax) afterwards to facilitate interpretability (see also);
It is loadings which "restore" the original covariance/correlation matrix (see also this thread discussing nuances of PCA and FA in that respect);
While in PCA you can compute values of components both from eigenvectors and loadings, in factor analysis you compute factor scores out of loadings.
And, above all, loading matrix is informative: its vertical sums of squares are the eigenvalues, components' variances, and its horizontal sums of squares are portions of the variables' variances being "explained" by the components.
Rescaled or standardized loading is the loading divided by the variable's st. deviation; it is the correlation. (If your PCA is correlation-based PCA, loading is equal to the rescaled one, because correlation-based PCA is the PCA on standardized variables.) Rescaled loading squared has the meaning of the contribution of a pr. component into a variable; if it is high (close to 1) the variable is well defined by that component alone.
An example of computations done in PCA and FA for you to see.
Eigenvectors are unit-scaled loadings; and they are the coefficients (the cosines) of orthogonal transformation (rotation) of variables into principal components or back. Therefore it is easy to compute the components' values (not standardized) with them. Besides that their usage is limited. Eigenvector value squared has the meaning of the contribution of a variable into a pr. component; if it is high (close to 1) the component is well defined by that variable alone.
Although eigenvectors and loadings are simply two different ways to normalize coordinates of the same points representing columns (variables) of the data on a biplot, it is not a good idea to mix the two terms. This answer explained why. See also.
![](https://oscdn.geek-share.com/Uploads/Images/Content/202005/08/924031063ebb25a0fb125f6794d38ab6.jpg)
ttnphns
In PCA, you split covariance (or correlation) matrix into scale part (eigenvalues) and direction part (eigenvectors). You may then endow eigenvectors with the scale: loadings. So, loadings are thus become comparable by magnitude with the covariances/correlations observed between the variables, - because what had been drawn out from the variables' covariation now returns back - in the form of the covariation between the variables and the principal components. Actually, loadings are the covariances/correlations between the original variables and the unit-scaled components. This answer shows geometrically what loadings are and what are coefficients associating components with variables in PCA or factor analysis.
Loadings:
Help you interpret principal components or factors; Because they are the linear combination weights (coefficients) whereby unit-scaled components or factors define or "load" a variable.
(Eigenvector is just a coefficient of orthogonal transformation or projection, it is devoid of "load" within its value. "Load" is (information of the amount of) variance, magnitude. PCs are extracted to explain variance of the variables. Eigenvalues are the variances of (= explained by) PCs. When we multiply eigenvector by sq.root of the eivenvalue we "load" the bare coefficient by the amount of variance. By that virtue we make the coefficient to be the measure of association, co-variability.)
Loadings sometimes are "rotated" (e.g. varimax) afterwards to facilitate interpretability (see also);
It is loadings which "restore" the original covariance/correlation matrix (see also this thread discussing nuances of PCA and FA in that respect);
While in PCA you can compute values of components both from eigenvectors and loadings, in factor analysis you compute factor scores out of loadings.
And, above all, loading matrix is informative: its vertical sums of squares are the eigenvalues, components' variances, and its horizontal sums of squares are portions of the variables' variances being "explained" by the components.
Rescaled or standardized loading is the loading divided by the variable's st. deviation; it is the correlation. (If your PCA is correlation-based PCA, loading is equal to the rescaled one, because correlation-based PCA is the PCA on standardized variables.) Rescaled loading squared has the meaning of the contribution of a pr. component into a variable; if it is high (close to 1) the variable is well defined by that component alone.
An example of computations done in PCA and FA for you to see.
Eigenvectors are unit-scaled loadings; and they are the coefficients (the cosines) of orthogonal transformation (rotation) of variables into principal components or back. Therefore it is easy to compute the components' values (not standardized) with them. Besides that their usage is limited. Eigenvector value squared has the meaning of the contribution of a variable into a pr. component; if it is high (close to 1) the component is well defined by that variable alone.
Although eigenvectors and loadings are simply two different ways to normalize coordinates of the same points representing columns (variables) of the data on a biplot, it is not a good idea to mix the two terms. This answer explained why. See also.
相关文章推荐
- 特征值与特征向量的数值计算; Matrix Eigenvalues and Eigenvectors Calculating
- Spark MLlib特征处理:PCA 主成分分析 ---原理及实战
- UFLDL——Exercise: PCA in 2D 主成分分析
- 主成分分析(PCA)是目前应用很广泛的一种代数特征提取方法
- 机器学习 —— 基础整理(四)特征提取之线性方法:主成分分析PCA、独立成分分析ICA、线性判别分析LDA
- 特征提取——主成分分析PCA(K-L变换)及几何解释
- SparkML之特征提取(一)主成分分析(PCA)
- opencv机器学习 Haar特征 LBP特征 adaboost集成学习 级联分类器 支持向量机SVM 主成分分析PCA 人工神经网络(ANN) 笑脸检测 SVM分类 笑脸 检测
- TensorFlow - 特征值与特征向量(Eigenvalues and eigenvectors)
- 主成分分析-特征向量计算改进(1.1)
- deep learning PCA(主成分分析)、主份重构、特征降维
- 从投影的角度理解pca:向量,投影,基,内积,坐标,维数,分散程度,方差,协方差矩阵,对角化,特征值分解,主成分分析PCA
- PCA(主成分分析)、主份重构、特征降维
- 机器学习降维算法一:PCA(主成分分析算法)
- 特征脸检测-主成分分析,简单明了,转载了
- 机器学习中的数学(4)-线性判别分析(LDA), 主成分分析(PCA)
- 线性判别分析(LDA), 主成分分析(PCA)
- 线性判别分析(LDA), 主成分分析(PCA)
- 主成分分析(principal components analysis, PCA)
- 主成分分析PCA--计算过程