UFLDL Exercise:PCA in 2D

这一节主要讲的是pca，pca白化，zca白化，主要的作用就是降维，具体方法见ufldl教程。不过有一个地方不懂，为什么数据的协方差矩阵的特征向量就是数据变化方向的基向量？

step 1:Implement PCA to obtain U

%% Step 1a: Implement PCA to obtain U
%  Implement PCA to obtain the rotation matrix U, which is the eigenbasis
%  sigma.

% -------------------- YOUR CODE HERE --------------------
u = zeros(size(x, 1)); % You need to compute this
avg = mean(x,2);
x = x - repmat(avg,1,size(x,2));
sigma = x * x' / size(x,2);
[u,s,v] = svd(sigma);

% --------------------------------------------------------
hold on
plot([0 u(1,1)], [0 u(2,1)]);
plot([0 u(1,2)], [0 u(2,2)]);
scatter(x(1, :), x(2, :));
hold off

结果如下

step 2: Compute xRot, the projection on to the eigenbasis

%% Step 1a: Implement PCA to obtain U
%  Implement PCA to obtain the rotation matrix U, which is the eigenbasis
%  sigma.

% -------------------- YOUR CODE HERE --------------------
u = zeros(size(x, 1)); % You need to compute this
avg = mean(x,2);
x = x - repmat(avg,1,size(x,2));
sigma = x * x' / size(x,2);
[u,s,v] = svd(sigma);

% --------------------------------------------------------
% hold on
% plot([0 u(1,1)], [0 u(2,1)]);
% plot([0 u(1,2)], [0 u(2,2)]);
% scatter(x(1, :), x(2, :));
% hold off

%%================================================================
%% Step 1b: Compute xRot, the projection on to the eigenbasis
%  Now, compute xRot by projecting the data on to the basis defined
%  by U. Visualize the points by performing a scatter plot.

% -------------------- YOUR CODE HERE --------------------
xRot = zeros(size(x)); % You need to compute this
xRot = u' * x;

% --------------------------------------------------------

% Visualise the covariance matrix. You should see a line across the
% diagonal against a blue background.
figure(2);
scatter(xRot(1, :), xRot(2, :));
title('xRot');

结果如下

step 3:Reduce the number of dimensions from 2 to 1.

%% Step 2: Reduce the number of dimensions from 2 to 1.
%  Compute xRot again (this time projecting to 1 dimension).
%  Then, compute xHat by projecting the xRot back onto the original axes
%  to see the effect of dimension reduction

% -------------------- YOUR CODE HERE --------------------
k = 1; % Use k = 1 and project the data onto the first eigenbasis
xHat = zeros(size(x)); % You need to compute this
xTilde = u(:,1:k)' * x;%数据的降维表示
xHat = u(:,1:k) * xTilde;%还原数据

% --------------------------------------------------------
figure(3);
scatter(xHat(1, :), xHat(2, :));
title('xHat');

结果如下

step 4:PCA Whitening

%% Step 3: PCA Whitening
%  Complute xPCAWhite and plot the results.

epsilon = 1e-5;
% -------------------- YOUR CODE HERE --------------------
xPCAWhite = zeros(size(x)); % You need to compute this
xPCAWhite = diag(1./sqrt((diag(s) + epsilon))) * xRot;

% --------------------------------------------------------
figure(4);
scatter(xPCAWhite(1, :), xPCAWhite(2, :));
title('xPCAWhite');

结果如下

step 5:ZCA Whitening

%% Step 3: ZCA Whitening
%  Complute xZCAWhite and plot the results.

% -------------------- YOUR CODE HERE --------------------
xZCAWhite = zeros(size(x)); % You need to compute this
xZCAWhite = u * xPCAWhite;

% % --------------------------------------------------------
figure(5);
scatter(xZCAWhite(1, :), xZCAWhite(2, :));
title('xZCAWhite');

结果如下