Distance between subspacess

Distance between subspaces
Definition of distance between subspaces

Theory of the distance

Conclusion of distance

Extension

Distance between subspaces

1. Definition of distance between subspaces

How to describe the distance between subspaces? The canonical angles between subspaces can be used to define the distance between subspaces. And there are several equivalent ways to do this. Here I will talk is the definition used in the reference “Minimax sparse principal subspace estimation in high dimension”, which is based on the projection matrices of subpaces, because of the convenience to represent subspaces by there projector matrices.

Definition: Let E and F be d-dimensional subspaces of Rp with orthogonal projectors E and F. Denote the singular values of EF⊥ by <
4000
/span>s1,s2,⋯. The canonical angles between E and F are the numbers θk(E,F)=arcsin(sk)

for k=1,2,⋯,d and the angleoperator between E and F is the d×d matrix

Θ(E,F)=diag(θ1,θ2,⋯,θd).

2. Theory of the distance

Since the nonzero singular values of E−F are the nonzero singular values of EF⊥, each counted twice. (which is the Proposition 2.1 in “Minimax sparse principal subspace estimation in high dimension”). Thus

∥sinΘ(E,F)∥2F=∑k=1ds2k=∥EF⊥∥2F=12∥E−F∥2F

If we also assume that VE and VF are the orthogonal bases for subspaces E and F respectively. Then from the Proposition 2.2 in “Minimax sparse principal subspace estimation in high dimension”, we have

12infQ∈Vd.d∥VE−VFQ∥2F≤∥sinΘ(E,F)∥2F≤infQ∈Vd.d∥VE−VFQ∥2F

This is to say the distance between two subspaces is equivalent to the minimal distance between their orthogonal bases.

3. Conclusion of distance

Let VE∈Vp,d and EF∈Vp,d be two column orthogonal matrix, which is to say EtE=FtF=Id, with right rotation. Then the following is true,

∥VE−VF∥2F≤∥VEVtE−VFVtFProjection matrices∥2F

4. Extension

If we assume that VE and VF span the d-dimensional principal subspaces (E,F) of ΣE and ΣF, then from the curvature lemma (lemma 4.2) given in “Minimax sparse principal subspace estimation in high dimension” , we have

∥sinΘ(E,F)∥2F=12∥VEVtE−VFVtF∥2F≤⟨ΣE,VEVtE−VFVtF⟩λd(ΣE)−λd+1(ΣE)

∥sinΘ(E,F)∥2F=12∥VEVtE−VFVtF∥2F≤⟨ΣF,VFVtF−VEVtE⟩λd(ΣF)−λd+1(ΣF)

Combining these together,

∥VEVtE−VFVtF∥2F≤1δ⟨ΣE−ΣF,VEVtE−VFVtF⟩≤1δ∥ΣE−ΣF∥F∥VEVtE−VFVtF∥F

where δ=min{λd(ΣE)−λd+1(ΣE),λd(ΣF)−λd+1(ΣF)}

from which , we know

∥VEVtE−VFVtF∥F≤1δ∥ΣE−ΣF∥F.

Thus

∥VE−VFOrthogonal bases∥2F≤∥VEVtE−VFVtFProjection matrices∥2F≤1δ∥ΣE−ΣFSymmetric matrices∥F