Spectral Decomposition

If $A$ can be decomposed like so (PDP^T),

$$\begin{array}{rl}A& =PD{P}^T\\ A& =\left[\begin{array}{ccc}{v}_1& v_2\dots & v_n\end{array}\right]\left[\begin{array}{cccc}{\lambda }_1& & & \\ & {\lambda }_2& & \\ & & \ddots & \\ & & & {\lambda }_n\end{array}\right]\left[\begin{array}{c}{v}_1\\ v_2\\ ⋮\\ v_n\end{array}\right]\end{array}$$

Then, $A$ has the following decomposition:

$$A={\lambda }_1\overrightarrow{u_1}{\overrightarrow{u_1}}^T+\cdots +{\lambda }_n\overrightarrow{u_n}{\overrightarrow{u_n}}^T=\sum _{i=1}^n{\lambda }_i\overrightarrow{u_i}{\overrightarrow{u_i}}^T$$

We must notice that $\overrightarrow{u_i}{\overrightarrow{u_i}}^T$ is an $n\times n$ matrix, assuming that $\overrightarrow{u_i}$ is $n\times 1$.

Proof

We see that $PD=\left[\begin{array}{cccc}P{d}_1& P{d}_2& \dots & P{d}_i\end{array}\right]$ $d_i$ are columns of $D$ We need to realize that that, $P{d}_i=P\left[\begin{array}{c}0\\ ⋮\\ 0\\ {\lambda }_i\\ 0\\ ⋮\\ 0\end{array}\right]=0+\cdots +0+p_i{\lambda }_i+0\cdots +0=p_i{\lambda }_i$. Hence $PD=\left[\begin{array}{cccc}{p}_1{\lambda }_1& p_2{\lambda }_2& \dots & p_1{\lambda }_1\end{array}\right]$ So, $PD{P}^T=\left[\begin{array}{cccc}{p}_1{\lambda }_1& p_2{\lambda }_2& \dots & p_1{\lambda }_1\end{array}\right]\left[\begin{array}{c}{u}_1^T\\ u_2^T\\ ⋮\\ u_i^T\end{array}\right]=A$ Using math we can conclude, $\boxed{A={\lambda }_1\overrightarrow{u_1}{\overrightarrow{u_1}}^T+{\lambda }_2\overrightarrow{u_2}{\overrightarrow{u_2}}^T+\cdots +{\lambda }_n\overrightarrow{u_n}{\overrightarrow{u_n}}^T=\sum _{i=1}^n{\lambda }_i\overrightarrow{u_i}{\overrightarrow{u_i}}^T}$

Example

Properties

• ${\lambda }_n\overrightarrow{u_n}{\overrightarrow{u_n}}^T$ will be $n\times n$ and have a Rank of 1.

Approximation

Using The SVD

$$A=\sum _{i=1}^r{\sigma }_i{\vec{u}}_i{\vec{v}}_i$$

${\vec{u}}_i,{\vec{v}}_i$ are the $i^{\text{th}}$ columns of $U$ and $V$ respectively. ${\sigma }_i$ is the Singular Values.

Example