Singular Value Decomposition allows factorisising an arbitrary matrix (real or complex), while a lot of other matrix factorization like with diagonalising etc. has often contraints.

A=UΣVT A = U \Sigma V^T

#
Manually

I’m showing this in an example, making it more visually appealing and understandable than some formal way (which I’m also quite bad at writing xD)

So lets assume we’re given the Matrix

A=(−3−12113) A = \begin{pmatrix} -3 && -1 && 2 \\ 1 && 1 && 3 \end{pmatrix}

1. Calculate G=ATA G = A^TA

AT=(−31−1123) A^T = \begin{pmatrix} -3 && 1 \\ -1 && 1 \\ 2 && 3 \end{pmatrix}

So based on this:

G=(104−3.21..13) G = \begin{pmatrix} 10 && 4 && -3 \\ . && 2 && 1 \\ . && . && 13 \end{pmatrix}

Note: the values with . don’t need to be calculated, since the result of AT∗A A^T * A is always symmetric. So using this property we get:

G=(104−3421−3113) G = \begin{pmatrix} 10 && 4 && -3 \\ 4 && 2 && 1 \\ -3 && 1 && 13 \end{pmatrix}

2. Eigenvalues of G

Now lets get the determinant of G−λI G - \lambda I and set it 0 (to get the eigenvalues).

G−λI=(10−λ4−342−λ1−3113−λ) G - \lambda I = \begin{pmatrix} 10 - \lambda && 4 && -3 \\ 4 && 2 - \lambda && 1 \\ -3 && 1 && 13 - \lambda \end{pmatrix}

Following the last column down, we get:

det(G−λI)=−3∗(4+3∗(2−λ))−((10−λ)+12)+(13−λ)∗((10−λ)∗(2−λ)−16)) det(G - \lambda I) = -3 * (4 + 3 * (2 - \lambda)) - ((10 - \lambda) + 12) + (13 - \lambda) * ((10 - \lambda) * (2 - \lambda) - 16))

which can be shortened in the polynom:

det(G−λI)=−λ3+25λ2−150λ det(G - \lambda I) = - \lambda ^3 + 25 \lambda ^2 - 150 \lambda

based on this we get: λ3=0 \lambda_3 = 0 (which one could also infer just based from that rank(A) = 2)

and the remaining polynom:

−λ2+25λ−150=0 - \lambda ^2 + 25 \lambda - 150 = 0

putting it into the abc-formula thing, we get:

λ1/2=−25±5−2λ2=10;λ1=15 \lambda_{1 / 2} = \frac{-25 \pm 5}{-2} \qquad \lambda_2 = 10 ; \quad \lambda_1 = 15

3. Σ\Sigma-Matrix

This gives us σ1=15;σ2=10 \sigma_1 = \sqrt{15}; \quad \sigma_2 = \sqrt{10}

And the following Σ \Sigma-Matrix:

Σ=(15000100)\Sigma = \begin{pmatrix} \sqrt{15} && 0 && 0 \\ 0 && \sqrt{10} && 0 \end{pmatrix}

4. Getting the Eigenspaces

Using the Eigenvalues λi \lambda_i and putting them into G−λI G - \lambda I and solving this linear system we get the Eigenspaces.

So for λ=0\lambda = 0 we have:

(104−3421−3113) \begin{pmatrix} 10 && 4 && -3 \\ 4 && 2 && 1 \\ -3 && 1 && 13 \end{pmatrix}

after some line additions etc. (you could do it in order with Gauß, but doing it like how I feel is faster for me)

(20−5421000) \begin{pmatrix} 2 && 0 && -5 \\ 4 && 2 && 1 \\ 0 && 0 && 0 \end{pmatrix}

which gives us:

E0=⟨(5−112)⟩ E_0 = \langle \begin{pmatrix} 5 \\ -11 \\ 2 \end{pmatrix} \rangle

For λ=10\lambda = 10 and λ=15\lambda = 15 :

I’m keeping it short, since its basically the same

λ=10\lambda = 10:

(04−34−81000)  ⟹  E10=⟨(534)⟩ \begin{pmatrix} 0 && 4 && -3 \\ 4 && -8 && 1 \\ 0 && 0 && 0 \end{pmatrix} \implies E_{10} = \langle \begin{pmatrix} 5 \\ 3 \\ 4 \end{pmatrix} \rangle

λ=15\lambda = 15:

(0004−1311−50)  ⟹  E15=⟨(51−7)⟩ \begin{pmatrix} 0 && 0 && 0 \\ 4 && -13 && 1 \\ 1 && -5 && 0 \end{pmatrix} \implies E_{15} = \langle \begin{pmatrix} 5 \\ 1 \\ -7 \end{pmatrix} \rangle

5. Getting V

For this we need to norm the bases of the eigenspaces.

15: (51−7)/52+12+(−7)2=(51−7)/(5∗3)\begin{pmatrix} 5 \\ 1 \\ -7 \end{pmatrix} / \sqrt{5^2 + 1^2 + (-7)^2} = \begin{pmatrix} 5 \\ 1 \\ -7 \end{pmatrix} / (5 * \sqrt{3})

10: (534)/50=(534)/(5∗2)\begin{pmatrix} 5 \\ 3 \\ 4 \end{pmatrix} / \sqrt{50} = \begin{pmatrix} 5 \\ 3 \\ 4 \end{pmatrix} / (5 * \sqrt{2})

0: (5−112)/150=(5−112)/(5∗6)\begin{pmatrix} 5 \\ -11 \\ 2 \end{pmatrix} / \sqrt{150} = \begin{pmatrix} 5 \\ -11 \\ 2 \end{pmatrix} / (5 * \sqrt{6})

Now we need to keep the order from highest eigenvalue to lowest and take the normed eigenspace bases as V:

V=15∗(5/35/25/61/33/2−11/6−7/34/22/6) V = \frac{1}{5} * \begin{pmatrix} 5 / \sqrt{3} && 5 / \sqrt{2} && 5 / \sqrt{6} \\ 1 / \sqrt{3} && 3 / \sqrt{2} && -11 / \sqrt{6} \\ -7 / \sqrt{3} && 4 / \sqrt{2} && 2 / \sqrt{6} \end{pmatrix}

6. Getting U

U is a 2x2 matrix here and we get the vectors by: ui=1σiAviu_i = \frac{1}{\sigma_i}Av_i

u1=115∗5∗3∗(−3−12113)∗(51−7)=15∗(−2−1) u_1 = \frac{1}{\sqrt{15} * 5 * \sqrt{3}} * \begin{pmatrix} -3 && -1 && 2 \\ 1 && 1 && 3 \end{pmatrix} * \begin{pmatrix} 5 \\ 1 \\ -7 \end{pmatrix} = \frac{1}{\sqrt{5}} * \begin{pmatrix} -2 \\ -1 \end{pmatrix}

u2=110∗5∗2∗(−3−12113)∗(534)=15∗(−12) u_2 = \frac{1}{\sqrt{10} * 5 * \sqrt{2}} * \begin{pmatrix} -3 && -1 && 2 \\ 1 && 1 && 3 \end{pmatrix} * \begin{pmatrix} 5 \\ 3 \\ 4 \end{pmatrix} = \frac{1}{\sqrt{5}} * \begin{pmatrix} -1 \\ 2 \end{pmatrix}

U=15(−2−1−12) U = \frac{1}{\sqrt{5}} \begin{pmatrix} -2 && -1 \\ -1 && 2 \end{pmatrix}

Verifying

Just multiply U∗Σ∗VT U * \Sigma * V^T and you should get A

Using our values, we get:

U∗Σ=(−23−20−3220) U * \Sigma = \begin{pmatrix} -2 \sqrt{3} && - \sqrt{2} && 0 \\ - \sqrt{3} && 2 \sqrt{2} && 0 \end{pmatrix}

VT=15∗(5/31/3−7/35/23/24/25/6−11/62/6) V^T = \frac{1}{5} * \begin{pmatrix} 5 / \sqrt{3} && 1 / \sqrt{3} && -7 / \sqrt{3} \\ 5 / \sqrt{2} && 3 / \sqrt{2} && 4 / \sqrt{2} \\ 5 / \sqrt{6} && -11 / \sqrt{6} && 2 / \sqrt{6} \end{pmatrix}

U∗Σ∗VT=A U * \Sigma * V^T = A