Lecture 8 (Mar 4)

Nonlinear Transforms

Denoising: Median filter.

Two different norms for the gradient in two dimensions:
$$
\|\nabla f\|_1 = \left| \frac{\partial f}{\partial x} \right| + \left| \frac{\partial f}{\partial y} \right| \\

\|\nabla f\|_2 = \left[ \left( \frac{\partial f}{\partial x} \right)^2 + \left(\frac{\partial f}{\partial y} \right)^2 \right]^{\frac{1}{2}}
$$

Eigenvalue Decomposition (EVD)

If $A$ is not square, there is no EVD.

Even if it's square, there is no guarantee that it has a complete set of e-vectors.

Singular Value Decomposition (SVD)

Geometrical observation: The image of the unit sphere under any $M\times N$ matrix is a hyperellipse.

Assume $A\in \mathbb{R}^{M\times N}$ is full rank, $M\ge N$.

Def ($N$ singular values of $A$): length of the $N$ principal semiaxes of $AS$, i.e. $\sigma_1, \sigma_2, \cdots, \sigma_N$ such that $\sigma_1 \ge \sigma_2 \ge \cdots \ge \sigma_N >0$.

Def ($N$ left singular vectors of $A$): $\{ u_1, u_2, \cdots, u_N \}$ orthonormal, principal semiaxes directions.

Def ($N$ right singular vectors of $A$): $\{ v_1, v_2, \cdots, v_N \}$, preimage of $\sigma_i u_i$. It turns out these vectors are orthonormal.

Reduced SVD:

$$

A=\hat{U} \hat{\Sigma} V^*

$$

where $\hat{U}\in\mathbb{R}^{M\times N}$, $\hat{\Sigma}\in\mathbb{R}^{N\times N}$ and $V\in\mathbb{R}^{N\times N}$.

Reading materials

Lecture Notes: pp.50-57