Lecture 20 (Apr 26)

Deconvolution

$g=h*f$, given both $g$ and $h$, want to determine $f$.

If only $g$ is given, we have a blind deconvolution problem.

$$
f=\mathcal{F}^{-1}\left[ \frac{\mathcal{F}(g)}{\mathcal{F}(h)} \right] \, .
$$

If there is noise, $g=h*f+n$, then
$$
f=\mathcal{F}^{-1}\left[ \frac{\mathcal{F}(g)}{\mathcal{F}(h)} \right]
- \mathcal{F}^{-1}\left[ \frac{\mathcal{F}(n)}{\mathcal{F}(h)} \right]
\, .
$$
Problems:
1. $n$ is an unknown;
2. $|\hat{h}_{p,q}| \sim 0$ will amplify the effect from the error term.

Optimization problem

Given $h$ (and so the matrix $H$) and $g$, want to determine $f$ such that
(1) $Hf=g$;
(2) $\min_f \|Hf-g\|_2^2$.

A circulant matrix $H$ can be diagonalized by Fourier transform, i.e. $H=WDW^{-1}$ where $W$ is the Fourier matrix. So $WDW^{-1} f =g$ and we also have $f=WD^{-1}W^{-1} g$, i.e. we determine $f$ by
1. compute the Fourier transform of $g$;
2. divide the Fourier coefficients by the Fourier coefficient of $h$;
3. compute the inverse Fourier transform.

Some matrix operations:
1) $\nabla_x (a^T x)=a$;
2) $\nabla_x (x^T A^T A x)=2 A^T A x$.

The minimizer to the minimization problem $\min_f \|Hf-g\|_2^2$ is given by
$$
f = (H^* H)^{-1} H^* g \, .
$$

Constrained optimization

$$
\min_x f(x)
$$
subject to $g(x)=0$.

Or, using a Lagrange multiplier to obtain an unconstrained optimization problem:
$$
\min_{x,\lambda} f(x)+\lambda g(x) \, .
$$

Constrained optimization formulation for deconvolution with noise:
$$
\min_f \|Q f\|_2^2
$$
subject to $\|Hf-g\|_2^2=\|n\|_2^2$, for some linear location-invariant operator $Q$, i.e. both $H$ and $Q$ can be diagonalized by the Fourier transform, $H=WDW^{-1}$ and $Q=WAW^{-1}$. The minimizer to the constrained problem can be solved by introducing a Lagrange multiplier. The solution is
$$
f=W\left(D^T D+\frac{1}{\lambda} A^T A \right)^{-1} D^T W^{-1} g \, .
$$

In the component form, we have
$$
f = \mathcal{F}^{-1} \left( \frac{d_i^*}{|d_i|^2 + \frac{1}{\lambda} |a_i|^2} \hat{g}_i \right) \, ,
$$
where $\hat{g}=\mathcal{F}(g)$.

Limitation

1) minimization w.r.t. $\lambda$: eye-norm, i.e. try various $\lambda$ and pick the best solution to human;
2) What is $Q$? For example, $Qf=\nabla^2 f$.

Reading materials

Lecture note: p.100-104