I am preparing myself for my upcoming math exam and one of the preparation exercises includes
(i) showing that the DCT-II is invertible
(ii) determining the formula of the inverse DCT-II
The DCT-II is given by
$$ y_k = \sum_{j = 0}^{n - 1} x_j \cdot \cos(\pi k(j + 1/2)/n), \qquad k=0\dots n-1 $$
I started by converting this equation into a matrix equation:
$$\begin{bmatrix} \vdots \\[0.3em] y_k \\[0.3em] \vdots \end{bmatrix} = \begin{bmatrix} \cdots && \cdots && \cdots \\ \vdots && \cos(\pi k(j + 1/2)/n) && \vdots \\ \cdots && \cdots && \cdots \\ \end{bmatrix} \cdot \begin{bmatrix} \vdots \\[0.3em] x_k \\[0.3em] \vdots \end{bmatrix} $$
Now the next step would be to show the existence of
$$ \begin{bmatrix} \cdots && \cdots && \cdots \\ \vdots && \cos(\pi k(j + 1/2)/n) && \vdots \\ \cdots && \cdots && \cdots \\ \end{bmatrix}^{-1} $$
and to finally determine the inverted matrix. What is the best way to do that?
You might consider this approach: http://www-math.mit.edu/~gs/papers/dct.pdf
The idea is that the dct vectors are eigenvector a of a particular difference operator related to ODE (laplacian eigenvector). inverse is also easy to show once you have orthogonality.
Another approach might be relation to FFT.