Why HOSVD is better than matrix SVD?

Question

For a Tensor $ R^{I \times J \times K}$ of images where $I$ represents $x$ axis, $J$ represents $y$ axis, and $K$ represents the number of images, we can apply HOSVD and do dimensionality reduction. However the same task can be done by individually computing the matrix SVDs of each image. So what is the additional benefit we are getting in computing HOSVD over SVD? (Computationally efficient is of no great use as almost both takes same time.)

Answer 1

If you do SVD on each slice, you've decoupled the problem into $K$ SVD problems that are no longer related to each other. If you perform a high-order SVD on a tensor, you're assuming that there is underlying multilinear structure that relates the values across all three indices $i,j,k$.

It is also important to note that there are many types of tensor decompositions that partially behave like the SVD for linear maps, and constructing such decompositions is an active area of research. See "Tensor Decompositions and Applications" by Kolda and Bader.