SVD for Seam Carving

Question

Could SVD be used for Seam Carving ? I am making a small program for a uni course and I'm looking for different ways to calculate pixel energy; which made me come across SVD.

Among others, I have read this link and this one and it makes me wonder if SVD could be used as a good energy function.
I still have trouble understanding how I could make sense of the matrices in that regard. Any advice ?

Answer 1

I doubt heavily about it, for Seam Carving you need to find an energy function that hints you about the different structures in the image, these are well represented by contours (thats why one of the best energy functions is gradient magnitude).

In SVD you decompose the matrix as a sum of rank 1 matrices, with their respective singular values

$ A = \sum \sigma_i u_i v_i^t $

So you are decomposing the image $A$ in $rank(A)$ ammount of images, and from them you would like to construct this energy function. Problem is that these rank 1 matrices look terrible for Seam Carving, as there are no contours from each of them individually (take for example the first image obtained in the first link you provided, that is a typical rank 1 matrix).

You could test this super fast (if you have matlab), here you have a code to see it:

% Get any image

I = double(imread('cameraman.tif'));

% Obtain the SVD decomposition

[U,S,V]=svd(I);

% Define J, an image composed of stacking up the rank 1 matrices

J=zeros(size(I));

% Here you start adding up all the matrices, this setting adds all matrices up but modifiying "a" and "b" you can get subsets of the SVD

a=0;

b=0;

for i=1+b:rank(I)-a

J = J + S(i,i)*U(:,i)*V(:,i)';

%S(i,i) are the singular values, and U(:,i)*V(:,i)' are the rank 1 matrices.

end

imshow(J/max(max(J)))