linear transformations with f transpose

Question

I am half confused with the following problem..

and

Then is equal to

and i have 4 choices:

I don't understand when it says Nu (f∘f), the nucleus of a linear transformation is not zero? I am quite confused. My idea was to get image and nucleus of f and then the associated matrix transposed. but I'm clearly confused.

Answer 1

You have$$(f\circ f)(x_1,x_2,x_3,x_4)=(-2x_2-x_3-x_4,2x_2,x_3+x_4,x_3+x_4).$$So,$$\ker(f\circ f)=\{(a,0,b,-b)\mid a,b\in\Bbb R\}.$$For any element $(a,0,b,-b)$ of $\ker(f\circ f)$, you have $f(a,0,b,-b)=(a-b,0,a-b,-a+b)$, which belongs to $\Bbb H$ if and only if $a-b=-a+b$, which means that $a=b$. So$$\ker(f\circ f)\cap f^{-1}(\Bbb H)=\{(a,0,a,-a)\mid a\in\Bbb R\}.$$