Let A = $\begin{bmatrix}1 & \tan x\\-\tan x & 1\end{bmatrix}$ then let us define a function $f(x)=\begin{vmatrix}A^{T}A^{-1}\end{vmatrix}$ then which of the following cannot be the value of $$g(x)=\underbrace{f(f(f(\cdots\cdots\cdots{f(x)))))}}_{n\text{times}}$$ where $n\ge{2}$
There are 4 options given
A) $f^nx$
B) $1$
C) $f^{n-1}x$
D) $nf(x)$
The correct answer is D $\Rightarrow$ that $nf(x)$ can not be the value of $g(x)$
Here's what I tried -
Method 1:
I used $$A^{-1}=\frac{Adj(A)}{|A|}$$
to simplify $$f(x)=\begin{vmatrix}\frac{A^{T}Adj(A)}{A}\end{vmatrix}$$
Now after a little calculation we can find out that, $$|A|=sec^2x$$
and$$Adj(A)=A^{T}$$
therefore $f(x)$ reduces to, $$f(x)=\frac{|A^T.A^T|}{sec^2x}$$
After solving $A^T.A^T$ and taking its determinant, we finally get to
$$f(x)=\frac{(1+\tan^2x)^2}{\sec^2x}=\sec^2x$$
From here , I have to solve further $$g(x)=\underbrace{\sec^2(\sec^2(\sec^2(\cdots\cdots\cdots{\sec^2(x)))))}}_{n\text{times}}$$
But cannot move ahead from this step
Method 2:
There can be another way of solving this by using $|Adj(A)|=|A|^{n-1}$ and other properties of adjoints, but I cannot solve using those also.
Note that $f(x)=|A^T\cdot A^{-1}|=|A^T|\cdot |A^{-1}|=|A|\cdot |A^{-1}|=1$, hence $g(x)=1$. So since $f^n x=1=f^{n-1}x$, answers A, B and C are correct, but $g(x)\ne n f(x)=n$ if $n\ne 1$.