Let inner product space V be defined over F or C and linear operators T on V, evaluate $T^{*}$ at the given vector in V.
$V=R^2, T(a,b)=(2a+b,a-3b), x=(3,5)$
I know $T^{*}$ is the conjugate transpose. But how am I supposed to approach this, need some general idea.
Using the definition of adjoints, you know that, $$\langle T^*(3, 5), (a, b)\rangle = \langle (3, 5), T(a, b)\rangle$$ for all $(a, b) \in \Bbb{R}^2$. In $\Bbb{R}^2$, this inner product is the dot product. Let $(x, y) = T^*(3, 5)$. We then have $$(x, y) \cdot (a, b) = (3, 5) \cdot T(a, b) = (3, 5) \cdot (2a + b, a - 3b),$$ for all $(a, b) \in \Bbb{R}^2$. We must solve this for $x$ and $y$.
First, try $(a, b) = (1, 0)$. Then, $$x = (x, y) \cdot (1, 0) = (3, 5) \cdot (2, 1) = 11.$$ Next, try $(a, b) = (0, 1)$. Then, $$y = (x, y) \cdot (0, 1) = (3, 5) \cdot (1, -3) = -12.$$ Thus, $T^*(3, 5) = (x, y) = (11, -2)$.