Consider a Hermitian vector bundle $\pi: E \to X$ where $X$ is a complex manifold. Being Hermitian means that there exists a hermitian inner product for each fiber $\pi^{-1}(p)$. So if $X,Y \in \mathbb{C}^n$ for even $n$ are two sections on the fiber $E_p$ over $p \in X$ then the Hermitian inner product is defined as $$ (X,Y) = X^T H \bar{Y} $$ where the bar means complex conjugation. I am bit confused now on what this actually means. The product is defined w.r.t. a Hermitian matrix $H$. Does this $H$ vary as we move around the base? And, of course, what happens if the fibers are isomorphic to $\mathbb{C}^{r \times r}$? This is still a complex vector space. What is the definition of the inner product there?
The reason I am confused is because in the book Kobayashi-Hitchin correspondence by Lubke and Teleman, in page 20 they define a bilinear form $$ (f_1,f_2)= \int_X \text{tr} (f_1 \cdot f_2^{*}) \text{vol}_g $$ where $g$ is a Riemannian metric on $X$ and $f^{*}$ means adjoint of $f$ with respect to $g$. I do not understand this either :(
Any help would be appreciated.
I'll break down the linear algebra first. Then the point will be that what I'm saying holds on each fiber, as long as we imagine the hermitian matrix $H$ as varying with the base (read: section of $\operatorname{End}(E)$).
Start with a vector space $V$ over $\mathbb{C}$. If $H$ is a hermitian matrix, we'll define a hermitian inner product $h(u,v) = v^T H \bar{u}$. Now, given any matrix $A \in \operatorname{End}(V)$, we can define its adjoint $A^*$ with respect to the metric $h$ by the formula $$ h(Au, v) = h(u, A^*v). $$ Then we can derive a formula for $A^*$: $$ \begin{align*} h(Au, v) &= v^T H \bar{A} \bar{u}\\ &= v^T (H\bar{A} H^{-1}) H \bar{u}\\ &= \bigg((H \bar{A} H^{-1})^T v \bigg)^T H \bar{u}\\ &= h(u, A^* v), \end{align*} $$ where now $A^* = (H \bar{A} H^{-1})^T = \bar{H} \bar{A}^T \bar{H}^{-1}$. This gives you a very concrete way of computing an adjoint of a matrix with respect to any hermitian inner product.
We can also define an induced hermitian inner product on the vector space $\operatorname{End}(V)$, which I will call $\tilde{h}$, and here's how I define it: if $A, B \in \operatorname{End}(V)$, then $$ \tilde{h}(A,B) = \operatorname{tr}(A^* B), $$ where $A^*$ is the same adjoint as above.
(By the way, a good exercise to do here is write down what $\operatorname{tr}(\bar{A}^T B)$ (so the simple case $H = \operatorname(id)$) is in terms of the entries of $A$ and $B$. You should find that it corresponds with the usual inner product of vectors in $\mathbb{C}^{r^2}$, so in some sense this is the most natural way we could have defined the inner product on $\operatorname{End}(V)$.)
So now think about a hermitian structure $h$ on a bundle $E\to X$. You can pretend that $h$ is defined by a matrix, which is absolutely true when you work in a local frame: if $e_1,\dots, e_r$ is a local frame for $E$, you get a matrix of smooth functions $H$, where the $i,j$ entry is the smooth function $H_{ij} = h(e_j,e_i)$.
Also, just like the linear algebra above, any endomorphism $f$ of $E$ has an adjoint $f^*$, defined again by $h(fs, t) = h(s,f^*t)$ for sections $s,t$ of $E$. In local coordinates we will again have the formula $$f^* = \bar{H} \bar{f}^T \bar{H}^{-1}.$$ Just beware: the transpose operation $(-)^T$ does not make sense globally! Only the adjoint $(-)^*$ makes sense globally.
Finally, this bilinear form that Teleman and Lübke define is just the globalized version of the induced metric $\tilde{h}$ I defined above: if $f_1,f_2$ are two endomorphisms, $\operatorname{tr}(f_1 f_2^*)$ defines a smooth function on $X$ (because fiberwise it gives a number), so integrating it over $X$ produces a number.