Let $\left(E, \langle \text{ }, \text{ }\rangle\right) $ be the space of continued functions on $\left[0,1\right]$ with $$ \langle f,g \rangle =\int_{0}^{1}f\left(t\right)g\left(t\right) \text{d}t $$ I'm studying a certain operator $U$ which is self-adjoint on $E$ given by $$ U : f \mapsto \int_{0}^{1}K\left(t,x\right)f\left(t\right)\text{d}t $$ with $K\left(t,x\right)=K\left(x,t\right)$ and $f$ a continuous function.
I've found all eigenvalues of $U$, let's call them $\left(\lambda_i\right)_{n \in \mathbb{N}}$
I consider the space $\left(c_n\right)_{n \in \mathbb{N}}$ given by $c_n : t \mapsto \sqrt{2}\cos\left(n \pi t\right)$ and $F_n=\text{Span}\left(\left(c_i\right)_{i \in [1,n]}\right)$. I know that the orthogonal projection of a function $f$ of $E$ onto $F_n$ is $$ P_{F_n}\left(f\right)=\sum_{k=1}^{n}\langle f,c_k\rangle c_k $$ I know that the $\left(c_i\right)_{i \in \mathbb{N}}$ is dense and are orthonormal like a " basis " in a finite dimensioned space. I've read that we can define, if it exists, $$ \text{Tr}\left(U\right)=\sum_{k=1}^{+\infty}\langle U\left(c_n\right),c_n\rangle $$ My question is, first, to understand how we can define this because there's no "matrix" behind this so i've difficulties understanding how ce wan show that or if it is just a notation. And, overall, I would like to know why we can extend the property of the Trace : $$ \sum_{i=1}^{+\infty}\lambda_i=\text{Tr}\left(U\right) $$ Could you provide me some help please ?
There is nothing to "show", it's a definition - and on top of that, there is a disclaimer that one defines this only provided that it makes sense, so there really is nothing to prove.
However, your claim that there is no matrix is wrong, there is a so-called infinite matrix, and the trace as per your first definition is the sum, if it exists, of all diagonal elements.
As for the equivalence of two definitions of Trace in that context, it is called Lidskii's theorem. See e.g. Trace ideals and their applications by B. Simon.