For any $N$, we can take $N$ diagonal $N\times N$ matrices $C_i$ which form an orthonormal set: Tr$(C_i,C_j)=\delta_{ij}$.
Now take any diagonal $N\times N$ matrix $D$, and form the matrix $M_{ij} = Tr(D C_i C_j)$ .
To my surprise, I find empirically that the determinant of $M$ is the determinant of $D$; in fact, the eigenvalues of $M$ are the eigenvalues of $D$.
For instance: if $N=2$, then $C_1 = diag\{1,1\}/\sqrt{2}$ and $C_2 = diag\{1,-1\}/\sqrt{2}$; then if $D=diag\{a,b\}$, $$M = \frac{1}{2}\left[\matrix{a+b & a-b \\ a-b & a+b}\right]$$ whose determinant is $ab$ and whose trace is $(a+b)$, showing it has eigenvalues $a,b$.
I suppose this must follow from some elementary facts about matrices, but so far I can't seem to prove it myself. Is it obvious? What argument or theorem does it follow from?
[Note added: since someone answered as though this were an obvious triviality, let me point out how it works for $N=3$. The $C_i$ are proportional to diag$\{1,1,1\}$, diag$\{1,-1,0\}$, diag$\{1,1,-2\}$, properly normalized for orthnormality; in this case if $D$ = diag$\{a,b,c\}$, the matrix $M$ is $$\left[ \begin{array}{ccc} \frac{a}{2}+\frac{b}{2} & \frac{a}{2 \sqrt{3}}-\frac{b}{2 \sqrt{3}} & \frac{a}{\sqrt{6}}-\frac{b}{\sqrt{6}} \\ \frac{a}{2 \sqrt{3}}-\frac{b}{2 \sqrt{3}} & \frac{a}{6}+\frac{b}{6}+\frac{2 c}{3} & \frac{a}{3 \sqrt{2}}+\frac{b}{3 \sqrt{2}}-\frac{\sqrt{2} c}{3} \\ \frac{a}{\sqrt{6}}-\frac{b}{\sqrt{6}} & \frac{a}{3 \sqrt{2}}+\frac{b}{3 \sqrt{2}}-\frac{\sqrt{2} c}{3} & \frac{a}{3}+\frac{b}{3}+\frac{c}{3} \\ \end{array} \right] $$ and it is not instantly obvious that the eigenvalues of this matrix are $a,b,c$.]
Let's take the case where $C_i$ is any orthonormal matrix (not necessarily diagonal), so $\mathrm{tr}(C_i'C_j)=N\delta_{ij}$. You can rescale the matrices by $\frac1{\sqrt N}$ to have the same property as in your question.
If we look at the matrix $M$, its eigenvalues will be given by the zeros of $\det(M-\lambda I)$.
So we look at $Z_{ij}=M_{ij}-\lambda \delta_{ij}=\mathrm{tr}(C_iDC_j')-\lambda\mathrm{tr}(\delta_{ij})$. I permuted the matrices inside the trace as it is cyclic. The multiplication by an orthogonal matrix doesn't change the trace so: $$Z_{ij}=\mathrm{tr}(C_i(D-\lambda I)C_j')$$
Now, in the case where $\lambda$ is one eigenvalue of $D$, we have that $D-\lambda I$ is singular. In that case, if we show that the matrix $Z$ is also singular then $M$ and $D$ must share the same eigenvalues (provided that eigenvalues are distinct but maybe this holds in general too).