Let $\mathcal{S} = \{(\delta,0,0),(0,\delta,0),(0,0,\delta)\}$. If we consider a random variable $S$ defined over $\mathcal{S}$ such that \begin{equation} S = \begin{cases} s, & \text{ if } s\in\mathcal{S}\\ 0, & \text{otherwise}. \end{cases} \end{equation} Then $\mathbb{P}(S=s) = 1/m$, where $m = |S|$ cardinality of $\mathcal{S}$,
What is $\mathbb{E}(S^{T}AS)$?
My attempt: $\mathbb{E}(S^{T}AS) = trace{(A\Sigma)} + \mu^{T}A\mu$, where $\mu$ is the expected value of $S$ and $\Sigma$ is the covariance matrix. In this case how to find $\Sigma$?
Since $|\mathcal{S}| = 3$ the easiest way will be by direct enumeration:
\begin{align*} \mathbf{E}[S^T A S] & = \frac{1}{3} \bigg( (\delta,0,0)^T A (\delta, 0,0) + (0,\delta,0)^T A (0,\delta, 0) + (0,0,\delta)^T A (0,0,\delta) \bigg) \end{align*}
If $A = (a_{i,j})$ with $1 \leq i \leq 3, 1 \leq j \leq3$ then
\begin{align*} (\delta,0,0)^T A (\delta, 0,0) & =a_{1,1}\delta^2 \\ (0,\delta,0)^T A ( 0,\delta, 0) & =a_{2,2}\delta^2 \\ (0,0,\delta)^T A ( 0,0,\delta) & =a_{3,3}\delta^2 \end{align*}
Hence
$$ \mathbf{E}[S^T A S] = \delta^2(a_{1,1} + a_{2,2} + a_{3,3}) = \delta^2 \text{Tr}(A) $$
If you are really keen to solve the problem using the covariance matrix, then let us denote $S = (s_1,s_2,s_3)$ for the random element of $\mathcal{S}$ and then the covariance matrix $\Sigma$ is given by
\begin{align*} \Sigma_{i,j} \, \colon & = \mathbf{E}[S_iS_j] - \mathbf{E}[S_i]\mathbf{E}[S_j] \\ & = \delta^2\mathbf{1}_{i,j} - \mathbf{E}[S_i]\mathbf{E}[S_j] \\ & = \delta^2\mathbf{1}_{i,j} - \left(\frac{\delta}{3} \times \frac{\delta}{3} \right) \\ & =\delta^2\left( \mathbf{1}_{i,j} - \frac{1}{9}\right) \end{align*} where $\mathbf{1}_{i,j} = 1$ if $i = j$ and is $0$ otherwise. That $\mathbf{E}[S_iS_j] =\mathbf{1}{i,j}$ follows since only one element of $S$ is non-zero at any one time. Hence we have
$$ \Sigma = \delta^2 \left( I - \frac{1}{9} J \right)$$
where $I$ is the $3 \times 3$ identity matrix, and $J$ is the $3 \times 3$ matrix with all entries equal to 1.