How do you read this formula?
$$\left\langle \dfrac{1}{\sqrt{2}} ( \alpha\beta - \beta\alpha) \bigg\rvert \hat{S^2} \bigg\rvert \dfrac{1}{\sqrt{2}} ( \alpha\beta - \beta\alpha)\right\rangle$$
$\alpha$ and $\beta$ are the two possible states of the electron, while $\hat{S^2}$ is the square of the total angular momentum operator
From your comment you appear to be considering the Casimir operator $ \hat{S} ^2$ instead, and not $\hat {S^y}$. In that case, you just do what your textbook teaches you. I'll skip the caret, since the Ss are always operators, and remind you of the coproduct, $$ S_i |\alpha\beta \rangle = S_i (|\alpha\rangle \otimes |\beta \rangle)\\ = ( S_i |\alpha\rangle ) \otimes |\beta \rangle + |\alpha\rangle \otimes (S_i |\beta \rangle) ~~~\leadsto \\ S_i S_i |\alpha\beta \rangle = ( S_i S_i |\alpha\rangle ) \otimes |\beta \rangle +2 (S_i |\alpha\rangle) \otimes (S_i |\beta \rangle) + |\alpha\rangle \otimes (S_i S_i |\beta \rangle) , $$ where the summation convention on all three indices i is implied.
You may take it from there and read off the four terms of your expression, $\left\langle \dfrac{1}{\sqrt{2}} ( \alpha\beta - \beta\alpha) \bigg\rvert {S}^2 \bigg\rvert \dfrac{1}{\sqrt{2}} ( \alpha\beta - \beta\alpha)\right\rangle. $
This is how you read it. However, I strongly suspect you are also asking: How do you evaluate it? In general, it is a mess and involves the Clebsch-Gordan series.
However, if $|\alpha\rangle$ and $|\beta\rangle$ are in the same representation, and notably the doublet, as per your comments, then, oh, the joy! $$ S^2 |\alpha\rangle = 3/4~|\alpha\rangle, ~~ S^2 |\beta\rangle = 3/4~|\beta\rangle, ~~~ \vec S| ( \alpha\beta - \beta\alpha)\rangle =0, $$ and therefore $$ S^2| ( \alpha\beta - \beta\alpha)\rangle = 0, $$ and you expression vanishes.