I have a dataset with distribution $X\sim\mathcal{N}(\mu,\sigma^2)$, where $\mu\gt\gt \sigma^2$, such that practically all values of $X$ are positive. I wish to calculate the variance of $X^2$ (from $\mu$ and $\sigma^2$), without having to square all the values in my dataset.
I think that the Chi-squared distribution (with $k=1$) could be relevant if I had a standard normal variable, but I'm unsure how I can relate this to my particular dataset.
In my particular situation, I initially have $Y\sim\mathcal{N}(0,\sigma^2)$, and am adding a constant $\mu$ to the samples (giving $X\sim\mathcal{N}(\mu,\sigma^2)$) before squaring them. I wish to select a value for $\mu$ to give a particular $Var(X^2)$.
Use the definition of variance:
$Var(X^2) = E((X^2)^2) - E(X^2)^2 = E(X^4) - E(X^2)^2$.
To get $E(X^2)$ recall that $Var(X) = E(X^2) - E(X)^2$. You can rearrange to get $E(X^2) = E(X)^2 + Var(X)$.
Getting $E(X^4)$ is a bit harder. You have $X = Y + \mu$ in your notation. Therefore
$$E(X^4) = E((Y+\mu)^4) = E(Y^4 + 4 \mu Y^3 + 6 \mu^2 Y^2 + 4 \mu^3 Y + \mu^4)$$
and by linearity of expectation this is
$$E(Y^4) + 4 \mu E(Y^3) + 6 \mu^2 E(Y^2) + 4 \mu^3 E(Y) + \mu^4.$$
Since the distribution of $Y$ is symmetric around zero, the odd powers $E(Y)$ and $E(Y^3)$ are zero. $E(Y^2) = Var(Y) = \sigma^2$, (since $Y$ has zero mean).
It remains to find $E(Y^4)$ and go back and put everything together. I leave this to you; this can be done for example by integration by parts.