If I have a set of data that $(x_i,y_i)$ could be visualized on a scatter plot, and I want to apply the least-square method to fit them using a Gaussian function:
$$ G(x) = B+A\exp\left[-\left(\frac{x-\mu}{\sigma}\right)^2\right] $$
I think this would give us an estimation of the expected value $\mu$. However, is there a way I can find the error of those 4 parameters $A,B,\mu,\sigma$ using this method?
I assume you are modeling your problem traditionally as $Y_i|x_i\sim \mathcal N(\beta_0+\beta_1x_i, \sigma^2)$ which indeed can be visualized on a scatterplot. In this case the MLE's of $\beta_0$ and $\beta_1$ are the least square estimators ($\hat\beta_1=\frac{\sum_{i=1}^n(Y_i-\bar Y)(x_i-\bar x)}{\sum_{i=1}^n (x_i-\bar x)}$, $\hat\beta_0=\bar Y-\hat\beta_1 \bar x$) and they have MSE (mean squared error) $E(\hat\beta_1-\beta_1)^2=Var(\hat\beta_1)=\sigma^2/s_x^2$ and $E(\hat\beta_0-\beta_0)^2=Var(\hat\beta_0)=\sigma^2\left(\frac 1n+\frac{\bar x^2}{s_x^2}\right)$ where $s_x=(\sum_{i=1}^n (x_i-\bar x)^2)^{\frac 12}$ owing to the fact that they are unbiased estimators.