I am trying to fit some exponential data ($y$ is the regression target vector, and $X$ is the data matrix, $\beta$ is the coefficients that we want to optimize).
However I am getting different results from minimizing $\Vert \! \log(y)-X\beta \,\Vert_2^2$ and $\Vert y-e^{X\beta}\Vert_2^2$.
Why?
There is no surprize that the results are different.
Consider a simpler situation of a constant regression, i.e. minimizing $||y - c||_2^2$ over constants $c\in \mathbb R$. It is easy to see that the minimizer is the sample mean of $y$: $$ \hat c = \frac1n \sum_{i=1}^n y_i. $$ For logarithms, $$ \hat a = \frac1n \sum_{i=1}^n \log y_i. $$ So there is no reason to expect that $\hat c = e^{\hat a}$. In fact, by the Jensen inequality, $\hat c\ge e^{\hat a}$, and the inequality is strict unless $y$ is constant.
From the practical point of view, you are fitting different models to your data. Minimizing $||y-\cdots||_2^2$ presumes the Gauss–Markov assumptions, in particular, homoscedastic errors, while minimizing $||\log y-\cdots||_2^2$ assumes that the variance of errors is proportional to response (in other words, it assumes homoscedastic relative errors).