Is there a general formula for the laplace transform $\mathcal L[f^2(t)]$ in terms of the laplace transform $\mathcal L[f(t)]$ and/or some other terms?
ps. The reason I'm asking is to solve the ODE $y''=y^2$. If this cannot be solved with laplace transforms I'd also like to know how then to solve it. But primarily I'd like to know the answer to the main question.
There is a formula for Laplace transforms of products:$$\mathcal L [fg](s)=\frac 1 {2\pi i} \int_{c - i\infty}^{c + i\infty} \mathcal L[f](z)\mathcal L[g](s-z)dz,$$ (with $Re(c)$ sufficiently large that the integral defining $\mathcal L[f](z)$ converges for all $z$ on the contour).
From this, one can obtain a formula for $\mathcal L[f^2](s) = \mathcal L[f.f](s)$.
But I don't think this is the right approach for your differential equation. Usually, Laplace transforms and Fourier transforms are used for linear differential equations. Your ODE isn't linear.
Can I suggest you solve the ODE in the following way?
Multiply both sides by $y'$: $$ y'y'' + y'y^2 = 0.$$ Notice that the left-hand side is $(\frac 1 2 y'^2+\frac 1 3 y^3)'$. So integrating gives $$ \frac 1 2 y'^2 + \frac 1 3 y^2 = c,$$ We can now separate variables, giving $$ \pm \int \frac{dy}{\sqrt{2c - \frac 2 3 y^3}} = \int dx = x + a.$$ which gives a solution in terms of an elliptic integral.