I tried to solve Newton's law $m\frac{d^2x}{dt^2}=F(t)$ from a Green's function point of view, assuming the particle is initially at rest at the origin and the force $F(t)$ only starts at $t=0$.
I wrote $m\frac{d^2G(t)}{dt^2}=\delta(t)$. I want $G(t)=0$ for $t<0$. For $t>0$ I have $G=at+b$. Imposing continuity at $t=0$ I get $b=0$. Imposing that $G'$ has a jump of $1/m$ at $t=0$ I get $a=1/m$. Thus $G(t)=t/m$ for $t>0$.
Then, the solution will be $x(t)=\int_0^t F(\tau)(\tau/m)d\tau$.
On the other hand, usual kinematics says $a(\tau)=F(\tau)/m$, with $v(s)=\int_0^s a(\tau)d\tau$ and $$x(t)=\int_0^t v(s)ds=\int_0^t ds\int_0^s a(\tau)d\tau=\int_0^t F(\tau)[(t-\tau)/m]d\tau.$$
This expression for $x(t)$ does not match my solution via Green's function!
The methodology in the OP was mostly correct. But, we need to move the source point from time $0$ to an arbitrary time $t'$.
Then, the Green (or Green's) function is a solution to the ODE
$$\frac{d^2G(t|t')}{dt^2}=\delta(t-t')$$
Then, with $G(t'|t')=0$ we have
$$G(t|t')=\begin{cases}0&,t<t'\\\\At+B&,t>t'\end{cases}$$
Enforcing continuity at $t'$ reveals $At'+B=0$, while enforcing discontinuity of the first derivative at $t'$ reveals $A=1$.
Hence, we find that
$$G(t|t')=\begin{cases}0&,t<t'\\\\t-t'&,t>t'\end{cases}$$
The solution to the problem of interest is
$$x(t)=\frac1m\int_0^t (t-t')F(t')\,dt'$$
as expected!