$e^{-x}=y(x)+2\int_0^x\cos (x-t)y(t)dt$
$L[e^{-x}]=L[y(x)]+2L[\cos (x)]L[y(x)]$
$\frac{1}{1+p}=L[y(x)](1+2L[\cos (x)])$
$\frac{1}{1+p}=L[y(x)](1+\frac{2p}{p^2+1})$
$\frac{\frac{1}{1+p}}{(1+\frac{2p}{p^2+1})}=L[y(x)]$
$\frac{\frac{1}{1+p}}{\frac{p^2+2p+1}{p^2+1}}=L[y(x)]$
$\frac{p^2+1}{(1+p)(p^2+2p+1)}=L[y(x)]$
$y(x)=L^{-1}[\frac{p^2+1}{(1+p)(p^2+2p+1)}]$
$y(x)=L^{-1}[\frac{p^2+1}{(1+p)(p+1)^2}]$
After solving using partial fractions I obtained,
$y(x)=L^{-1}[\frac{1}{(x+1)}]-L^{-1}[\frac{2}{(x+1)^2}]+L^{-1}[\frac{2}{(x+1)^3}]$
$y(x)=e^{-x}-L^{-1}[\frac{2}{(x+1)^2}]+L^{-1}[\frac{2}{(x+1)^3}]$
Could someone please let me know if my work is correct and help me figure out what the inverse term becomes?
Since you have applied the convolution property, I assume you are familiar with shifting theorems. The $p$-shifting theorem states that $$\mathcal{L}\{e^{-at} f(t)\} = F(p+a).$$
The Laplace transform of a polynomial is $$\mathcal{L}\{t^n\} = \frac{n!}{p^{n+1}}.$$ Then, a straightforward application of the shifting theorem (in the $p$-variable) implies that $$\mathcal{L}^{-1}\bigg\{ \frac{1}{(p+a)^{n+1}} \bigg\} = \frac{t^n e^{-at}}{n!}.$$ For instance, one of your terms is $$\mathcal{L}^{-1}\bigg\{ \frac{2}{(p+1)^{3}} \bigg\} = \frac{2 t^2 e^{-t}}{2!} = t^2 e^{-t}.$$