So I have this function:
$$f(t)=\frac{1}{(1-it)^{n+1}}$$
And I have the Fourier Transform defined as
$$\hat{f}(\lambda)=\frac{1}{\sqrt{2\pi}}\int_\mathbb{R}f(t)e^{-\lambda.it}dt $$
Now my exercise is to prove that:
$\hat{f}=Ce^{-\lambda}*\lambda^n$ for $\lambda >0$ and $0$ otherwise for some constant $C$
I am having problems with the integration tried doing it by parts but i never get anything conclusive.
Well, so what we need to show is that $$ C\frac{1}{\sqrt{2\pi}}\int_0^\infty e^{-\lambda}\lambda^n e^{i\lambda t} \,d\lambda = \frac{1}{(1-it)^{n+1}} \text{.} $$ We have $$ \begin{eqnarray} \int_0^\infty e^{-\lambda}\lambda^n e^{i\lambda t} \,d\lambda = \int_0^\infty \underbrace{\lambda^n}_{u} \underbrace{e^{it\lambda-\lambda}}_{v'} \,d\lambda &=& \underbrace{\lambda^n\frac{e^{it\lambda-\lambda}}{(it-1)}\Bigg|_0^\infty}_{=0} -n\int_0^\infty \underbrace{\lambda^{n-1}}_{=u'}\underbrace{\frac{e^{it\lambda-\lambda}}{(it-1)}}_{=v} \,d\lambda \\ &=& \frac{-n}{it - 1} \int_0^\infty \lambda^{n-1}e^{it\lambda-\lambda} \,d\lambda \text{.} \end{eqnarray} $$ Doing this repeatedly shows that $$ I_n = \frac{n}{1 - it} I_{n-1} \text{ where } I_n = \int_0^\infty \lambda^n e^{it\lambda - \lambda} \,d\lambda \text{.} $$ Since $I_0 = \frac{e^{it\lambda - \lambda}}{it -1 }\bigg|_0^\infty = \frac{1}{it - 1}$, it follows that $$ \int_0^\infty e^{-\lambda}\lambda^n e^{i\lambda t} \,d\lambda = I_n = \frac{n!}{(1 - it)^{n+1}} \text{,} $$ and thus that $$ \hat f(\lambda) = \underbrace{\left(\frac{\sqrt{2\pi}}{n!}\right)}_{=C}\,e^{-\lambda}\lambda^n \quad\Leftrightarrow\quad f(t) = \frac{1}{(1 - it)^{n+1}} \text{.} $$
You can also directly compute the forward transform $\hat f$ of your $f$, i.e. $$ \hat f(\lambda) = \frac{1}{\sqrt{2\pi}}\int_{-\infty}^\infty h(z) \,dz \text { where } h(z) = \frac{e^{-i\lambda z}}{(1 - iz)^{n+1}} $$ by contour integration.
The integrand can be rewritten as $$ h_\lambda(z) = i^{n+1}\frac{e^{-i\lambda z}}{(z + i)^{n+1}} \text{,} $$ which has a single pole at $z=-i$. Let $\gamma_N^\lambda$ be the closed path that goes from $-N$ to $N$ along the real axis and then back to $-N$ along a half-circle $C^\lambda_N$ in the upper half plane if $\lambda \leq 0$, and the same path except the half-circle $C^\lambda_N$ lies in the lower half plane if $\lambda > 0$ instead. Note that $z=-i$ only lies within $\gamma_N^\lambda$ if $\lambda > 0$, i.e. the contour integral is zero otherwise. Also note that for $\lambda < 0$, the contour goes around $z=-i$ clock-wise. The inequalities $$\begin{eqnarray} |(z + i)^{n+1}| &=& |z + i|^{n+1} \geq (|z| - 1)^{n+1} \\ |e^{-i\lambda z}| &=& e^{\lambda\textrm{Im}(z)} \leq \begin{cases} 1 &\text{if $\lambda > 0$, $\textrm{Im}(z) < 0$} \\ 1 &\text{if $\lambda \leq 0$, $\textrm{Im}(z) > 0$} \\ \end{cases} \\ \int_{C^\lambda_N} h_\lambda(z) \,dz &\leq& \pi N \sup_{C^\lambda_N} h(z) \leq \pi \frac{N}{(N - 1)^{n+1}} \end{eqnarray}$$ show that for $n > 0$, $\int_{C^\lambda_N} h(z) \,dz \to 0$ as $N \to \infty$, and it follows that $$ \int_{-\infty}^\infty h(t) \,dt = \oint_{\gamma^\lambda_N} h(z) \,dz = \begin{cases} -i2\pi\textrm{Res}(h,-i) &\text{if $\lambda > 0$} \\ 0 &\text{if $\lambda \leq 0$.} \end{cases} $$
So all that remains to do is to compute the residue of $h$ at $z=-i$. Since $h(z)=\frac{p(z)}{q(z)}$ where $p(z)$ is holomoprhic at $z=-i$ and $q(z)$ has a zero of order $n+1$ at $z=-i$, we have $$ \textrm{Res}(h,-i) = \frac{1}{n!}\lim_{z\to -i} \frac{d^n}{dz^n} \left((z + i)^{n+1} h(z)\right) = \frac{1}{n!}p^{(n)}(-i) = \frac{i^{n+1}(-i\lambda)^n}{n!}e^{-i\lambda(-i)} = \frac{i}{n!}\lambda^ne^{-\lambda} \text{.} $$
Therefore, for $n > 0$, $$\begin{eqnarray} \hat f(\lambda) &=& \begin{cases} \frac{1}{\sqrt{2\pi}}\left(i2\pi\right)\left(-\frac{i}{n!}\lambda^ne^{-\lambda}\right) &\text{if $\lambda > 0$} \\ 0 &\text{if $\lambda \leq 0$} \end{cases} \\ &=& \begin{cases} \frac{\sqrt{2\pi}}{n!}\lambda^ne^{-\lambda} &\text{if $\lambda > 0$} \\ 0 &\text{if $\lambda \leq 0$.} \end{cases} \\\end{eqnarray} $$ That case $n = 0$ needs to be handled separately.