I want to show that $ (e^{\alpha X_t} \int^ t_ 0 e ^{-\alpha X_u}du, t \geq 0) $ is a Markov process whereas $(\int^ t_ 0 e ^{-\alpha X_u}du, t\geq 0)$ is not. Here $X_t$ is a Levy Process and nothing has been said about $\alpha$.
I could express $$Y_t = e^{\alpha X_t} \int^ t_ 0 e ^{-\alpha X_u}du = e^{\alpha (X_t-X_s)}Y_s+ e^{\alpha (X_t-X_s)}\int^ t_ s e ^{-\alpha (X_u-X_s)}du $$ Then how does one use the independence to show the markov process and for showing that other one is not a markov process.
I think there is a typo in the identity which you mentioned in your question; it should read
$$Y_t = e^{\alpha(X_t-X_s)} Y_s + e^{\alpha(X_t-X_s)} \int_s^t e^{-\alpha(X_u-X_s)} \, du.\tag{1}$$
In order to show that $(Y_t)_{t \geq 0}$ is a Markov process, we can use the following statement
Denote by $\mathcal{F}^X$ the canonical filtration of the Lévy process $(X_t)_{t \geq 0}$. Using the above lemma and $(1)$, we find from the independence of the increments of $(X_t)_{t \geq 0}$ that for $s \leq t$
$$\mathbb{E}(f(Y_t) \mid \mathcal{F}_s^X) = g(Y_s) \tag{2} $$
for $$g(y) = \mathbb{E}\left[ f \left( e^{\alpha (X_t-X_s)} y + e^{\alpha(X_t-X_s)} \int_s^t e^{-\alpha (X_u-X_s)} \, ds \right) \right] .$$
Since the canonical filtration $\mathcal{F}^Y$ of $(Y_t)_{t \geq 0}$ satisfies $\mathcal{F}_t^Y \subseteq \mathcal{F}_t^X$ for all $t$, the tower property gives
$$\mathbb{E}(f(Y_t) \mid \mathcal{F}_s^Y) = g(Y_s).$$
In particular,
$$\mathbb{E}(f(Y_t) \mid \mathcal{F}_s^Y) = \mathbb{E}(f(Y_t) \mid Y_s),$$
and this shows that $(Y_t)_{t \geq 0}$ is a Markov process.
In order to show that
$$Z_t := \int_0^t e^{-\alpha X_u} \, du$$
is not a Markov process, you actually have to assume that $(X_t)_{t \geq 0}$ is a non-trivial Lévy process. Using a very similar reasoning as above, we find
$$\mathbb{E}(f(Z_t) \mid \mathcal{F}_s^X) = Z_s + c e^{-\alpha X_s} $$
where
$$c := \mathbb{E} \left( \int_s^t e^{-\alpha (X_u-X_s)} \, du \right).$$
Since $e^{-\alpha X_s}$ is $\mathcal{F}_s^Z$-measurable, the tower property gives
$$\mathbb{E}(f(Z_t) \mid \mathcal{F}_s^Z) =Z_s +c e^{-\alpha X_s}. \tag{3}$$
It is intuitively clear that there cannot exist a function $h$ such that
$$e^{-\alpha X_s}= h(Z_s), \tag{4}$$
and therefore $(3)$ shows that $(Z_t)_{t \geq 0}$ is not Markovian. (Note that $(4)$ would mean that for $F(s) := e^{-\alpha X_s}$ you can determine the value $F(s)$ if you know the value of the integral $\int_0^s F(u) \, du$... this is pretty absurd.)