Let $\{ X_n \}_{n\geq0}$ be a biased random walk on $\mathbb{Z}$ such that $P(X_{n} = X_{n - 1} + 1) = p$ and $P(X_{n} = X_{n - 1} - 1) = 1 - p$ where $p \notin \{0, 1/2, 1\}$. The initial condition is $X_{0} = 0$.
Fact $(\star)$: $P(X_n = 0 | X_0 = 0) \leq \beta^n$ where $0 < \beta < 1$.
Proof 1 of $(\star)$: We can quickly calculate $$ S(\lambda) = \sum_{n = 0}^{\infty} P(X_{n} = 0 | X_{0} = 0) \lambda^n = \frac{1}{\sqrt{1 - 4 p(1 - p) \lambda^2}} $$ with the usual first step analysis. Identifying the series expansion of $(1 - x^2)^{-1/2}$ gives $P(X_n = 0 | X_0 = 0) \leq [\sqrt{4p(1 - p)}]^n$. Since $p \notin \{0, 1/2, 1\}$, we have $0 < 4p(1 - p) < 1$, so the fact is proved.
Proof 2 of $(\star)$: Via Sirling's formula and the Binomial distribution $$ P(X_{2n} = 0 | X_{0} = 0) = {2n \choose n} p^{n} (1-p)^{2n - n} \sim \frac{4^n}{\sqrt{\pi n}} p^n (1 - p)^n \leq (4p(1- p))^n $$ with again proves the fact.
Question: Is there a probabilistic proof of $(\star)$, i.e. one that does not require such explicit computations, but somehow uses the transience assumption (namely, $p \neq 1/2$) directly and some kind of large deviations estimate?
Suppose WLOG that $\frac{1}{2}\lt p= \frac{1}{2}+\delta$. You have
$S_n = \sum_{i=1}^n Y_i$
for $n\geq 1$ and $S_0 := 0$
where each $Y_i$ is an iid modified Bernouli taking $+1$ with probability $p$ and $-1$ with probability $1-p$
$S_n'$ is the equivalent sum of actual Bernoulis
(i.e. $S_n' = \sum_{i=1}^n \mathbb 1_i$ with $\mathbb 1_i = \frac{1}{2}(Y_i +1)$)
$P\big(S_{n} = 0\big)$
$\leq P\big(S_{n}\leq 0\big) $
$= P\big(S_{n}' \leq \frac{1}{2}n\big)$
$= P\big(S_{n}'\leq (p-\delta) n\big)$
$= P\big(S_{n}' - E\big[S_n'\big]\leq -\delta n\big)$
$\leq \exp\big(\frac{-2(-\delta n)^2}{n}\big)$
$=\exp\big(-2\delta^2 n\big)$
by the Chernoff Bound
This sort of concentration result about the mean works nicely for any walk on the integers that has positive drift when the generators have an MGF that exists in a satisfactory interval (which is always the case for bounded random variables)