I want to find the $p$-value (manually) of the following Hypothesis testing.
$$H_0:\mu\leq 3 \quad \text{vs} \quad H_1:\mu >3$$
The main thing I know is that
$$P(\mathrm{Re}\,j \mid \mu \leq 3)=P(X\geq 3 \mid \mu \leq 3)= e^{-1} \approx0.36$$
Can I use the $z$ value and use the formula probability of $z$? Or from where can I start?
Your null hypothesis is that your exponential distribution has a rate $\mu$ which is $\leq 3$. Your alternate hypothesis is that $\mu \geq 3$. Now, you get some observation, $x$. What is the probability that this sample is consistent with the null-hypothesis? Meaning, what is the probability that the null hypothesis would generate a sample $\geq x$? Conditional on $\mu$, this is simply $e^{-\mu x}$. Since your null hypothesis is that $\mu \leq 3$, you integrate over it to get the p-value:
$$p = \int\limits_0^3 e^{-\mu x}d \mu = \frac{1-e^{-3x}}{x}$$
Now, you can set a threshold on this p-value and reject the null hypothesis if it is lower than your threshold.