I have a question that's been bothering me. If we can convolve any arbitrary input with a system's impulse response to get the system's total response for that input, then if we convolve the impulse response with a unit impulse, won't we just end up with the very impulse response we just convolved with?
I hope my question is clear. Thanks!
On
Yes.
$$ y(t) = g(t) \star u(t) := \underbrace{\int\limits_{-\infty}^{\infty} g(t-\tau)u(\tau) d\tau}_{\mathcal{I}} = \int\limits_0^{t} g(\tau)\delta(t-\tau) d\tau = g(t),$$
where $\int\limits_{-\infty}^{\infty}(.)$ changes to $\int\limits_0^{t}(.)$ due to causality (input can only affect future outputs) and the convention that $(\forall \tau > t) \: u(\tau) \equiv 0$.
Note:
The integral $\mathcal{I}$ is well-defined since, by integration by parts,
$$
\begin{align}
\int_\limits{-\infty}^{\infty}g(\tau)\delta(t-\tau) d\tau & = \left.-\int_\limits{-\infty}^{\infty} g'(\tau)(-\sigma(t-\tau)) d\tau + g(\tau)\sigma(t-\tau)\right\rvert_{\infty}^{-\infty} \\
& = \int_\limits{-\infty}^{t} g'(\tau) d\tau + \lim_{\tau \rightarrow -\infty} g(\tau)\sigma(t - \tau) - \lim_{\tau \rightarrow \infty} g(\tau)\sigma(t - \tau) \\
& = (g(t) - g(0)) + g(0) \\
& = g(t),
\end{align}
$$
where $\sigma(x)$ is the Heaviside function:
$$ \sigma(x) =
\begin{cases}
1 & \text{if } x \ge 0 \\
0 & \text{otherwise}.
\end{cases}
$$
More generally, according to Kaplan's Advanced Calculus (5E), p.529 (7.122) we have that
$g(x) \star \delta^{(k)} (x-c) = \delta^{(k)}(x-c) \star g(x) = g^{(k)}(x-c)$
Yes, if we convolve the impulse response with the unit impulse(i.e $\delta (t)$) we do get the impulse response back. The impulse response is the system's response to an impulse. That's why it is called an impulse response. So if we give $\delta (t)$ as input to a linear time invariant system, the output is the impulse response. Infact, convolution of any function with unit impulse is the function itself.
Linear time invariant(LTI) system is necessary because output is the convolution of input and the impulse response only for LTI systems.