I've come across the word "punctured elliptic curve" here and there, but none of the basic texts on the topic (Husemoller, Silverman) define or mention it.
What point is removed from the curve (the one at infinity I assume?) and what makes punctured elliptic curve interesting?
Punctured curves can be used to construct extensions of Galois representations. This an example of Deligne's philosophy of mixed motives. A pure motive is a motive attached to a smooth projective variety; according to Deligne not necessarily projective smooth varieties should live in a suitable category of extensions of pure motives. I will explain how a punctured elliptic curve gives rise to a three dimensional Galois representation, which is a (generally nontrivial) extension of the trivial representation by the Tate module of $E$.
To explain, let $E/K$ be an elliptic curve and $p$ a prime number not equal to $\text{char} K$.
Then we can construct a $2$-dimensional $p$-adic Galois representation $$V_p(E) := T_p(E) \otimes_{\mathbb Z_p} \mathbb Q_p,$$
where $T_p(E)$ is the $p$-adic Tate module of $E$, a free $\mathbb Z_p$-module of rank $2$ on which $G_K=\text{Gal}(\overline K /K)$ acts continuously.
Now, write $H$ for geometric $p$-adic cohomology (so that $H(X) = H_{ét}(X_{\overline K}, \mathbb Q_p)$, etc). Consider the open curve $E_P = E-{\{P, \mathcal O\}}$, where $\mathcal O$ is the origin on $E$. One has the long exact sequence of the pair $(E, E_P)$ which takes the form
$$0 \to H^0_{\{P, \mathcal O\}}(E) \to H^0(E) \to H^0(E_P) \to H^1_{\{P, \mathcal O\}}(E) \to H^1(E) \to H^1(E_P) \to \dots$$
where $H^i_{\{P, \mathcal O\}}$ denotes cohomology with supports on ${\{P, \mathcal O\}}$. Theorems on the fundamental class show that $H^2_{\{P, \mathcal O\}}(E) = \mathbb Q_p(-1)\oplus \mathbb Q_p(-1)$ (one summand for each of $P, \mathcal O$), and $H^i_{\{P, \mathcal O\}}(E) = 0$ for $i \neq 2$. We also have $H^2(E) = \mathbb Q_p(-1)$. Therefore we have a sequence
$$0 \to H^1(E) \to H^1(E_P) \to \mathbb Q_p(-1)\oplus \mathbb Q_p(-1) \to \mathbb Q_p(-1) $$
and the last map is $(x,y) \mapsto x+y$. By choosing the element $(-1,1)$ in the kernel of the last map, we deduce an exact sequence
$$0 \to H^1(E) \to H^1(E_P) \to \mathbb Q_p(-1) \to 0$$
or
$$0 \to V_p(E) \to H^1(E_P)(1) \to \mathbb Q_p \to 0$$
thus we have obtained an extension $\eta_P \in \text{Ext}^1(\mathbb Q_p, V_p(E)) = H^1(\mathbb Q, V_p(E))$. This gives a map
$$E(K) \to H^1(\mathbb Q, V_p(E))$$
which is nothing but the map of Kummer theory.
Remark: The Hodge-theoretic analogue of the Galois cohomology group $\text{Ext}^1(\mathbb Q_p, V_p(E))$ is $$\text{Ext}_{MHS}^1(\mathbb Z, H^1(E(\mathbb C), \mathbb Z)(1))$$ where the Ext is now taken in the category of mixed Hodge structures, and $H^1$ is Betti cohomology. One can show that there is a canonical isomorphism
$$\text{Ext}_{MHS}^1(\mathbb Z, H^1(E(\mathbb C), \mathbb Z)(1)) = E(\mathbb C),$$
and this suggest that the Galois cohomology group $H^1(\mathbb Q, T_p(E))$ is really an embodiment of the Mordell-Weil group, and explains why it is so useful in understanding the Mordell-Weil group.