In electromagnetism, the electromagnetic field tensor can be expressed as $$F_{\mu \nu} = \partial_\mu A_\nu - \partial_\nu A_\mu.$$
If we let $A= A_\mu dx^\mu$, since $F= \frac{1}{2} F_{\mu \nu} dx^\mu \wedge dx^\nu$, we get that $$F = dA.$$
I just wanted to check that I'd gotten my terminology right: is $A_\mu$ the gauge field, and then $A$ the connection, and $F$ the curvature? I am questioning myself as I thought that $A_\mu$ was called the vector potential in $B = \nabla \times A$. Also, is it sufficient to say that $A$ is the connection if this is right, or is it the connection over a principle fiber bundle etc. I don't know if I'm being ambiguous, and need to be more precise since I am considering these in a particular context, namely electromagnetism.
In mathematical terminology, the word "connection" sometimes refers to the differential operator $\nabla:\Gamma(E)\to \Omega^1_M(E) = \Gamma(T^*M\otimes E)$. But if you choose a reference connection $\nabla_0$, or perhaps if there already is a preferred reference connection (like the trivial one $\partial$ in a trivial bundle or a local trivialization of a bundle), then you can write any connection as $$\nabla = \nabla_0 + A$$ for some $A\in \Omega^1_M(\text{End}(E))= \Omega^1_M(E\otimes E^*)$. For this reason we sometimes write $\nabla_A$. And since, once the reference is chosen, $A$ determines $\nabla_A$ completely by the above formula, the word "connection" sometimes refers to $A$ as well!
Then there is the picture in the principal bundle (say with structure group $G$). A "connection" can be defined as a certain kind of 1-form $A\in \Omega^1_P(\mathfrak g)$ with values in the lie algebra $\mathfrak g$ of the lie group $G$. This is equivalent to the stuff on the vector bundle $E$ in some sense, and the $A$ here on $P$ has a direct relationship with the $A$ defined before.
The short answer is that "connection" refers to many equivalent objects.
In physics terminology, the $A$ is the vector potential for the electromagnetic field. And in the gauge theory formulation of electromagnetism, $A$ is a connection on a bundle as above.