I know that section of $TM$ is a vector field on manifold. And I understand that we need a connection to differentiate vectors that live in two different spaces.
But still couldn’t grasp the idea how defining the connection that way will allow us to somehow compare or connect those spaces.
Like in affine connection: It’s a bilinear map that sends two vector fields to another one.
But how this connection help us to compare the vector space at point $p$ to vector space of point $q$?
Let $\nabla$ be an affine connection on a manifold $M$. As you point out, this is just a operator $\nabla:\mathfrak{X}M\times\mathfrak{X}M\to\mathfrak{X}M$ satisfying certain axioms.
The the way this "connects" tangent spaces is allowing us to associate paths between points with isomorphisms between the tangent spaces above those points. The path part is often crucial, the resulting isomorphism often depends on the choice of path.
Let $\gamma:[0,1]\to M$ be a smooth, rectifiable path with $\gamma(0)=p$ and $\gamma(1)=q$. We can define a vector fields along $\gamma$ as a maps $V:[0,1]\to TM$ such that $V(t)\in T_{\gamma(t)}M$. A connection allows us to differentiate these vector fields with respect to time, producing another vector field along $\gamma$: $$ \dot{V}=\nabla_{\dot{\gamma}}V $$ Where on the right side we work locally and chose arbitrary extensions of $\dot{\gamma}$ and $V$.
Using this, we can define a "parallel transportation" map $P_\gamma:T_pM\to T_qM$. Define $P_\gamma(v)=V(1)$, where $V$ is the solution of the initial value problem $\dot{V}=0$, $V(0)=v$. One can verify that this operator is a linear isomorphism, and has lots of other convenient properties.
One could define a connection to be a collection of parallel transportation operators satisfying certain axioms, but in practice the "infinitesimal" version $\nabla$ is much, much easier to work with.