I am very new to Lie groups and manifolds. In my self study, many times I have come across "differential (or derivative) of the left translation" (for example, here). I don't fully understand what is meant by differential/derivative here. I would appreciate any attempt to explain this concept both formally and intuitively.
For example, what does the differential/derivative here mean and how is it defined? Does the definition require charts in general (I have seen how smooth maps are defined between two manifolds)? Would it have a more convenient form in the special case of matrix Lie groups? What does it do?
Part of the answer given to this question is definitely relevant. But I am not sure where does $$dL_{g}(v) = \frac{d}{dt}\bigg|_{t=0} L_{g}\exp(tv)$$ come from? (I know that $\exp(tv)$ is a curve on $G$ that passes through the identity element at $t=0$ and $v$ is its tangent vector at the identity). Is this the definition of differential of $L_{g}$? Is there a backstory?
To make this question more concrete, let me set up a standard notation. Let $G$ be a matrix Lie group. $L_g : G \to G$ for any $g \in G$ maps an element of $G \ni p$ to $ L_g(p) := g p$. Then, the differential of $L_g$, denoted by $dL_g$ is a mapping from a tangent space at any $p \in G$ to the tangent space at $L_g(p)$.
As you note, $dL_g:T_hG\to T_{gh}G$ (or the differential of any smooth map) is already a well-defined linear a map between tangent spaces without further reference to the group structure. In terms of velocities of curves, it is defined by: $$ dL_g\left(\left.\frac{d}{dt}\gamma(t)\right|_{t=t_0}\right)=\left.\frac{d}{dt}\left(L_g(\gamma(t))\right)\right|_{t=t_0} $$ Equivalently, in terms of derivations: $$ [dL_g(V_h)](f)=V_h(f\circ L_g) $$ These two definitions of tangent vectors are equivalent: we may equate every velocity with a derivation given by $$ \left(\left.\frac{d}{dt}\gamma(t)\right|_{t=t_0}\right)(f)=\left.\frac{d}{dt}\left(f(\gamma(t))\right)\right|_{t=t_0} $$ If this isn't already familiar, it might be worth checking that the above definitions of the differential agree.
Understanding its relationship with the exponential map will require establishing a few facts about the Lie algebras and the exponential map.
The Lie algebra $\text{Lie}(G)$, can be identified with two vector spaces:
To go from the first to the second, we can just evaluate the vector field at $e$. To go from a tangent vector $v\in T_e M$ to a vector field $V$, we let $V_e=v$, and the value of $V$ at any other point $g$ must be $V_g=dL_gv$ by left invariance. Because of this, we often use the same symbol to refer to both a tangent vectors at $e$ and the corresponding left-invariant vector field. Instead, I'll treat elements $V\in\text{Lie}(G)$ as left-invariant vector fields and use subscripts $V_e\in T_eM$ to refer to their values at particular points.
The exponential map $\exp:\text{Lie}(G)\to G$ is given by $$ \exp(V)=\gamma_V(1) $$ Where $\gamma_V$ is the unique integral curve of the left-invariant vector field $V$ satisfying $\gamma_V(0)=e$ and $\frac{d}{dt}\gamma(t)=V_{\gamma(t)}$ (often called the one parameter subgroup generated by $V$). A consequence of this is that $\gamma_V(t)=\exp(tV)$.
Let $U_e\in T_e G$, and $U$ be the corresponding left-invariant vector field. We can use definition of the differential, and the fact that $\left.\frac{d}{dt}\exp(tU)\right|_{t=0}=U_e$ to arrive at the formula you give: $$ \left.\frac{d}{dt}\left[L_g\exp(tU)\right]\right|_{t=0}=dL_g\left.\frac{d}{dt}\exp(tU)\right|_{t=0}=dL_gU_e $$ We see, somewhat more explicitly, this is only an expression for how $dL_g$ acts on tangent vectors at the identity.