In my lecture notes, geodesic is defined to be a curve that is parametrized proportional to arc length and an extremal to the length functional: $$l(\alpha)=\int_a^b |\alpha'(t)|dt \quad (1)$$ In the section Riemannian geometry in the Wikipedia page Geodesic, it said that geodesic can also be defined as an extremal to the energy functional: $$E(\alpha)=\frac{1}{2}\int_a^b |\alpha'(t)|^2dt\quad (2)$$ Due to the fact that $(l(\alpha))^2 \leq (b-a)E(\alpha)$ with equality iff $\alpha$ is parametrized proportional to arc length, it is easy to show that if $\alpha$ is a geodesic as defined in $(1)$, then it is a geodesic as defined in $(2).$
But how can I show that if $\alpha$ is an extremal to the energy functional, then it is parametrized by arc length? Or is it a condition that is required for these two definitions to be equivalent?
If $\gamma$ is an extremum of the energy functional $E$, then $\gamma$ is parameterized proportional to arc length.
Unfortunately, I don't see a "tricky" proof of this fact, and so I'm afraid that the explanation I'm going to try to give may use too much machinery (given that you are learning the definition of a geodesic). At minimum you will need to be familiar with the idea of a covariant derivative along a curve (see do Carmo, section 2.2, proposition 2.2).
To (hopefully) review, given a curve $\gamma$ in a Riemannian manifold, there's a well-defined way to take the "derivative" of a vector field $V$ defined along $\gamma$ (i.e., for each $t$ in the domain of $\gamma$ we have a vector $V(t) \in T_{\gamma(t)} M$, and these vectors should vary smoothly in $t$). This derivative we denote by $D_t V$ or $\nabla_{\gamma'(t)} V$ depending on whether $\gamma$ is clear from context.
In order to prove that extrema of the energy functional are parameterized proportional to arc length, the standard thing to do is to just use the first variation formula, a fundamental result in Riemannian Geometry. I'll outline the setup and result; this is done more carefully in do Carmo, section 9.2.
Let $\Gamma: (-\epsilon, \epsilon) \times (a, b) \to M$ be a smooth map. We write $\gamma_s(t)$ for $\Gamma(s,t)$, and we think of $\Gamma$ as a "variation" of paths around $\gamma_0$. We make the further restriction that $\Gamma$ fixes the endpoints of $\gamma$; that is, $\gamma_s(a) = \gamma(a)$ and $\gamma_s(b) = \gamma(b)$ for all $s$.
The statement that $\gamma$ is an extremum for the energy functional is exactly the statement that for all such variations $\gamma_s$ with $\gamma_0 = \gamma$, we have $\left. \frac{d}{ds}\right|_{s=0} E(\gamma_s) = 0$. It's therefore tempting (and useful) to explicitly compute the derivative $\left. \frac{d}{ds}\right|_{s=0} E(\gamma_s)$, for arbitary curves $\gamma$ and arbitrary variations $\gamma_s$. We obtain the following:
First variation formula, simple form. If $\Gamma$ is a smooth variation fixing the endpoints of $\gamma$, then $$ \left. \frac{d}{ds} \right|_{s=0} E(\gamma_s) = -2 \int_a^b \langle \partial_s \Gamma, D_t \gamma_s' \rangle dt. $$ Here $D_t \gamma'_s$ is the covariant derivative of $\gamma'_s$ along the curve $t \mapsto \gamma_s(t)$ (which might sometimes be written $\nabla_{\gamma'_s} \gamma'_s$).
Proving this isn't particularly difficult once the machinery is familiar. The point now is that since $\Gamma$ is (relatively) arbitrary, it follows that any extremum $\gamma$ of the energy functional must satisfy $D_t \gamma'(t) = 0$.
I'll remark that the equation $D_t \gamma'(t) = 0$ is often called the geodesic equation, and geodesics are often defined to be those curves that satisfy the geodesic equation. (One must then use an argument like this one to show that such curves minimize the length functional.)
Finally, it follows from $D_t \gamma'(t) = 0$ that $||\gamma'(t)||$ is constant in $t$, since $$ \frac{d}{dt} ||\gamma'(t)||^2 = 2 \langle D_t \gamma'(t), \gamma'(t) \rangle = 0, $$ and of course it follows from this that $\gamma$ is parameterized proportional to arc length, since the arc length of $\gamma$ from $t_1$ to $t_2$ is just $$ \int_{t_1}^{t_2} || \gamma'(t) || dt. $$