Two line bundles $L,L'$ on $X/k$ are called algebraically equivalent if $L = M|_{X\times t}$ and $ L'=M|_{X\times t'} $ for some line bundle $M$ on $X \times T$ with $T$ smooth and irreducible and two closed points $t,t' \in T$.
Why is this relation transitive? My idea was to use $T \times T'$ as a parameter space.
Yes, your idea is the right one. Here are the details.
Let's use notation like $M_a$ to abbreviate $M_{\mid X \times a}$.
Say that $L_1$, $L_2$ and $L_3$ are bundles on $X$, and there are bundles $M$ and $M'$ on $X \times T$ and $X \times T'$ respectively and points $a,b \in T$, $c,d \in T'$ such that
$$M_a = L_1 \\ M_b = L_2 \\ M'_c = L_2 \\ M'_d = L_3$$
We can twist $M'$ by $\pi^* L_2^{-1}$ to get a new bundle $N'$ such that
$$N'_c = \mathcal{O} \\ N'_d = L_2^{-1} \otimes L_3.$$
Now let $\pi_1$ and $\pi_2$ be the projections from $X \times T \times T'$ to $X \times T$ and $X \times T'$ respectively. Define
$$P = \pi_1^* M \otimes \pi_2^* N'.$$
Then
$$P_{(a,c)} = M_a \otimes \mathcal{O} = M_a = L_1 \\ P_{(b,d)} = M_b \otimes N'_d = L_2 \otimes L_2^{-1} \otimes L_3 = L_3.$$