For example, my equations are
$\lambda_1=\sqrt{\dfrac{\alpha^2}{2}+\sqrt{k^4+\dfrac{\alpha^4}{4}}}$
$\lambda_2=\sqrt{-\dfrac{\alpha^2}{2}+\sqrt{k^4+\dfrac{\alpha^4}{4}}}$
$\lambda_1\tan\!\big(\!\lambda_2L\!\big)-\lambda_2\tanh\!\big(\!\lambda_1L\!\big)=0$
$\alpha$ is a known quantity whereas the unknowns are $\lambda_1, \lambda_2$ and $k$. Since we have 3 equations and 3 unknowns it is solvable.
But how do we figure out if the equation system has multiple solutions and is of this sort? I'm interested in the first five or ten solutions.
Any help on how to approach is highly appreciated.
In fact, you only have one equation for one unknown $(k)$ since $\lambda_1$ and $\lambda_2$ are explicit functions of $k$ for given values of $\alpha$ and $L$.
The equation being transcendental, numerical methods would be required to find the zeros of $$f(k)=\lambda_1\tan(\lambda_2L)-\lambda_2 \tanh(\lambda_1L)$$
To avoid the discontinuities, I should better consider function $$g(k)=\lambda_1\sin(\lambda_2L)-\lambda_2 \tanh(\lambda_1L)\cos(\lambda_2L)$$
Edit
Consider the case of the $n^{\text{th}}$ root when $n$ is large. The terms in $\alpha$ will be negligible and $$g(k) \simeq k (\sin (k L)-\cos (k L) \tanh (k L))$$ Since the hyperbolic tangent is quickly close to $1$, the solutions are close to the roots of $\tan(kL)=0$ that is to say $k L=m \pi$ that is to say that, at the limit, the distance between two roots will be approximately $\frac \pi L$.
In any manner, the first root of the equation is such that $$ 0 < k_1 < \frac {3.9266} L $$
For illustration, let us take the values $\alpha=0.1714$ and $L=1286$ you gave in comments and compute the first roots $$\left( \begin{array}{ccc} n & k_n & k_{n}-k_{n-1} \\ 1 & 0.02051017643 & \\ 2 & 0.02901018957 & 0.0085000131 \\ 3 & 0.03553910523 & 0.0065289157 \\ 4 & 0.04105159902 & 0.0055124938 \\ 5 & 0.04591800842 & 0.0048664094 \\ 6 & 0.05032863202 & 0.0044106236 \\ 7 & 0.05439677919 & 0.0040681472 \\ 8 & 0.05819647310 & 0.0037996939 \\ 9 & 0.06177928968 & 0.0035828166 \\ 10 & 0.06518289794 & 0.0034036083 \\ 11 & 0.06843583053 & 0.0032529326 \\ 12 & 0.07156033949 & 0.0031245090 \\ 13 & 0.07457417172 & 0.0030138322 \\ 14 & 0.07749177297 & 0.0029176012 \\ 15 & 0.08032508633 & 0.0028333134 \\ 16 & 0.08308412382 & 0.0027590375 \\ 17 & 0.08577737829 & 0.0026932545 \\ 18 & 0.08841212654 & 0.0026347482 \\ 19 & 0.09099466086 & 0.0025825343 \\ 20 & 0.09353046214 & 0.0025358013 \\ \end{array} \right)$$ while $\frac \pi {1286}=0.00244292$.
So, after a very few solutions, a good estimate of the next is just $$k^{(0)}_n=2k_{n-1}-k_{n-2}$$
The proof $$\left( \begin{array}{ccc} n & k^{(0)}_n= 2k_{n-1}-k_{n-2} & k_n \\ 3 & 0.0375102027 & 0.03553910523 \\ 4 & 0.0420680209 & 0.04105159902 \\ 5 & 0.0465640928 & 0.04591800842 \\ 6 & 0.0507844178 & 0.05032863202 \\ 7 & 0.0547392556 & 0.05439677919 \\ 8 & 0.0584649264 & 0.05819647306 \\ 9 & 0.0619961669 & 0.06177928962 \\ 10 & 0.0653621061 & 0.06518289794 \\ 11 & 0.0685865063 & 0.06843583053 \\ 12 & 0.0716887631 & 0.07156033949 \\ 13 & 0.0746848485 & 0.07457417172 \\ 14 & 0.0775880040 & 0.07749177297 \\ 15 & 0.0804093742 & 0.08032508633 \\ 16 & 0.0831583997 & 0.08308412382 \\ 17 & 0.0858431613 & 0.08577737829 \\ 18 & 0.0884706328 & 0.08841212654 \\ 19 & 0.0910468748 & 0.09099466086 \\ 20 & 0.0935771952 & 0.09353046214 \\ \end{array} \right)$$
Update
It even seems that we could generate an estimate for the first solution
$$k^{(0)}_1=a\,\Bigg[\frac{6 (a L-\tanh (a L))}{a L \left(a^2 L^2+3\right)-3 \tanh (a L) \left(a^2 L^2+a L \tanh (a L)+1\right)}\Bigg]^{\frac 14}$$ For the worked example, this gives $0.018110$