My question is:
If $\tan\frac{\pi}{11}\cdot \tan\frac{2\pi}{11}\cdot \tan\frac{3\pi}{11}\cdot \tan\frac{4\pi}{11}\cdot \tan\frac{5\pi}{11} = X$ and $\tan^2\frac{\pi}{11}+\tan^2\frac{2\pi}{11}+\tan^2\frac{3\pi}{11}+\tan^2\frac{4\pi}{11}+\tan^2\frac{5\pi}{11}=Y$ then find $5X^2-Y$.
I couldn't find any way to simplify it. Please help. Thanks.
The main building block of our solution will be the formula \begin{align*}\prod_{k=0}^{N-1}\left(x-e^{\frac{2k i\pi}{N}}\right)=x^N-1.\tag{0} \end{align*} It will be convenient to rewrite (0) for odd $N=2n+1$ in the form \begin{align*} \prod_{k=1}^{n}\left[x^2+1-2x\cos\frac{\pi k}{2n+1}\right]=\frac{x^{2n+1}-1}{x-1}. \tag{1} \end{align*} Replacing therein $x\leftrightarrow -x$ and multiplying the result by (1), we may also write \begin{align*} \prod_{k=1}^{n}\left[\left(x^2-1\right)^2+4x^2\sin^2\frac{\pi k}{2n+1}\right]=\frac{1-x^{4n+2}}{1-x^2}. \tag{2} \end{align*}
Setting in (1) $x=-i$, we get $$\left(2i\right)^n\prod_{k=1}^n\cos\frac{\pi k}{2n+1}=\frac{i^{2n+1}-1}{i-1} \qquad \Longrightarrow\qquad \prod_{k=1}^n2\cos\frac{\pi k}{2n+1}=1.$$
Setting in (2) $x=1$ and computing the corresponding limit on the right, we get $$\prod_{k=1}^n2\sin\frac{\pi k}{2n+1}=\left[\lim_{x\to 1}\frac{1-x^{4n+2}}{1-x^2}\right]^{\frac12}=\sqrt{2n+1}.$$
Combining the two results yields $$\boxed{\quad\prod_{k=1}^n\tan\frac{\pi k}{2n+1}=\sqrt{2n+1}\quad}$$ and to find $X$, it suffices to set $n=5$.
To find $Y$, let us rewrite (1) in the form (set $x=-e^{i\gamma}$) $$\prod_{k=1}^n \left[2\cos\gamma+2\cos\frac{\pi k}{2n+1}\right]=\frac{\cos\frac{\left(2n+1\right)\gamma}{2}}{\cos\frac{\gamma}{2}}$$ Taking the logarithm and differentiating twice with respect to $\gamma$, we find $$\sum_{k=1}^{n}\frac{1}{\left(\cos\gamma+\cos\frac{\pi k}{2n+1}\right)^2} =-\frac{1}{\sin\gamma}\frac{\partial}{\partial \gamma}\left(\frac{1}{\sin\gamma}\frac{\partial}{\partial \gamma}\ln \frac{\cos\frac{\left(2n+1\right)\gamma}{2}}{\cos\frac{\gamma}{2}}\right).\tag{3}$$