For (a), what does it mean by three vectors given are closed under Lie bracket? Does it mean that $[X,Y]=Z$ or something like this? But when I compute $[X,Y]$ , it is something messy and doesn't look like any one of $X,Y,Z\dots$Could anyone please explain?
If $\frak{g}$ is your Lie Algebra, then showing closure here means exactly what you think, that is for any $X,Y \in \frak{g}$ we must have $[X,Y] \in \frak{g}$
This is a bit long, but you have to use the bilinearity of the bracket
\begin{align*} [X,Y ] &= [-\sin\phi \partial_\theta - \cos\phi\cot\theta \partial_\phi, \cos \phi \partial_\theta - \cot\theta \sin\phi \partial_\phi] \\ &= [-\sin\phi \partial_\theta, \cos \phi \partial_\theta - \cot\theta \sin\phi \partial_\phi] + [- \cos\phi\cot\theta \partial_\phi, \cos \phi \partial_\theta - \cot\theta \sin\phi \partial_\phi]\\ &=[-\sin\phi \partial_\theta, \cos \phi \partial_\theta]+ [ -\sin\phi \partial_\theta, - \cot\theta \sin\phi \partial_\phi] + [- \cos\phi\cot\theta \partial_\phi, \cos \phi \partial_\theta] + [- \cos\phi\cot\theta \partial_\phi, - \cot\theta \sin\phi \partial_\phi]\\ &=-\frac{-\sin^2\phi}{\sin^2\theta}\partial_\phi - \sin\phi\cos\phi\cot\theta \partial_\theta + \sin\phi\cos\phi\cot\theta \partial_\theta - \frac{\cos^2\phi}{\sin^2\theta} \partial_\phi + \cot^2\theta \partial_\phi\\ &=\partial_\phi\\ &=Z\\ [X,Z] &= [-\sin\phi \partial_\theta - \cos\phi\cot\theta \partial_\phi, \partial_\phi] \\ &=[-\sin\phi \partial_\theta,\partial_\phi] + [- \cos\phi\cot\theta \partial_\phi, \partial_\phi]\\ &=\cos\phi \partial_\phi - \sin\phi \cot\theta \partial_\phi \\ &=Y\\ [Y,Z ] &= [ \cos \phi \partial_\theta - \cot\theta \sin\phi \partial_\phi, \partial_\phi] \\ &=[\cos \phi \partial_\theta,\partial_\phi] + [- \cot\theta \sin\phi \partial_\phi, \partial_\phi]\\ &=-\sin\phi \partial_\theta - \cos\phi\cot\theta \partial_\phi\\ &=X \end{align*}