This question was from an exam taken in January 2022 on a course on introductory multivariable calculus and was worded exactly as follows:
"For a general surface $S$ bounded by a closed curve $C$ show using Stokes theorem that for a vector field, $\mathbf{F}(\mathbf{r})$
$$\int_S\nabla\times(\mathbf{F}\times\mathrm{d}\mathbf{S})=\alpha\int_C\mathbf{F}\times\mathrm{d}\mathbf{r}$$
and identify the constant $\alpha$."
The "$\mathrm{d}\mathbf{S}$" is used to denote a surface integral and the "$\mathrm{d}\mathbf{r}$" to denote a line integral.
I have asked a couple of mathematicians who work in applied mathematics and they have not been able to prove this either. The LHS is apparently the area of concern - having "$\nabla\times(\mathbf{F}\times\mathrm{d}\mathbf{S})$" seems to be what's throwing people off.
I have been told that the answer should result in a vector even though, in the course, vector-valued integrals were never defined and so the notions of "$\times\mathrm{d}\mathbf{S}$" and "$\times\mathrm{d}\mathbf{r}$" were also not defined, so while I personally believe this question was unfair, I'm asking more about whether or not it's possible.
If this is a mistake, could you explain why? And if it isn't, can you prove it?
It resembles the following vector variant of Stokes' theorem $$\iint_S(\mathbf{n}\times\nabla)\times \mathbf{F}\,dS=-\int_C\mathbf{F}\times\mathrm{d}\mathbf{r}.$$ Proof. Note that $$ (\mathbf{n}\times\nabla)\times \mathbf{F}= \left| \begin{array}{ccc} \mathbf{i} & \mathbf{j} & \mathbf{k} \\ n_2\frac{\partial}{\partial z}-n_3\frac{\partial}{\partial y} & n_3\frac{\partial}{\partial x}-n_1\frac{\partial}{\partial z} & n_1\frac{\partial}{\partial y}-n_2\frac{\partial}{\partial x} \\ F_1 & F_2 & F_3 \end{array} \right| $$ and therefore, by comparing the $x$ components of both sides we find $$ \begin{align}\left(\iint_S(\mathbf{n}\times\nabla)\times \mathbf{F}\,dS\right)_x &=\iint_S \left(n_3\frac{\partial F_3}{\partial x}-n_1\frac{\partial F_3}{\partial z} -n_1\frac{\partial F_2}{\partial y}+n_2\frac{\partial F_2}{\partial x}\right)dS\\ &=\iint_S \left(-\frac{\partial F_3}{\partial z}-\frac{\partial F_2}{\partial y},\frac{\partial F_2}{\partial x},\frac{\partial F_3}{\partial x}\right)\cdot d\mathbf{S}\\ &=\iint_S \left(\nabla \times\left(0,F_3,-F_2\right)\right)\cdot d\mathbf{S}\quad \text{(Stokes' Theorem)}\\ &=\int_C \left(0,F_3,-F_2\right)\cdot d\mathbf{r}\\ &=\left(-\int_C\mathbf{F}\times\mathrm{d}\mathbf{r}\right)_x. \end{align}$$ Similarly we can verfy that the equality holds also for $y$ and $z$ components.
P.S. I wonder if one can play (at least in a formal way) with the cross-product properties in order to obtain $\nabla\times(\mathbf{F}\times d\mathbf{S})$: $$\nabla\times(\mathbf{F}\times d\mathbf{S})= (\nabla\times \mathbf{F})\times d\mathbf{S}+(d\mathbf{S}\times \nabla)\times \mathbf{F}= (\nabla\times \mathbf{F})\times d\mathbf{S}+(\mathbf{n}\times \nabla)\times \mathbf{F}dS.$$ Note that the term $(\nabla\times \mathbf{F})\times d\mathbf{S}$ is related to a variant of the divergence theorem.