Let $$f(x):=a_1+a_2 \sin(x)+a_3 \sin(x)^2$$ $$g(x):=b_1+b_2 \sin(x)+b_3 \sin(x)^2$$ $$h(x):=c_1+c_2 \sin(x)+c_3 \sin(x)^2$$
then Mathematica shows that the determinant of the matrix
$$\left(\begin{matrix} f(x) & g(x)& h(x) \\ f'(x) & g'(x) & h'(x) \\ f''(x) & g''(x) & h''(x) \end{matrix} \right)$$ is just
$$2 (-a_3 b_2 c_1 + a_2 b_3 c_1 + a_3 b_1 c_2 - a_1 b_3 c_2 - a_2 b_1 c_3 + a_1 b_2 c_3) \cos(x)^2$$
which is a suprisingly simple result given that the expressions from the chain-rule may become rather cumbersome.
I would like to know, is there a smart way to conclude this result without explicitly calculating everything and regrouping terms in order to see the result?
You have $$f(x)=p(\sin x),\ \ g(x)=q(\sin x),\ \ h(x)=r(\sin x)$$ for three degree-two polynomials. Then $$ f'(x)=p'(\sin x)\cos x,\ \ f''(x)=p''(\sin x)\cos x-p'(x)\sin x. $$ So the determinant is \begin{align} \begin{vmatrix} p(\sin x)&q(\sin x)& r(\sin x)\\ p'(\sin x)\cos x&q'(\sin x)\cos x& r'(\sin x)\cos x\\ p''(\sin x)\cos^2 x-p'(\sin x)\sin x&q''(\sin x)\cos^2 x-q'(\sin x)\sin x&r''(\sin x)\cos^2 x-r'(\sin x)\sin x \end{vmatrix}\\ \end{align} We can decompose this as the sum of $$ \begin{vmatrix} p(\sin x)&q(\sin x)& r(\sin x)\\ p'(\sin x)\cos x&q'(\sin x)\cos x& r'(\sin x)\cos x\\ p''(\sin x)\cos^2 x &q''(\sin x)\cos^2 x &r''(\sin x)\cos^2 x \end{vmatrix} = c^3\,\begin{vmatrix} p(\sin x)&q(\sin x)& r(\sin x)\\ p'(\sin x) &q'(\sin x) & r'(\sin x) \\ p''(\sin x) &q''(\sin x) &r''(\sin x) \end{vmatrix}, $$ where $c=\cos x$, and \begin{align} \begin{vmatrix} p(\sin x)&q(\sin x)& r(\sin x)\\ p'(\sin x)\cos x&q'(\sin x)\cos x& r'(\sin x)\cos x\\ -p'(\sin x)\sin x& -q'(\sin x)\sin x& -r'(\sin x)\sin x \end{vmatrix}=0 \end{align} For the determinant with the polynomials, we have $$ \begin{vmatrix} a_1+a_2s+a_3s^2& b_1+b_2s+b_3s^2& c_1+c_2s+c_3s^2\\ a_2+2a_3s&b_2+2b_3s&c_2+2c_3s\\ 2a_3&2b_3&2b_3 \end{vmatrix} $$ Doing row reduction (first row minus $s$ times the second, and second row minus $s$ times the third), this last determinant equals $$ \begin{vmatrix} a_1& b_1& c_1\\ a_2&b_2&c_2\\ 2a_3&2b_3&2b_3 \end{vmatrix} $$ So the determinant is equal to $$ \begin{vmatrix} a_1& b_1& c_1\\ a_2&b_2&c_2\\ 2a_3&2b_3&2b_3 \end{vmatrix} \,\cos^3x $$