My question is in the title. Let's say there are two quaternions $\mathbf q_1$ and $\mathbf q_2$, and both of them reside in the upper hemisphere of $\mathbb S^3$.
Is the result of their multiplication $\mathbf q_3 = \mathbf q_1 * \mathbf q_2$ also guaranteed to lie in the upper hemisphere or are there any corner cases when $\mathbf q_3$ will move to the lower hemisphere?
For example, are there any issues when the quaternions lie on the equator?
No.
Consider scalars $a, b$ and imaginary quaternions $v, w$ (which we can think of as 3D vectors). Then $$ (a+v)(b+w) = [ab - v\cdot w] + [aw + bv + v\times w]. $$ These quaternions $a+v$ and $b+w$ are unit quaterions if $$ a^2 + |v|^2 = 1,\quad b^2 + |w|^2 = 1 $$ so because $a, b > 0$ we can write the scalar term of the product as $$ \sqrt{1-|v|^2}\sqrt{1-|w|^2} - v\cdot w. $$ We can choose any $|v|,|w| < 1$; choose $v = w$ to get $$ 1 - 2|v|^2. $$ Clearly if $|v| \geq 1/\sqrt2$ then our resultant quaterinion is outside the northern hemisphere.
To summarize, we've shown that for any $q$ in the northern hemisphere with scalar part $\geq1/\sqrt2$, it's square $q^2$ is outside the northern hemisphere.
From a higher level perspective, the group of unit quaternions $\mathbb S^3$ naturally gives us a double cover of $\mathrm{SO}(3)$, explicitly $$ \rho : \mathbb S^3 \to \mathrm{SO}(3),\quad \rho_q(v) = qv\bar q $$ where $v \in \mathbb R^3$ is thought of as an imaginary quaternion and $\bar q = q^{-1}$ is the conjugate. We have $\ker\rho = \{\pm1\}$, meaning that both $q$ and $-q$ represent the same rotation. If the upper hemisphere $\mathbb S^3_+$ were closed under under multiplication then it would be a subgroup of $\mathbb S^3$ which is isomorphic to the "subgroup" $$ \mathrm{SO}(3)\setminus\{180^\circ\text{ rotations}\} $$ but the set of all $180^\circ$ rotations is not a subgroup. Equivalently, the equator of $\mathbb S^3$ is not a subgroup: if $v,w$ are imaginary then $$ vw = -v\cdot w + v\times w $$ which need not be imaginary.