Vector reflection with limited angle

Question

I am struggling with following challenge in my free time programming project $-$ how is it possible to make reflection vector that reflects along normal with angle that is not larger than some $\alpha$?

I have already seen classical reflection formula $ r = d - 2 (d \cdot n) n $, which unfortunately does not provide answer to my problem.

I tried to fiddle around with JS atan2() function, as described here, but it didn't work for all cases and I would appreciate some elegant general solution instead of patching special cases.

Thank you in advance.

Answer 1

I'm assuming $n$ is a unit vector.

1. If $d \cdot n \le 0$: ERROR

2. $\alpha = \arccos(d \cdot n)$

3. If $\alpha < \alpha_\text{max}$, return $r = d - 2(d \cdot n) n$

4. Else, $v = d - (d \cdot n) n$ is a vector perpendicular to $n$, on the $d$ side of the $nd$ plane.

5. Let $w = \frac{1}{\|v\|} v$.

6. Set $r = \cos(\alpha_\text{max}) n - \sin(\alpha_\text{max}) w$; return $r$.

What I've done is test as @mvw suggests, and when the test fails, I've computed a vector in the $dn$-plane whose angle is the max angle, and then computed its reflection.

Answer 2

Maybe I fail to see your problem but just use the scalar product $$ u \cdot n = \lVert u \rVert \cos \alpha $$

So test for $$ \frac{\sum_i u_i n_i}{\sqrt{\sum_i u_i^2}} = \frac{u \cdot n}{\lVert u \rVert} \ge \cos \alpha_{\tiny \mbox{max}} $$

to see if $\alpha \le \alpha_{\tiny \mbox{max}}$ for the tested vector $u$.