I have an equation that allows me to draw a regular n-gon.
$$R(\theta) = \dfrac{r}{\cos\left(\theta-\dfrac{2\pi}{N}\left\lfloor\dfrac{N\theta+\pi}{2\pi}\right\rfloor\right)}, $$
where r is the radius of the inscribed circle and N is the number of corners of a polygon.
I have a problem to change this equation to make a polygon with a corner rounded with some radius $r_2$. Anyone has any idea how to approach it?
I attach links to images of what I mean as well as link to wolfram with the formula above.
Hexagon:
Hexagon with rounded corners:
A heuristic attempt (that is it won't be quite a circle at the corner, and the radius might take some interpretation) would be to replace your existing $R(\theta)$ with a smoothed version $S$, such that $$S(\theta)\approx {1\over 2n+1}\sum _{i=-n}^nS(\theta + i \Delta)$$ for some small value of $\theta$, roughly $r_2 \propto \Delta$