I am trying a derivation of the polar form of an ellipse using vector notation. Beginning with a definition of an ellipse as the set of points in $\vec{R}^2$ for which the sum of the distances from two points is constant, I have $|\vec{r_1}|+|\vec{r_2}| = c$
Thus, $|\vec{r_1}|^2+|\vec{r_1}||\vec{r_2}|=c|\vec{r_1}|$
ellipse diagram, Inductiveload on Wikimedia
Choosing a coordinate system (similar to the one in the diagram), such that $\vec{r_1}=\vec{r}=x\vec{i}+y\vec{j}$, and $\vec{r_2}=\vec{F_2 P}=(x-a)\vec{i}+y\vec{j}$
Noticing that $|\vec{r_1}||\vec{r_2}|=\frac{\vec{r_1}\cdot\vec{r_2}}{\cos(\theta)}=\frac{x^2-ax+y^2}{\cos(\theta)}=\frac{r^2-ax}{\cos(\theta)}$
and using $x=r\cos(\theta)$, we get
$r^2+\frac{r^2}{\cos(\theta)}-ar=cr$, thus I get a polar equation for an ellipse of $r=\frac{a+c}{1+\sec(\theta)}$
whereas I expect $r=\frac{a(1-e^2)}{1+e\cos(\theta)}$. Please tell me where this argument takes the wrong turn!
I'm not sure yet what exactly you are doing yet, but here at least is an approach which works.
I think the issue is that you claim that $x=r \cos(\theta)$ and $\vec{r}_1 \cdot \vec{r}_2=r_1 r_2 \cos(\theta)$. It is not obvious to me that these angles would be the same.
Label the foci as $F$ and $F'$. Let $\vec{r}$ be the vector which points from $F$ to a point $p$ ont he ellipse. Let $\vec{r}'$ be the vector which points from $F'$ to $p$. Let $d$ be the distance between $F$ and $F'$. Let the $a$ represent the semi-major axis of this ellipse. By considering one of the points of the ellipse which lies on the major axis we can conclude that,
$$ r+r'=2a.$$
If $\theta$ is the angle between $\vec{r}$ and the line connecting $F$ to $F$' then we can use the law of cosines to write,
$$ r^2 + d^2+2rd\cos(\theta) = r'^2$$
$$ r^2 + d^2+2rd\cos(\theta) = (r-2a)^2$$
$$ \color{blue}{ r^2} + d^2+2rd\cos(\theta) = \color{blue}{ r^2}-4ar+4a^2$$
$$ d^2+2rd\cos(\theta) = -4ar+4a^2$$
$$ 2rd\cos(\theta) +4ar= 4a^2-d^2$$
$$ 2r(d\cos(\theta) +2a)= 4a^2-d^2$$
$$ r= \frac{a(1-e^2)}{e\cos(\theta) +1}$$