Jacobian Calculation with Homographic Coordinates

127 Views Asked by At

I'm trying to work my way through an image processing paper (Creating Full View Panoramic Image Mosaics and Environment Maps) and am stuck on how the Jacobian is calculated, especially terms ($J_{17}, J_{18}, J_{27}, J_{28}$) in Section 3. The math is done in homography coordinates.

$$\boldsymbol{x}=[x,y,1]^T$$ $$D= \begin{bmatrix} d_0 & d_1 & d_2 \\ d_3 & d_4 & d_5 \\ d_6 & d_7 & 0 \end{bmatrix}$$

The tilda $\sim$ indicates that the left/right side are equal to scale - the below matrix is element-wise divided by its final term. $$\boldsymbol{x}''=[x'',y'',1]^T\sim(I+D)x = \begin{bmatrix} (1+d_0)x + d_1y + d_2 \\ d_3x + (1+d_4)y + d_5 \\ d_6x + d_7y + 1 \end{bmatrix} $$ This yields: $$\boldsymbol{x''}= \begin{bmatrix} \frac{(1+d_0)x + d_1y + d_2}{d_6x + d_7y + 1} \\ \frac{d_3x + (1+d_4)y + d_5}{d_6x + d_7y + 1} \\ 1 \end{bmatrix} $$ Now taking the Jacobian w.r.t $d=[d_0,d_1,...,d_8]^T$ $$J_d(x)=\frac{\partial{x''}}{\partial{d}}= \begin{bmatrix} x & y & 1 & 0 & 0 & 0 & -x^2 & -xy \\ 0 & 0 & 0 & x & y & 1 & -xy & -y^2 \end{bmatrix} $$ I can see how the the terms in columns 1-6 could be calculated from $(I+D)x$ but not from $x''$, moreover the last 2 columns make absolutely no sense to me. Also its possible $\boldsymbol{x}''=[x'',y'']$ as it would explain $J$ having 2 vs. 3 rows. The paper's vague there.