I came across an interesting problem in my research (not a mathematician). Here it goes:
Suppose, there is a 2D lattice $\Lambda$ in the X-Y plane with basis vectors $\vec{a}$ and $\vec{b}$, which are not orthogonal to each other. I want to find a rectangle in this lattice, whose area is the minimum of all possible rectangles. So far, this is what I have:
Consider two vectors in the lattice:
$\vec{v_1} = m_1 \vec{a} + n_1 \vec{b}$
$\vec{v_2} = m_2 \vec{a} + n_2 \vec{b}$
If they are orthogonal, then $\vec{v}_1 \cdot \vec{v}_2 = m_1 m_2 \left| \vec{a} \right|^2 + n_1 n_2 \left| \vec{b} \right|^2 + (m_1 n_2 + n_2 m_1) \vec{a} \cdot \vec{b} =0 $
The area subtended by the two vectors is the norm of the cross product, i.e. $\left| \vec{v}_1 \times \vec{v}_2 \right| = \left| \vec{a}_1 \times \vec{b}_2 \right| \left|m_1n_2 - m_2n_1 \right|$
Therefore, the area is minimized if $\left|m_1n_2 - m_2n_1 \right|$ is minimized and greater than zero. And the constraint is:
$m_1 m_2 \left| \vec{a} \right|^2 + n_1 n_2 \left| \vec{b} \right|^2 + (m_1 n_2 + n_2 m_1) \vec{a} \cdot \vec{b} =0$
I am stuck after this! May be some kind of optimization with a constraint using Lagrange multipliers? But there are too many variables and $m_1, m_2, n_1, n_2 \in \mathbb{Z}$. Any suggestions will be greatly appreciated. Thank you for your time.
There seems to be an answer if the lattice can be rotated and scaled so that all inner products of lattice vectors are integers (positive definite..) and there is no common integer factor of the entries in the gram matrix; $\gcd(A,B,C)=1.$ We also may demand that $A$ be the squared length of the shortest nonzero lattice vector.
http://en.wikipedia.org/wiki/Unimodular_lattice#Definitions
http://en.wikipedia.org/wiki/Gramian_matrix#Gram_determinant
As far as i can tell, in this case, the best answer is a non-primitive lattice, given by identity $$ \left( \begin{array}{rr} 1 & 0 \\ -B & A \end{array} \right) \left( \begin{array}{rr} A & B \\ B & C \end{array} \right) \left( \begin{array}{rr} 1 & -B \\ 0 & A \end{array} \right) = \left( \begin{array}{cc} A & 0 \\ 0 & A (AC - B^2) \end{array} \right) $$ This is a short version of Gauss-Legendre composition for integral binary quadratic forms.
Oh, almost forgot. You actually want the lattice reduced. Furthermore, if $B$ is a multiple of $A,$ then the lattice is already $SL_2 \mathbb Z$ equivalent to a diagonal one.