Let $X,Y$ be finite sets such $\lvert X\rvert \leqslant \lvert Y\rvert,\\ \lvert X\cup Y\rvert = 16,\\ \rvert X\cap Y\rvert = 10,\\ \lvert X\times Y\rvert = 168$
Find: $\lvert X\times X\rvert$
setting $\lvert X\rvert=n$ and $\rvert Y\rvert=m$, using the inclusion exclusion principle i get, $n+m=26$
Do I assume that since $\lvert X\times Y\rvert=168$ implies $nm=168$ implies that $m=168/n$ then solve for $n$?
Yes. If $X$ and $Y$ are finite sets (here they are, because their union is finite), then the cardinality of $X \times Y$ is the cardinality of $X$ times the cardinality of $Y$, or in mathematical terms , $|X \times Y| = |X||Y|$. This would give:$mn=168$ Further, since $|X|+|Y|=|X \cup Y| + |X \cap Y|$, it follows that $|X| + |Y|=16+10=26$ i.e. $m + n = 26$. Now, solve the system $mn=168,m+n=26$, by putting $m=168/n$, and get the equation $n^2-26n+168=0$. Now, use the quadratic formula that you know very well: $$ n=\frac{-26 \pm \sqrt{26^2-4*168}}{2} = \frac{-26 \pm \sqrt{4}}{2} = 12/14, m =14/12 $$
and you will notice that $m=12,n=14$, since $m \leq n$.