I've been trying to do the following problem:
Let $ABCD$ be a convex quadrilateral and points $E$ and $F$ on sides $AB,CD$ such that:
$\frac{AB}{AE}=\frac{CD}{DF}=n$
If $S$ is the area of $AEFD$ show that $\frac{AB*CD+n(n-1)AD^2+n^2*DA*BC}{2n^2}\ge S$
I attempted to solve it in the following way:
$\frac{AB*CD}{2n^2}=\frac{AE*DF}{2}$
$\frac{n^2*DA*BC}{2}=\frac{DA*BC}{2}$
$\frac{n(n-1)AD^2}{2n^2}=\frac{(n-1)AD^2}{2n}$
so we have that $\frac{AB*CD+n(n-1)AD^2+n^2*DA*BC}{2n^2}=\frac{AE*DF}{2}+\frac{DA*BC}{2}+\frac{(n-1)*AD^2}{2n}$
This is as far as I got, I couldn't think of any theorems or how to correlate these results with the area of $S$. Could you please continue my thought pattern and finish it off? Or explain why it can't be solved using my thought pattern and show an alternative approach?
Although what you have started to do is correct, use of the given condition $\left(\mathrm{i.e.}\space \frac{AB}{AE}=\frac{DC}{DF}=n\right)$ alone is not going to lead us to the required proof. You have to introduce the area of the quadrilateral $S$ into the picture by bringing certain theorems, inequalities and other observations into play. We start by proving a simple lemma, which we need later. We also assume that $\infty\ge n\ge 1$.
$\underline{\mathrm{1.\space Lemma}}$
$TUVW$ is a convex quadrilateral (see $\mathrm{Fig.\space 1.1}$). Points $X$ and $Y$ divide its sides $TU$ and $VW$ respectively, such that $\frac{TU}{TX}=\frac{WV}{WY}=n$. The following inequality is true for all $n$, while the equality holds only when the two sides $UV$ and $WT$ are parallel to each other. $$\left(\frac{n-1}{n}\right)WT+\left(\frac{1}{n}\right)UV\ge XY \tag{1}$$
$\underline{\mathrm{1.1.\space Proof}}$
Draw a line parallel to $WT$ through $X$ to meet the diagonal $UW$ at $Z$. This makes $\frac{WU}{WZ}=n$. Now, draw another line, this time parallel to $UV$, through $Y$ to meet the diagonal $UW$ at some point $Z_1$. This gives us $\frac{WU}{WZ_1}=n$. Since $\frac{WU}{WZ_1} = \frac{WU}{WZ}$, the two points $Z_1$ and $Z$ are one and the same.
When we apply the triangle inequality to the triangle $XYZ$, we have $ZX+YZ\gt XY$. If $WT$ is parallel to $UV$, it is obvious that $Z$ lies on $XY$. Therefore, only in this particular instance, we can write $ZX+YZ= XY$. Therefore, in general, it is true that $$ZX+YZ\ge XY. \tag{2}$$ We know that $ZX= \left(\frac{n-1}{n}\right)WT$ and $YZ=\left(\frac{1}{n}\right)UV$. When we substitute these values in inequality (2), we get, $$\left(\frac{n-1}{n}\right)WT+\left(\frac{1}{n}\right)UV\ge XY.$$
$\underline{\mathrm{2.\space Ptolemy’s\space Theorem\space and\space Inequality}}$
Consider $\mathrm{Fig.\space 1.2}$. Ptolemy’s theorem states that, for the cyclic quadrilateral shown in that diagram, the sum of products of the two pairs of opposite sides equals the product of its diagonals, i.e. $AC\times BD = AB\times CD + BC\times DA$. However, for a quadrilateral which is not cyclic, Ptolemy’s theorem becomes an inequality, i.e. $AC\times BD \lt AB\times CD + BC\times DA$. Therefore, in general, it is true that, $$AC\times BD \le AB\times CD + BC\times DA. \tag{3}$$
$\underline{\mathrm{3.\space An\space Area\space Inequality\space for\space Convex\space Quadrilaterals}}$
Consider $\mathrm{Fig.\space 1.3}$. The area $S$ of the quadrilateral shown in that diagram can be expressed as $2S= AC\times BD\sin\left(\phi\right)$, where $\phi$ is the angle between its two diagonals $AC$ and $BD$. For a given pair of diagonals, the left-hand side of this expression has its maximum value, if they are perpendicular to each other, i.e. $\phi=90^o$. Therefore, we can write, $$S\le \frac{AC\times BD}{2}. \tag{4}$$
$\underline{\mathrm{4.\space Proof\space of\space the\space Inequality}}$
Now, we have all the tools we need to prove the given inequality. Please pay your attention to $\mathrm{Fig.\space 2}$ and also note that we start off by proving a slightly different inequality, i.e. $$\frac{AB\times CD+n\left(n-1\right)AD^2+nDA\times BC}{2n^2}≥S, \tag{5}$$ in which the last term of the numerator on the left-hand side is not $\color{red}{n^2}DA\times BC$, but $\color{red}{n}DA\times BC$.
First of all, we expand the left-hand side of inequality (5) to to get, $$\frac{AB\times CD+n\left(n-1\right)AD^2+nDA\times BC}{2n^2}\qquad\qquad =\frac{1}{2}\left[\frac{AB}{n}\times\frac{CD}{n}+AD\left\{\left(\frac{n-1}{n}\right)AD+\frac{1}{n}BC\right\}\right].$$
As exactly as you have done, we too proceed by eliminating $AB$ and $DC$ using the given relations $AB=nAE$ and $CD=nDF$. $$\frac{AB\times CD+n\left(n-1\right)AD^2+nDA\times BC}{2n^2}\space\quad\qquad =\frac{1}{2}\left[AE\times DF+AD\left\{\left(\frac{n-1}{n}\right)AD+\frac{1}{n}BC\right\}\right]$$
Now, we use the lemma we have proven above to obtain the following inequality. $$\frac{AB\times CD+n\left(n-1\right)AD^2+nDA\times BC}{2n^2}≥\frac{1}{2}\left(AE\times DF+AD\times EF\right)$$
Next, with the help of Ptolemy’s inequality, we simplify the right-hand side of the above inequality as shown below. $$\frac{AB\times CD+n\left(n-1\right)AD^2+nDA\times BC}{2n^2}≥\frac{1}{2}AF\times DE$$
Finally, we use the area inequality for quadrilaterals to complete our proof of inequality (5), which is valid for all $\infty\ge n\ge 1$. $$\frac{AB\times CD+n\left(n-1\right)AD^2+nDA\times BC}{2n^2}≥S $$
As shown below, we can now multiply the last term of the numerator on the left-hand side of inequality (5) by $n$ to obtain the inequality you have stated in your question, because $n\ge 1$, i.e. $$\frac{AB\times CD+n\left(n-1\right)AD^2+n^2DA\times BC}{2n^2}≥S. \tag{6}$$
Inequality (6) still preserves its validity, however, it becomes weaker than inequality (5).