Given a scalene triangle $P$, does there exist a similar scalene triangle $P'$, such that when $P'$ is put inside $P$, all three of $P'$ vertices's touch all three of the inner edges of $P$?
If so, is there a function that you can use to find the smallest possible $P'$ that will still touch all of the inner edges of $P$?
In reply to the first question, let $P$ be a scalene triangle $ABC$. Bisect its sides at $D$, $E$, $F$, and complete triangle $DEF$. Then by VI, 2 of Euclid's Elements, $FE$, $ED$, $DF$ are parallel to $BC$, $AB$, $CA$, respectively, and therefore triangle $DEF$, which we may call $P'$, is similar to $P$.
It can be seen that$$P=4P'$$Whether $P'$ is the smallest (or largest?) similar triangle with vertices lying on the sides of $\triangle ABC$ remains to be considered.