Problem:
Evaluate: $$\dfrac{\displaystyle\int_0^1 x^{\frac{5}{2}}(1-x)^{\tfrac{7}{2}}dx}{\displaystyle\int_0^1 \dfrac{x^{\frac{5}{2}}(1-x)^{\tfrac{7}{2}}dx}{(x+3)^8}} $$
They give a solution in which substituting $x=\dfrac{4y}{3+y}$ in the integral in the denominator makes the two functions proportional.
How does one come up with this particular function?
The hinted solution probably goes something like this:
Let $I_n$/$I_d$ denote the numerator/denominator integrals, respectively. First, $(\star)$ substitute $x\mapsto1-x$ in $I_d$ to get
$$I_d = \int_0^1 \frac{x^{5/2} (1-x)^{7/2}}{(x+3)^8} \, dx = \int_0^1 \frac{(1-x)^{5/2} x^{7/2}}{(4-x)^8} \, dx$$
Now, we proceed by considering the substitution $y=\dfrac1{4-x} \iff x=\dfrac{4y-1}y$, which is not beyond the realm of trial and error. This ends up eliminating the denominator and the subsequent integral is elementary, e.g. amenable to Euler substitution.
$$\begin{align*} I_d &= \int_{\tfrac14}^{\tfrac13} \left(1-\frac{4y-1}y\right)^{5/2} \left(\frac{4y-1}y\right)^{7/2} y^8 \, \frac{dy}{y^2} \\ &= \int_{\tfrac14}^{\tfrac13} (1-3y)^{5/2}(4y-1)^{7/2} \, dy \end{align*}$$
Rather than evaluating $I_d$ in the tedious way, at this point we can instead observe that scaling the integration variable by a factor of $12$ and shifting by $3$ units would transform the interval acc. to $\left[\dfrac14,\dfrac13\right]\mapsto[3,4]\mapsto[0,1]$. In other words, we modify the substitution to
$$y=\frac{\color{red}{12}}{4-x} \color{red}{-3} = \frac{3x}{4-x} \implies x=\color{red}{\frac{4y}{3+y}}$$
Finally, we reuse $(\star)$ to recover a multiple of $I_n$:
$$\begin{align*} I_d &= \frac1{12} \int_3^4 \left(1-\frac y4\right)^{5/2}\left(\frac y3-1\right)^{7/2} \, dy & y\mapsto\dfrac y{12} \\ &= \frac1{12} \int_0^1 \left(1-\frac{y+3}4\right)^{5/2}\left(\frac{y+3}3-1\right)^{7/2} \, dy & y\mapsto y+3 \\ &= \frac1{12\cdot3^{7/2}\cdot4^{5/2}} \int_0^1 (1-y)^{5/2} y^{7/2} \, dy \\ &= \frac{I_n}{2^7 \cdot 3^{9/2}} & (\star) \end{align*}$$
I can't speak to why the hinted solution would begin with $(\star)$. We can just as easily cook up and proceed with
$$y = \frac{4x}{x+3} \iff x = \frac{3y}{4-y} \\ \implies I_d = \int_0^1 \frac{\left(\frac{3y}{4-y}\right)^{5/2} \left(1-\frac{3y}{4-y}\right)^{7/2}}{\left(\frac{3y}{4-y}+3\right)^8} \, \frac{12}{(4-y)^2} \, dy = \frac1{2^7 \cdot 3^{9/2}} \int_0^1 y^{5/2} (1-y)^{7/2} \, dy$$