Part a.) I was able to understand that you can use integral properties to flip the limits which would essentially make the u sub an x sub but with reverse a and b
part b.) I get this mostly but wouldn't it be $1/1$ for $I(0,a)$ therefore making $I(a,0)\ne I(0,a)$?
part c.) I used $u=x^a$ and got $a-1$ and $b+1$ which I understand are pulled from the new integral but why can they be pulled from the new integral and how do they prove that $a\ge1$ and $b\ge0$?
part d.) Just plug in. I get it. Simple.
part e.) On the second line I can't understand fully how $a$ gets to be 1. I understand that every iteration moves the the numbers further from $0$ so therefore if we set $a-1=0$ we can understand that $b+a$ will always give us the correct iteration regardless but if we do that how does $a$ not become zero when we plug it back therefore making the numerator $-1$ not $1.$ It feels like $a$ is simultaneously two numbers. My only guess is that we are simply taking $a$ and making it one because we set the rules to be that $a$ can be no lower I just don't know how you can explain that mathematically.
This is so over my head.
E: Thank you to the moderator that improved my title.
(a) Correct.
(b) In part (a) you have proven that $I(a, b) = I(b, a)$. Let $b=0$, then $$I(a,0) = I(0,a) ~~~~~~~~~~~~~~~~~~~~~~~ (\spadesuit)$$ To prove the second bit, by definition we have that $$I(a,b) = \int_0^1 x^a (1-x)^b\, dx $$
Let $b=0$ then we have $$I(a, 0) = \int_0^1 x^a (1-x)^0\,{dx} = \int_0^1 x^a \,{dx} = \frac{1}{1+a} $$
So $\displaystyle I(a, 0) = \frac{1}{1+a}.$ Now with $(\spadesuit)$ we've proven that $\displaystyle I(a, 0) = I(0, a) = \frac{1}{1+a}.$
(c) This is integration by parts:
$$\int_{\alpha}^{\beta} f'(x)g(x)\,{dx} = f(x)g(x)\bigg|_{\alpha}^{\beta}-\int_{\alpha}^{\beta} f(x)g'(x)\,{dx} ~~~~~ (\clubsuit)$$
If you take $\displaystyle f(x) = \frac{x^{a+1}}{a+1}$ and $g(x) = (1-x)^b$. Then $$ \begin{cases} f'(x) = x^a\\ g'(x) = -b(1-x)^{b-1} \end{cases}$$
By $(\clubsuit) $ we have $$\begin{aligned} I(a,b) &= \int_0^1 x^a (1-x)^b \, dx \\& =\int_0^1 \bigg(\frac{x^{a+1}}{a+1} \bigg)'(1-x)^b\,{dx} \\& =\bigg(\frac{x^{a+1}}{a+1} \bigg)(1-x)^b\bigg|_0^1 - \int_0^1 \frac{x^{a+1}}{a+1}[(1-x)^b]' \,{dx} \\& = - \int_0^1 \frac{x^{a+1}}{a+1}[b(1-x)^{b-1}] \,{dx} \\& = \frac{b}{a+1}\int_0^1 x^{a+1}(1-x)^{b-1} \, {dx} \\& = \frac{b}{a+1} I(a+1, b-1). \end{aligned} $$
Where the last equality is by comparing $\int_0^1 x^{a+1}(1-x)^{b-1}\,{dx}$ to $I(a, b)= \int_0^1 x^{a}(1-x)^{b}\,{dx}$ seeing that it's equal to $I(a+1, b-1)$.
To get the second equality we again use $(\clubsuit)$ in the same manner (but also the note below).
If you take $\displaystyle f(x) = x^a$ and $\displaystyle g(x) = -\frac{1}{b+1}(1-x)^{b+1}$. Then $$ \begin{cases} f'(x) = ax^{a-1}\\ g'(x) = (1-x)^b \end{cases}$$
we have $$\begin{aligned} I(a,b) &= \int_0^1 x^a (1-x)^b \, dx \\& =\int_0^1 x^a\bigg( -\frac{1}{b+1}(1-x)^{b+1} \bigg)'\,{dx} \\& =x^a\bigg( -\frac{1}{b+1}(1-x)^{b+1}\bigg) \bigg|_0^1 - \int_0^1 (x^a)'\bigg(-\frac{(1-x)^{b+1}}{b+1}\bigg) \,{dx} \\& = \int_0^1 \frac{a x^{a-1}}{b+1}\cdot (1-x)^{b+1} \,{dx} \\& = \frac{a}{b+1}\int_0^1 x^{a-1}(1-x)^{b+1} \, {dx} \\& = \frac{a}{b+1} I(a-1, b+1). \end{aligned} $$
Where the last equality is by comparing $\int_0^1 x^{a-1}(1-x)^{b+1}\,{dx}$ to $I(a, b)= \int_0^1 x^{a}(1-x)^{b}\,{dx}$ seeing that it's equal to $I(a-1, b+1)$.
Note/addendum: The assumptions $a \ge 1, b \ge 0$ are given to you.
To understand why this is necessary: the equality $I(a,b) = \frac{a}{b+1}I(a-1, b+1)$ breaks down for $a=0$ because then the left hand-side is $I(0,b) = \int_0^1 (1-x)^b \,{dx} = \frac{1}{1+b}$ for $b \ge 0$ but the right-hand side is $I(-1, b+1) = \int_0^1 \frac{(1-x)^{b+1}}{x}\,{dx}$ which doesn't converge.
Also: since we had $I(a, b) = I(b, a)$ from part (b) we don't have to use integration by parts again to arrive the second formula. We can use that symmetry to get one from the other but also we have to switch the assumptions as well; i.e. $b \ge 1, a \ge 0$.
(d) Yes. It's just plug-in.
(e) Look at $I (a,b) = \displaystyle \frac{a}{b+1} I(a-1, b+1).$ You stop the iterations when you get to $I(0,\cdots)$. That's when the $a-1$ becomes $0$ or consequently when $a=1$. The numerator is always one bigger than the entry in the $X$ position in $I(X, \cdots)$. So when that entry is $0$, the numerator is $1$.