Let $f$ be a holomorphic function in the Hardy space $H^2(U)$ with $\parallel f\parallel_{H^2}\leq 1$. For a given $\alpha\in U$(the open unit disk) Find the upper bound of $|f'(\alpha)|$.
We know that $\parallel f\parallel_{H^2}=(\int_0^{2\pi}|f(re^{i\theta})d\theta|)^{1/2}$. Consider $\varphi_{-\alpha}(z)=\frac{z+\alpha}{1+\bar{z}\alpha}$ and let $g=f\circ\varphi_{-\alpha}$.
If $\parallel g\parallel_{H^2}\leq 1$, then $$|f'(\alpha)|=\frac{|g'(0)|}{|\varphi'_{-\alpha}(0)|}\leq \frac{|g(0)|}{r(1-|\alpha|^2)}$$ by Cauchy's estimate for some $0<r<1$. Since $$|g(0)|=\frac{1}{2\pi}|\int_0^{2\pi}g(re^{i\theta})d\theta|\leq \frac{1}{2\pi}\int_0^{2\pi}|g(re^{i\theta})|^2rd\theta \cdot \int_0^{2\pi}rd\theta$$ by mean value principle and Cauchy-Schwartz inequality.
Then $$|f'(\alpha)|\leq \frac{1}{1-|\alpha|^2}$$.
I want to ask whether $\parallel g\parallel_{H^2}\leq 1$ is true. If not, how can I get the upper bound of $|f'(\alpha)|$?
Your estimation $$ |f'(\alpha)|\leq \frac{1}{1-|\alpha|^2} $$ is wrong.
Let $f(z)\in H^2(U)$ with $\parallel f\parallel_{H^2}\leq 1$ and it's Taylor expansion be $\sum_{n=0}^\infty a_nz^n \,(|z|<1)$.
Then \begin{align} &\int_0^{2\pi} |f(re^{i\theta }|^2\,d\theta=\int_0^{2\pi} \left(\sum_{n=0}^\infty a_nz^n\right)\overline{\left(\sum_{n=0}^\infty a_nz^n\right)}d\theta =2\pi\sum_{n=0}^\infty |a_n|^2r^{2n},\\ &\parallel f\parallel_{H^2}=\sup_{r<1}\left(\frac{1}{2\pi}\int_0^{2\pi} |f(re^{i\theta }|^2\,d\theta \right)^\frac{1}{2}=\left(\sum_{n=0}^\infty |a_n|^2\right)^\frac{1}{2}\le 1. \end{align}
Using Cauchy-Schwarz inequality we have \begin{align} |f^\prime(\alpha )|&=\left| \sum_{n=1}^\infty na_n\alpha ^{n-1}\right|\le \left( \sum_{n=1}^\infty |a_n|^2\cdot\sum_{n=1}^\infty n^2|\alpha |^{2(n-1)}\right)^\frac{1}{2}\\ &\le \left(\sum_{n=1}^\infty n^2|\alpha |^{2(n-1)}\right)^\frac{1}{2}=\frac{\sqrt{1+|\alpha |^2}}{(1-|\alpha |^2)^\frac{3}{2}},\tag{1} \end{align} since $$ \sum_{n=1}^\infty n^2x^{n-1}=\frac{1+x}{(1-x)^3}\quad (|x|<1).$$
The estimation $(1)$ is sharp. Fix $\alpha $ with $0<\alpha <1$ and let $\beta =\frac{\sqrt{1+\alpha ^2}}{(1-\alpha ^2)^\frac{3}{2}}$. Define $$ f(z)=\frac{1}{\beta }\sum_{n=1}^\infty n\alpha ^{n-1}z^n.$$ Then $f\in H^2(U), \,\parallel f\parallel_{H^2}\leq 1$ and $$ |f^\prime(\alpha )|=\frac{\sqrt{1+\alpha ^2}}{(1-\alpha ^2)^\frac{3}{2}}.$$ Note that $$\frac{\sqrt{1+\alpha ^2}}{(1-\alpha ^2)^\frac{3}{2}}>\frac{1}{1-|\alpha|^2}.$$