Given the triangular area $$\mathcal{D}=\{(x,y)\in\mathbb{R}^2|x\gt0,y\gt0,x+y\lt3\}$$ and the sequence $(a_n)_{n\in\mathbb{N}}$ with $$a_n=\int_{\mathcal{D}}(3xy-x^2y-xy^2)^n\lambda_2(dx dy)$$ The notation $\lambda_2(dx dy)$ means, that we integrate on 2-dim Lebesgue Measure $\lambda_2$.
Questions:
a) Find $b\in\mathbb{R}$ and $c\gt0$, such that $$\mathscr{l}_n:=a_nn^bc^n\overset{n\rightarrow\infty}{\longrightarrow}{r\in\mathbb{R}^+}$$
b) Find the value of $r$.
Part of my idea:
Use affine-linear transformation$$\tilde{x}=n^\beta(x-1)\quad\tilde{y}=n^\beta(y-1)$$for appropriate $\beta\in\mathbb{R}$, then observe the asymptotics of the integral after transformation.
Define $\operatorname{f}:\mathbb{R}^2\rightarrow\mathbb{R}^2$, $(x,y)\mapsto(\frac{\tilde{x}}{n^\beta}+1,\frac{\tilde{y}}{n^\beta}+1)$
Jacobian $|\det D_f|=\left|\det\begin{pmatrix} \frac{1}{n^\beta} & 0 \\0 & \frac{1}{n^\beta}\\ \end{pmatrix}\right|$=$n^{-2\beta}\gt0,\forall{n}\in\mathbb{N}$
Then $$a_n=n^{-2\beta}\int_\mathcal{D}(3(\frac{\tilde{x}}{n^\beta}+1)(\frac{\tilde{y}}{n^\beta}+1)-(\frac{\tilde{x}}{n^\beta}+1)^2(\frac{\tilde{y}}{n^\beta}+1)-(\frac{\tilde{x}}{n^\beta}+1)(\frac{\tilde{y}}{n^\beta}+1)^2)^nd\lambda_2(x,y)\quad=n^{-2\beta}\int_{\mathcal{D}}(\frac{-\tilde{x}^2n^{\beta}+n^{3\beta}-\tilde{x}^2\tilde{y}-\tilde{x}\tilde{y}^2-n^{\beta}\tilde{y}^2+n^{\beta}\tilde{x}\tilde{y}}{n^{3\beta}})^nd\lambda_2(\tilde{x},\tilde{y})$$
and that's where i get stuck. How do i proceed with the calculation?
Thanks in advance!
As a corollary of the beta integral formula, you have for nonnegative integers $a$ and $b$ that $$\int_0^1 x^a (1-x)^b \,dx = \frac{a! b!}{(a + b + 1)!}.$$
Now as a further corollary of this, let us fix a constant $C > 0$; then the substitution $x = Ct$ will give $$\int_0^C x^a (C-x)^b \,dx = C \int_0^1 (Ct)^a [C(1-t)]^b\,dt = C^{a+b+1} \int_0^1 t^a (1-t)^b\,dt = \frac{C^{a+b+1} a! b!}{(a+b+1)!}.$$
Therefore, in the desired integral, let us evaluate the integral with respect to $y$ first: by the above formula, $$\int_0^{3-x} x^n y^n(3-x-y)^n\,dy = x^n \frac{(3-x)^{2n+1} n! n!}{(2n+1)!}.$$ And then, $a_n$ is the integral of this from $x=0$ to $x=3$: $$a_n = \int_0^3 \frac{x^n (3-x)^{2n+1} n! n!}{(2n+1)!} \,dx = \frac{n! n!}{(2n+1)!} \cdot 3^{3n+2} \frac{n! (2n+1)!}{(3n+2)!} = 3^{3n+2} \frac{(n!)^3}{(3n+2)!}.$$
From here, as in the answer by Robert Z, you can use Stirling's formula to conclude that $a_n\sim \frac{2\pi}{n\sqrt{3}}$ as $n\to\infty$.