Suppose we choose a random $n\times n$ matrix $A$ over $\mathbb F_2$ by setting each entry of the matrix equal to $0$ with probability $0.5$ and $1$ with probability $0.5$ (this is equivalent to choosing from among the $n\times n$ matrices with equal probability). I am trying to calculate the probability that $A^2=0$ asymptotically for large $n$. I believe that the following is a closed form of this probability:
$$P(A^2=0)=\frac{1}{2^{n^2}}\sum_{k=0}^{\lfloor n/2\rfloor} \frac{(2^n-1)(2^n-2)(2^n-4)...(2^n-2^{2k})}{k!}$$
QUESTION: Can anyone help me evaluate the asymptotic behavior of the above expression for large values of $n$?
NOTE: Because of the $\lfloor n/2\rfloor$, I anticipate that we may have to break it down into even and odd cases that have different asymptotic behaviors. Also, whatever asymptotic expression we come up with might make use of the following constant: $$\nu=\prod_{k=1}^\infty \bigg(1-\frac{1}{2^k}\bigg)\approx 0.28879...$$ which is the asymptotic probability that a randomly chosen matrix is invertible, for large $n$.
Very good question. This is a significant increase in the average level of questions in recent months...
I feel that your formula (if I understand it correctly) does not work; for $n=2$, you obtain $3/2^4$ while the correct solution is $4/2^4=2^{-2}$. More generally $prob(A^n=0)=2^{-n}$ (because one has $n$ independent relations on $F_2$ linking the entries of $A$).
For $n=3$, random tests indicate that $prob(A^2=0)\approx 0.04$, while your formula gives $\approx 0.342$.
When $n$ is large, a generic nilpotent matrix satisfies with a large probability $A^{n-1}\not= 0$. Then $prob(A^2=0)$ is much smaller than $2^{-n}$. Random tests seem to indicate that $prob(A^2=0)\approx 2^{-n^2/2}$.
EDIT 1. I see that the OP does not have the strength to comment on my answer. So I ignore him and continue the job. The subject is studied in
https://oeis.org/search?q=4%2C22%2C316%2C6976&language=french&go=Chercher
More precisely, a correct formula for the number of matrices s.t. $A^2=0$ is
$(1)$ $a_n=coeff(\dfrac{\Pi_{i\geq 1}(1+u/2^i)}{\Pi_{i\geq 1}(1-u^2/2^i)},u^n)(2-1)...(2^n-1)$.
EDIT 2. Let $\alpha_0=1$ and, if $j\geq 1$, then let $\alpha_j=\Pi_{1\leq i\leq j}(1-2^{-i})$. Note that $\alpha_j\rightarrow \nu\approx 0.28878809508660242126$ and $\alpha_j\in [\nu,1]$. We deduce that
$\textbf{Lemma}$. For a large enough $n$ and $r\leq n/2$, $\dfrac{\alpha_n}{\alpha_r\alpha_{n-2r}}\in[1,3.5]$.
Another formula for $a_n$ is
$(2)$ $a_n=1+\sum_{r=1}^{\lfloor n/2\rfloor}2^{2r(n-r)}\dfrac{\alpha_n}{\alpha_r\alpha_{n-2r}}$.
$\textbf{Proposition}$. $a_n\in \Omega(2^{n^2/2})$ and $prob(A^2=0)=\Omega(2^{-n^2/2})$.
$\textbf{Proof}$. According to the Lemma, it suffices to show that $b_n=\sum_{r=1}^{\lfloor n/2\rfloor}2^{2r(n-r)}\in \Omega(2^{n^2/2}).$
If $n$ is even, then $b_n\sim 2^{n^2/2}(1+2^{-2}+2^{-2.2^2}+2^{-2.3^2}+\cdots)$, that is, $b_n\sim k_{even}2^{n^2/2}$, where
$k_{even}\approx 1.2539100649300971568$.
In the same way, when $n$ is odd, we obtain $b_n\sim k_{odd}2^{n^2/2}$, where $k_{odd}\approx 0.751473630649698988$. $\square$
$\textbf{Remark}$. Experimentally, we see that
$ 2^{-n^2/2}\leq prob(A^2=0)\leq 2\times 2^{-n^2/2}$.
More precisely, calculations seem to show
$\textbf{Conjecture}$. $a_n\sim \tau .2^{n^2/2}$, where
if $n$ is even, then $\tau\approx 1.6793780841750303241$
if $n$ is odd, then $\tau\approx 1.5494800120725299756$.