This summation came up in a problem I was working on. My first attempt in coming up with a formula yielded the result $\binom{n-1}3\binom{n-2}3-\sum\limits_{k=3}^{n-3}\binom{k+1}2\binom k3$ at which point I was unable to simplify any further. I then devised a second method to achieve a formula and came up with the result $\sum\limits_{k=5}^n\binom{k-3}2\binom{k-1}3$. Both appear to be equivalent, which I was unable to prove directly, but could with induction if needed. I tried using this to come up with a closed formula for one summation or the other but was unable to figure out a way to do so.
If worse came to worse, I could treat the summand as a quadratic multiplied by a cubic, in which case, the result of the sum would be some degree 6 polynomial, but is there a nicer closed form for one of these summations?
Note $$\binom{k-3}{2}\binom{k-1}{3}=\frac{1}{12}(k-1)(k-2)(k-3)^2(k-4)$$ Consider the expression $$\binom k5(5k-13)-\binom{k-1}{5}(5k-18)$$ Factorise out $(k-1)(k-2)(k-3)(k-4)/120$, $$\frac{(k-1)(k-2)(k-3)(k-4)}{120}\bigg(k(5k-13)-(k-5)(5k-18)\bigg)$$ Notice that the expression in brackets simply reduces to $30k-90=30(k-3)$, and so $$3\binom{k-3}{2}\binom{k-1}{3}=\binom k5(5k-13)-\binom{k-1}{5}(5k-18)$$ Sum this up from $k=5$ to $k=n$ (RHS is telescoping series) so that the required sum $S(n)$ is $$\boxed{S(n)=\frac13\binom{n}{5}(5n-13)}$$ This is a pretty nice closed form. How did I guess that form? WA ;)
Hope this helps. :)