Let $\{A,B,C\}$ denote angles of a triangle in degrees such that $A,B,C\in N$.
How many un-ordered triplets of $\{A,B,C\}$ will give acute angled triangles.
My Attempt:
I tried to find number of integral solutions to equation $A+B+C=180$ using
Number of triangles $=$coefficient of $x^{180}$ in expansion $\left(x+x^2+x^3+...+x^{89}\right)^3$
but was not able to arrive at the given answer. The answer is $675$
On
Let $A$, $B$, $C$ with $$1\leq A\leq B\leq C\leq 89, \qquad A+B+C=180$$ be the three angles. We then have $$A=1+x,\quad B=1+x+y,\quad C=1+x+y+z$$ with $$x\geq0,\quad y\geq0,\quad z\geq0,\qquad x+y+z\leq88,\quad 3x+2y+z=177\ .$$ The last equation implies $z=177-(3x+2y)$. Introducing this into $x+y+z\leq88$ leads to the conditions $$x\geq0,\quad y\geq0,\qquad 2x+y\geq89,\quad 3x+2y\leq177\ .\tag{1}$$ We now have to compute the number of lattice points $(x,y)$ satisfying $(1)$. I gave this over to Mathematica that found $675$ solutions.
HINT
The generating function counts the no. of ordered triplets, but there is not a $3!$-to-$1$ mapping between ordered and unordered triplets, because e.g. $\{50,60,70\}$ maps to $3!$ ordered triplets but $\{50,50,80\}$ maps to only $3$ ordered triplets and $\{60,60,60\}$ maps to only $1$ ordered triplet. So you cannot just divide the answer by $6$. Personally I don't see an easy way to convert from one answer to another.
Instead a somewhat simple solution is to let $A \le B \le C < 90$ and just enumerate. Obviously $60 \le C \le 89$, so just count the number $f(C)$ of solutions with that $C$ value and sum over $[60, 89]$.
$A+B = 180 -C$, so once you know the set of possible values for $A$, that counts $f(C)$.
If not for the other constraints, then $f(C)$ would have been simply $180-C-1$ as we allow $A$ to range from $1$ to $180-C-1$.
So you need to take care of the constraints $A \le B \le C$ by consider how the value of $C$ restricts the range of $A$.