I often come across figures like this on the net, or as contest problems, asking to find the number of a specific type of polygon in the figure (triangles, in this case). But I've never really found a method for solving it, rather than blindly counting and hoping it's the right answer. For complete graphs, the problem is trivial:- ${n \choose k} \times (n-k)$, where $n$ is the number of vertices of the complete graph, and $(k+1)$ is the number of sides of the polygon in question. But I was wondering if there was a general algorithm for going about these problems. Any ideas?
DISCUSSION SO FAR: My above answer obtained for the complete graph is incorrect, as I hadn't considered the possibility of collinear vertices, and hence degenerate polygons (Thanks to Carl's brilliant answer, for pointing this out!). So here's the discussion so far: One can obtain the number of triangles (including degenerate) in any graph from the trace of its adjacency matrix $A$, by the formula $\text{Tr}(A^3)/3!$ (read Carl's post for the explanation). Unfortunately, this cannot be used for higher polygons, since cycles of path lengths $k$, $ k>3$ are attainable in less than $k$ vertices. So, focusing for now on obtaining the number of triangles in a graph, the algorithm is to compute $\text{Tr}(A^3)/3!$, and then to subtract the number of degenerate triangles from it. So now, the essential question is: Is there is a method for systematically enumerating distinct sets of collinear nodes in a graph?
This post illustrates why $\text{Tr}(A^k)$ gives the number of cycles of length $k$ in the graph with adjacency matrix $A$. We have to be careful since a cycle is not the same thing as a polygon. For example, we can make a 4-cycle by moving between two nodes only: XYXY is a 4-cycle, but not a quadrilateral.
For triangles it turns out that this is not a problem since you can't form a 3-cycle that isn't also a triangle. (Caveat: you might form a degenerate (flat) triangle if three nodes are colinear. There is no way of detecting whether points are colinear using an abstract graph, represented as an adjacency matrix.)
Even with triangles, we still have to be careful of double counting. $\text{Tr}(A^3)$ is the number of 3-cycles in an undirected graph, but this gives 6 for the simple case of a 3-node graph that is itself triangle. $$A=\left(\begin{array}{ccc}0&1&1\\1&0&1\\1&1&0\end{array}\right) \Rightarrow \text{Tr}(A^3)=6$$ This is because all of the following cycles are triangles, even though they all form the same triangle: XYZ, XZY, YXZ, YZX, ZXY, ZYX.
Since each triangle gets counted 6 times, the total number of triangles in a graph with adjacency matrix $A$ is $$\text{Tr}(A^3)/6 = \text{Tr}(A^3)/3!$$
The example in the original question will have quite a few degenerate triangles since there are so many colinear vertices, making this method less than ideal. You can however compute all the triangles using the trace of the adjacency matrix and then try to count and subtract the number of degenerate triangles.