As referred in the Visual group theory Book by Nathan Carter- The unofficial definition of a group says that :
A group is a collection of actions satisfying the rules:
1. there is a predefined list of actions that never change.
2. Every actions is reversible.
3. Every actions is deterministic.
4. Any sequence of consecutive actions is also an action.
then the appearence of a Cayley's Diagram should be such that the above rules are satisfied.
Now the question is that the following figure satisfies the above rules but is still not a cayley's diagram:

The reason the book states is that it is so because the group lacks regularity i.e. it does not repeats every one of its internal pattern throughout the whole diagram.
I can't understand what role does 'regularity' in cayley's diagram has to do with condition for being a group.
A graph can be a Cayley diagram only if all the subgraphs consisting of blue edges (and the adjacent vertices) are isomorphic, and all the subgraphs containing red edges (and the adjacent vertices) are isomorphic (this generalises to more generators). This is essentially what "regularity" is getting at.
In this example, it is not regular because the top red region is different from the bottom two red regions. To utilise this to find a contradiction, note that the blue edges imply that there is a symmetry along them (obtained by swapping as follows: $1\leftrightarrow5$, $2\leftrightarrow6$, $3\leftrightarrow7$, $4\leftrightarrow 8$). However, this is not a symmetry of your graph!