Consider a binary operation $*$ acting from a set $X$ to itself. It's useful and standard to work with operations which are associative, such that $(a*b)*c = a*(b*c)$. What about operations which are not associative?
Is there any way to characterize all different possible types of such binary operations $*$ which are not associative? Eg. Can we say that if $*$ is not associative, then it must instead satisfy one of set of other possible properties, depending on any other additional operations that we have on our set $X$?
If we also add some additional structure to our set $X$ so that we can add elements together and multiply by scalars, it's standard to quantify the amount that two elements of $X$ commute with each other under $*$ by calculating the commutator $[a,b] = a*b - b*a$. Is it ever useful to consider an 'associative commutator' $[abc] = (a*b)*c - a*(b*c)$, for a given non-associative $*$?
Finally, I know from Lie algebras that if $*$ anticommutes then it can be natural to consider a Jacobi identity
$(a*b)*c = a*(b*c) - b*(a*c)$
Are there other natural extensions of associativity in different settings? Why do Lie algebras use this Jacobi identity and not for example
$(a*b)*c = a*(b*c) + k b*(a*c)$
Where k is a scalar?
"What about operations which are not associative?" In many areas we encounter non-associative algebra structures, e.g., in operad theory, homology of partition sets, deformation theory, geometric structures on Lie groups, renormalisation theory in physics and many more.
In a certain sense one can answer your question what else can happen. One way is, to classify all nonassociative algebras defined by the action of invariant subspaces of the symmetric group $S_3$ on the associator of the considered laws, see for example here. But of course these are not all possibilities.
A well known example of a non-associative algebra structure related to Lie algebras are pre-Lie algebras (also called left-symmetric algebras). They satisfy the identity $$ (x,y,z)=(y,x,z) $$ for all $x,y,z\in A$, where $(x,y,z)$ is the associator. In particular, associative algebras are a trivial example where both sides are zero, i.e., with $0=0$. Then the commutator $$ [x,y]=xy-yx $$ is a Lie bracket, see Is there a relationship between associators and commutators?
Pre-Lie algebra arise in algebra, geometry and and physics, see my survey article here. They play an important role for crystallographic groups, fundamental groups of affinity flat manifolds (Milnor), Gerstenhaber deformation theory, Rota-Bater operators and Yang-Baxter equations, just to name a few key words.