I've come across a definition for a symmetry in category theory a few times. This answer puts it most succinctly:
A symmetry of a morphism $\phi:A\to B$ means a pair $(\alpha,\beta)$ of automorphisms of $A$ and $B$ respectively, such that $\beta \circ \phi = \phi \circ \alpha$.
I've been exploring weaker definitions of symmetries, such as semisymmetries which relax $\alpha$ and $\beta$ from automorphisms to just endomorphisms. And there are quasi-symmetries as well. Most of these, acting on $\alpha$ and $\beta$ substantially change the objects being studied. For instance, while symmetries form a group, semisymetries only form a monoid.
Lately I've been questioning a more subtle question... what about the properties of $\phi?$ In the category theoretic definition of symmetry above, it is a morphism. This means it has all of the properties of a morphism, such as the associative property of morphism compositions.
Are there any properties of such symmetries that depend on $\phi$ being a full morphism, or is that just a fundamnetal tool of category theory so they just use it? Phrased another way, and perhaps more answerably, are there important properties of symmetries which cannot be proven if composition with $\phi$ lacks the associative property? As best as I can tell, the group theoretic behavior of symmetries does not depend on $\phi$ being associative, only that $\alpha$ and $\beta$ are associative within their categories, and I believe all profound statements about symmetries derive from this forming a group.
The paper "Plots and Their Applications - Part 1: Foundations" by Salvatore Tringali, available at https://arxiv.org/abs/1311.3524v1, explores how characterizations of objects by universal properties interacts with isomorphisms between objects when associativity and existence of identities are dropped from the notion of a category. Issues arise as to whether and when isomorphisms exist, for which the existence of identities is moreso significant than associativity.
Tringali calls such a category-like structure with associaitivty and existence of units dropped a plot. In particular, it is in a plot that non-associative morphisms are naturally situated. For any plot, one can form the plot of morphisms whose objects are morphisms and whose morphisms are commutative squares. In particular, a symmetry of a non-associative morphism would then be an endomorphism of the morphism considered as an object in the plot of morphisms.