Often we regard certain mathematical expressions, or elements thereof, as arbitrary, in the sense that they have no apparent reason or cause, whereas more beautiful or natural seeming expressions feel more meaningful or useful. For example, $ e^{i \pi} + 1 $, $ \ln 2 $, and $ \sqrt{x^2 + y^2 + z^2} $ as opposed to $ e^{1/2}-1 $, $ \log 15 $, or $ x+2y-1016z $. Mathematicians, through their creativity, have endlessly defined, invented or discovered objects, ideas and relationships, but only the most somehow compelling results end up in final theories, while the more apparently superfluous stuff gets disregarded and forgotten. The perception certainly varies from person to person, and with the amount and kind of mathematics they've been exposed to (here I'm thinking of the monstrous moonshine and the value in some modular form expansion, or Hardy's taxi number on the way to Ramanujan), so is the concept necessarily subjective or could a rigorous definition of the idea be crystallized out of this sort of thing? It sounds sort of foggy and intractable like "irreducible complexity" but perhaps it could fare better in the pure abstract.
How might we quantify meaningfulness vs. arbitrariness in mathematics, if only in principle - i.e. even if actually computing such a measure is in general infeasible with our limited resources and knowledge?
You can if you like give some precise mathematical definitions for "meaningful" and "arbitrary", but such definitions would be arbitrary rather than meaningful.
Less preciously: you're not going to be able to give such a definition that agrees with the way actual mathematicians use these terms. Proof: the use of these terms is only somewhat consistent, not perfectly consistent. "Meaningful" and "arbitrary" are more like sociological concepts than mathematical concepts. You might have better luck studying them as a naturalist would: i.e., rather than trying to envelop them in a single formal theory, study them in practice and see what various meaningful mathematical concepts and constructions (usually) have in common.
By the way, you haven't given any motivation for wanting to formalize these concepts. Why would you want to do so? As you seem to realize, it is very unlikely that some kind of formal theory along these lines would be helpful in one's actual study of mathematics.