Suppose $x,y,z$ are coded as decimal or their binary representations in an appropriate DFA. Is $L$ regular?
My intuition tells me that the answer is no, because there are infinitely many combinations such that $x+y=z$ and a DFA must contain a finite amount of memory. Is this correct?

This is a funny question, and the answer is: it depends.
First, if $x$, $y$ and $z$ are given sequentially, then pumping lemma implies that triplet $\langle 1(0^n),0,1(0^m)\rangle$ would have to be accepted for multiple values of $m$ and $n$, not necessarily equal.
On the other hand, if the numbers are given simultaneously (e.g. on three tapes, or perhaps by triplets of digits), then it is regular. Consider the case when the automaton processes the numbers right-to-left, the only thing the automaton needs to know is the "carry flag" (or a small number), which, in case of addition can be only $0$, $1$, or $2$. With the other direction, it is just the reversed language.
Also, finite state transducers are even able to carry out addition for numbers given in parallel (i.e. generate $z$ from $x+y$), but there is a difference between deterministic and non-deterministic transducers, so the processing order would matter there.
I hope this helps $\ddot\smile$