Let us say that I have a set of symbols, $S$.
The symbols can be operated on by a set of $n$-ary operators, $O$.
Importantly, some of these operators are in the set of symbols, i.e. $S \cap O \neq \emptyset$.
If I am searching MathSciNet for work done with algebras that have such a flavour, what keywords should I use?
I would say $n$-ary trees in which each internal node can be labeled with an operator from $O$ and all leaf nodes can be operators or symbols. If the number of symbols is finite, then $S$ will be finite too, since the maximum size of $O$ is finite. If you take a tree that has operator leaves, then it is an operator, and if all its leaves are filled with symbols, then what you actually have is the result of a composition of your operators, that is completed, so you do indeed have a symbol value.
So essentially what you have is all parse trees for a calculator with operators in $O$, lol.