It's been a while since I've had to notate math properly and I could only get so much from googling. I've checked my math extensively but I feel like my notation and formatting is improper, any tips?
The formula is a modified combination that involves: $m$ elements labeled $n_1$ through $n_m$, the total sum of the previous elements $n_0$, and the size of the resulting subset $r$. With the condition that $r$ is the smallest element, heres what I have:
$C_{mex}(n := \{n_1...n_m\}, n_0 := \sum_{i=1}^mn_i, r) = C(n_0,r) - \sum_{i=2}^m(n_i \sum_{j=2}^i(C(i,j) \cdot C(n_0-j,r-j))) $
where $\forall n_i \in n , r < n_i$
Primarily I'm wondering about the declarations and the condition. As a secondary question, how would I write a second condition that each element needs to be in the set of natural numbers? Any help is appreciated!
If I understand correctly, you want your function to depend upon $r$ and the integers $n_1,\dots,n_m$. (The quantity $n_0$ can be expressed in terms of the $n_j$, it seems.) So I suggest the modification $$ C_{mex}(r,n_1...n_m) = C\biggl( \sum_{k=1}^{m} n_k,r \biggr) - \sum_{i=2}^m n_i \sum_{j=2}^i C(i,j) C \biggl( \sum_{k=1}^{j-1} n_k -j,r-j \biggr) . $$ (I'm not sure I got the upper limits as you intended when converting away from the $n_0$ notation.) Notice also how eliminating unnecessary parentheses makes the expression cleaner.
In general mathematicians prefer the binomial coefficient notation $\binom ab$ to $C(a,b)$, so the right-hand side could also be written as $$ \binom{ \sum_{k=1}^{m} n_k}{r} - \sum_{i=2}^m n_i \sum_{j=2}^i \binom ij \binom{ \sum_{k=1}^{j-1} n_k -j}{r-j}. $$