I know that a similar question has been asked regarding finitism, but I'm interested in ultafinitism. That is, we define a set of numbers that has a specific upper limit. For argument's sake - let's say there are only 2 numbers: 0 and 1. So 1+1 is undefined because there is no number 2...
Does Gödel incompleteness theorem hold for these numbers? What about if the upper limit is higher - 100 or a googol?
So the question then is twofold - first, would an arithmetic limited to a sufficiently small subset of natural numbers be complete. And second, if so how high can we make that limit before it becomes incomplete?
I'm not really sure what you're asking. Certainly for any natural number $n$, the structure $\mathbb{N}_n$ consisting of the natural numbers up to $n$ (with $+$ and $\cdot$ interpreted relationally in order to be a legitimate structure) is trivially decidable, so in that sense Godel doesn't apply to it(s theory).
But there's a huge issue here: checking whether a sentence of length $<n$ is true in $\mathbb{N}_n$ requires more than $n$ steps in general. So the decidability of $\mathbb{N}_n$ is not satisfying from an ultrafinitist standpoint, since the completeness itself isn't "justifiable within $\mathbb{N}_n$." Meanwhile, the non-ultrafinitist won't be impressed either since $\mathbb{N}_n$ is too limited a structure to treat even basic arithmetic. So this dodge doesn't seem satisfying from any perspective.
It's worth noting that Godel's second incompleteness theorem can be avoided non-trivially with a trick of arguably ultrafinitist spirit: there are theories which interpret a decent amount of arithmetic but which prove their own consistency, and a key component of such theories is that they do not prove that multiplication is total, so in a weak sense have ultrafinitistic flavor. These were first studied by Dan Willard.1
However, Godel's first incompleteness theorem still applies to these. Indeed, no theory which can prove each true quantifier-free sentence about $\mathbb{N}$ and, for each $k\in\mathbb{N}$, can prove $$Init_k:\quad\forall x(\underline{k}<x\vee\bigvee_{0\le i\le k}x=\underline{i})$$ (where "$\underline{m}$" denotes the numeral corresponding to $m$) is going to be complete, consistent, and computably axiomatizable (see e.g this paper of Ritter). Note that such theories are not required to prove that multiplication is total, or that $<$ is a linear ordering of the universe, or so on: they are truly quite weak.
1Dan Willard: Self-Verifying Axiom Systems, the Incompleteness Theorem and Related Reflection Principles, Journal of Symbolic Logic 66 (2):536-596 (2001). DOI: 10.2307/2695030, JSTOR, author's website