I am self-learning about Godel's incompleteness theorems and think that this analogy is good:
When trying to prove that infinite primes exist we assume finite number of them exist and then we arrive at a number that cannot be built by those primes.
Similarly we assume a finite number of axioms exist and arrive at a statement that cannot be proved by those axioms.
So axioms in a theory are small indivisible units that can be used to build bigger things but for a complete picture of the theory we might need infinite number of them.
PS: This question might be vague, but I want to know if this analogy is erroneous.