Since the entire observable part of the universe can only be in a finite number of physically distinguishable states, it seems rather strange that an efficient formal description of the universe would require the notion of uncountable sets.
Just like we're better off accepting that infinitesimal numbers do not exist, rather than developing an unwieldy formalism in the form of non-standard analysis, one can consider if we would be better off doing away with uncountable objects.
From what I have read, people have argued in similar ways, but not much progress has been made. So, what is holding up the development of a better analysis based on only computable quantities?
I can see what you mean by doing away with the uncountable reals, just to discard all the superfluous ones that make the set uncountable rather than countable, but there are a couple things that should be noted. Also, you seem to imply this can be accomplished by using only the computable reals. That is known to not work, which I discuss later, so I will focus on doing away with the uncountable reals, as you say.
First, the reals are uncountable as defined in standard set theory relative to that theory. Countability is a relative thing, the existence of a bijection depends on what functions are available. So whether or not a bijection exists between the reals and the integers depends on what bijections the theory makes available. As a matter of fact, it is generally accepted there is a countable model of ZFC, which would define the set of reals to be countable (relative to some meta-context, not relative to the ZFC in which the set is defined). However, that does not help much, because reals defined as part of such a model still leave real numbers that cannot be identified in any meaningful way.
Second, it is not all that clear what reals make the set of reals uncountable. What countable subset do you accept, while rejecting the others? This is tricky. For example, if you decide to only accept real numbers defineable in some language, you can use a straightforward definition that goes through definable numbers in that language to unambiguously define a real number not defineable in the language.
Alternatively, If you require each digit to be computable, you loose the least upper bound property, which is a pretty big deal.
So, you see, there is no obvious way to reject the superfluous reals. To be clear, it is not known to be impossible to do so without major problems, it is just not a simple task.
So, you ask what is holding up progress in defining reals in a way that makes them countable and retain all their expected properties? I'd say, first off, that there are not enough people asking this question! Additionally, it is a really hard problem. You can be sure that thousands of hours of brilliant thought have gone into it, especially back when uncountability was viewed with suspicion, in the early days of the idea.
I absolutely agree with you, I think the reals would more naturally describe reality if they were defined so as to be inherently countable, but at the same time retain all their properties. And it is absolutely not known that this is impossible. It is just that most mathematicians are comfortable with the reals as they stand, and not a lot of work is being done on making this happen.