Here are the definitions of computable. There seems to be no direct analogy to measurable functions. Why can't we make similar definitions to measurable (for computable)? By that I mean, perhaps the set operations are exchanged for others or added to.
For example. Clearly any finite union of computable sets would also be computable, as well as the elementwise summation over the sets.
I think computability should be about construction not decidability. Then a set $A$ is computable in the traditional sense if $\chi_A$ is constructible.
First of all, at a higher level I think it's worth mentioning the extremely well-developed analogy between the hyperarithmetic hierarchy of sets of natural numbers and the Borel hierarchy of sets of reals; this was originally studied by Addison, and has since then been a fundamental pillar of descriptive set theory and computability theory. This is a bit high-level, but it is an analogy I think you will find very nice.
More specifically directed at your question: there is a "$\sigma$-algebra like" structure in computability, but it's not on the computable sets - instead, it's on the computably enumerable sets. Namely, the c.e. sets are closed under computably enumerable unions: if $X\subseteq\mathbb{N}$ is c.e., then $\bigcup_{x\in X}W_x$ is also c.e. So the c.e. sets are closed under "simple" countable unions.
Clearly we need the word "simple" here, since every set of natural numbers is a union of countably many computable sets (for instance, its singleton subsets). Also, we can't apply this trick to the computable sets: we can take a computable union of computable sets and get a non-computable set. Specifically, fixing your favorite noncomputable c.e. set $W_e$ with a given computable enumeration $W_e=\{n_i: i\in\mathbb{N}\}$, there is a computable function $f$ such that $W_{f(i)}=\{n_i\}$ for each $i$. This means that $W_e$ is a computable union of computable (indeed, single-element) sets. In fact, this shows that the c.e. sets are exactly what you get by starting with the finite sets (with canonical indices) and closing under computable unions! Note that "computable union" is defined in terms of indices.
This reflects a common idea in computability theory: the computably enumerable sets are often more well-behaved than the computable sets. While the class of computable sets is closed under complements and the class of computably enumerable sets is not, as we delve deeper the class of computably enumerable sets quickly becomes the more nicely structured of the two, in most cases. And going back to functions, note that a function is computable iff its graph is computably enumerable, and in fact a "computably enumerable" function is just a computable function, so we are still looking at computable functions when we think of c.e. sets.
Now your sentence
suggests that you may also be interested in a definition of computable sets/functions/etc. which describes them as those sets/functions/etc. gotten by closing a family of simple objects under some fixed set of operations. Such a characterization does exist: for example, the definition of $\mu$-recursion constitutes such a characterization! We begin with the constant functions and projections, and close under primitive recursion, substitution, and minimization. And this is not the only one: $\lambda$ calculus and general recursion also fit this pattern.
Incidentally, I strongly disagree with your above-quoted sentence - I think computability is fundamentally about deciding basic facts about sets, and other characterizations are useful tools for studying computability but not its a priori definition.
Of course, this is highly subjective, but let me situate this opinion with some brief history. The initial notions of computation - Church's $\lambda$-calculus, Kleene's $\mu$-recursion, and Godel-Herbrand's general recursive functions - were all of a "construction-y" flavor (see e.g. Church's paper on the Entscheidungsproblem. There were those who argued that these definitions captured the informal computability exactly; however, this was not universally accepted (for instance, Godel was initially doubtful). This doubt was probably motivated by the ease with which previous construction-y notions (e.g., primitive recursion) could be "diagonalized out of" to produce informally computable functions not in that class (e.g., the Ackermann function). By contrast, Turing's definition (which was fundamentally non-hierarchical, but rather revolved around the notion of decidability) was quickly accepted.