I'm studying model theory and mathematical logic by myself and found two different definitions of a theory: (1) by a formalized theory $\mathscr{T}$ we shall understand any triple $(\mathcal{L},C,\mathcal{A})$, where $\mathcal{L}$ is a (first order) language, $C:\mathcal{P}(F)\rightarrow\mathcal{P}(F)$ is a consequence operation ($F$ is the set of all formulas of $\mathcal{L}$) and $\mathcal{A}$ is a subset of $F$ (found in The mathematics of metamathematics by Rasiowa and Sikorski); (2) a (first order) theory $\mathscr{T}$ of $\mathcal{L}$ is a collection of sentences of $\mathcal{L}$, where a sentence is a formula with no free variables (found in Model theory by Chang and Keisler).
My questions are (1) how many definitions of theory are there?, (2) what is the most common (used) one now? and (3) is there a reason for the difference or evolution in the definition?
There are numerous definitions of "theory", too many to count. This is a general pattern in logic: different authors often have slightly different definitions of the same concepts. For the beginner, it is important to think about both the formal definitions and the ideas behind them, because the ideas will be shared by many authors even if their formal definitions don't match.
There are a few key distinctions you can use to get a sense of where an author is coming from, and what kinds of ideas are important to them.
The first key distinction has to do with which logic the author is working in (or whether there is a fixed logic at all). In the first definition from the question, the authors allow the "theory" to include its own consequence relation - essentially, its own set of deduction rules. The second definition takes the theory to be just a set of sentences. In that case, the deduction rules will have to be known from context, and most likely will be the standard rules of first-order logic.
The clue that the authors might take these approaches comes from the topics of their books. One book is a general book about logic, which might well want to look at various deductive systems. The second book is about model theory, where it is common to just work in first-order logic.
The second key distinction is between definitions that allow a theory to be essentially any set of sentences - so, essentially, a set of axioms - versus definitions that require the theory to be closed under the relevant deductive system.
In model theory, because the set of sentences true in a given model is automatically closed under the relevant deductive system, it is relatively common to just assume that the theory is closed under that deductive system in the first place. Then, instead of having to say that various sentences are provable from the theory, the model theorist can just say the sentences are in the theory.
In other contexts, especially when computability is involved, it is less common to make the assumption that theories are deductively closed. The set of consequences of a set of axioms is not, in general, computable from the set of axioms. So, if we want to track what is computable from what, we have to take care when we replace a set of axioms with its set of consequences. This is particularly relevant because the sets of consequences of formal theories such as Peano Arithmetic or ZFC set theory are not computable.