At least in physicist's thinking, information, vaguely, is something that allows one to select a subset from a set.
Say, a system can be in states A and B, we have done a measurement on it (extracted information), then it is in either A or B. Now we are able to say, in which state among all the possible states is the system in.
Now, consider numbers, say number e. One can write many digits to specify the boundaries of an interval to which e belongs. In fact, arbitrarily good precision can take arbitrarily large amount of information to specify the decimal representation of the number.
Now, however, one can also write an expression for e, say $e = \sum_{n}\dfrac{1}{n!}$, and it seems like e is defined to an arbitrary precision straight away. That is to say, here we have something, for which we would have needed infinite amount of information.
The question then: does this expression contain infinity of information/any information at all? Or, can information be defined at all for expressions?
One might surely argue that given an expression, one still has to perform infinite number of evaluations to obtation a decimal expression of the number. But then a number of other questions would arrive: "Is it evaluations of faculties and additions then that produce information?", "Is the information only about decimal representations of numbers, but not the numbers themselves?", and perhaps many more.