It is written that a bounded operator, A, on a separable infinite dimensional Hilbert Space, H, can be represented by an infinite matrix using an orthogonal basis, $α_{ij}= (Ae_j,e_i)$, where $Ae_i=∑^∞_{j=1}α_{ij}e_j$
Firstly, Can't all linear operators be represented by matrices, why is this only true for bounded operators?
Secondly, isn't this definition circular? $Ae_j$'s definition relies on $α_{ij}$ but $α_{ij}$'s defintion relies on $Ae_j$
Yes, but an orthogonal basis in a Hilbert space is not a true basis in the sense of abstract vector spaces (sometimes called a Hamel basis in functional analysis). To be a true basis, it would have to be the case that every vector could be represented as a finite linear combination of the basis vectors. A Hilbert space contains vectors such as $\sum \frac{1}{i} e_i$ for which this is not the case.
If you had a true basis, you could represent a linear operator as a matrix in terms of it, but true bases are not easy to describe for a Hilbert space. For one thing, they must always be uncountable (this follows from the Baire category theorem) and so your matrix would have uncountably many rows and columns.
A matrix with respect to an orthonormal basis only needs countably many rows and columns; but at the level of linear algebra, it only tells you what the operator does to vectors which are finite linear combinations of the basis vectors. The space of such vectors is dense, and so if you also know that the operator is continuous (i.e. bounded), then you also know what the operator does to all other vectors. If the operator were not bounded, then you could not deduce from the matrix what happened to a vector like $\sum \frac{1}{i} e_i$ above.
It's two definitions. If you have the operator, the first definition tells you how to define the matrix; you'll have to have some other description of the operator in order to use it. Likewise, if you have the matrix described in some form, the second definition tells you how to define the operator.