From the linear algebra books that I've encountered, they either discuss exclusively about finite-dimensional vector spaces, or assume that the reader already knows about infinite-dimensional vector space, Hamel basis, etc.
What books explain the concept of infinite-dimensional vector space and its structures?
Infinite-dimensional vector spaces are general enough that there is not a whole lot of interesting theory about them. To get anywhere you need to make some restrictions to the subject.
Probably where you want to go is functional analysis - the study of (usually infinite-dimensional) vector spaces with topological structure. As usual, Lang has an introductory book - Real and functional analysis - that could be helpful.