Examples of spaces I've come across include vector spaces, inner-product spaces, and metric spaces. Examples of structures I've met include rings, fields, and groups.
I have always understood spaces and structures to be glorified sets - sets that have something extra associated with them, like a distance function, addition function, or multiplication function that satisfy certain criteria.
For example:
A set is a metric space if it's associated with a distance function satisfying a few conditions.
A set is a field if it's associated with two binary operations satisfying the field axioms.
There are definitely similarities in these definitions. So is there a fundamental difference between a "space" and a "structure"?
What is the guiding principle behind calling something a space or a structure? For instance, we could name the field a "field space", but we don't - why not?
A (mathematical) space is a set endowed with certain "structure" on it. It doesn't come quite often to talk about the distinctions between "space" and "structure" in daily studies since they refer to different aspects of points of interest.
For example, a vector space is a set endowed with the algebraical structure, and a topological space is a set endowed with the structure concerning the collection of its subsets. Usually the structure we observe serves as a list of axioms to the given set. Furthermore, a map from one set to another such that it "preserves" structure gives us a way to identify the image set as a space like the domain set. Thus we are usually interested in how to give structure to a set to make it a space and identifying unfamiliar sets with the one we know.
Furthermore, considering the "guiding principle" of a naming "field", I prefer to leave it as a historical reason because mathematically a "vector space" is more like an algebraical object, yet people still give it the nice geometric name "space".