I'm just using this example to gain a bit of intuition concerning higher categories (about which I have essentially no knowledge); because it's the first example that came to my mind, and because it seems like an interesting/basic toy model.
Here are my thoughts : we consider a topological space $X$, its points are objects, the paths between points are the $1$-morphisms, and homotopies between paths are $2$-morphisms. For starters I'll stop here.
We have a clear notion of composition, however it's pretty clear that for paths for instance, composition is not associative. But it is associative up to a $2$-morphism, in both directions: but we can't really say that it's up to a $2$-isomorphism because if two (homotopic) paths are different, the composition of the two homotopies (one in each direction) will not be the identity homotopy (and of course, composing any homotopy with the identity homotopy will not yield the same homotopy, which is another, somewhat related issue).
But we may say that the composition of the two homotopies is the identity, up to a $3$-morphism (again in both direction): that's where the higher morphisms come in. However, still we are stuck with the issue that this $3$-morphism cannot be a strict isomorphism.
So if we try to climb up the ladder of $n$-morphisms, we never get a strict isomorphism : those don't exist (except for identities); and to be honest it wouldn't make much sense to talk about isomorphisms when composition with the identity never yields the same result.
My questions about this are the following: is there any interesting way to axiomatize this sort of behaviour, where composition is associative up to higher morphisms, and composition with identity is doing nothing up to higher morphisms ? For our toy model of a topological space, what can we do to stop this escalation at a certain stage $k$ to say that for $k$- and higher morphisms composition will be strictly associative and composition with identity will be strictly doing nothing (to get an $(\infty, k)$-category, if I understand well) ? And finally where can I learn more about this kind of stuff, where the author(s) doesn't already assume that we have some knowledge about higher categories (say I have some background in ordinary category theory) ?
One way to make this precise is to say that you're after the $\infty-$groupoid associated to your space. It's a "groupoid" of some kind because, as you say, every morphism is "invertible" up to a higher morphisms. It's actually a central thesis of higher category theory that $\infty$-groupoid theory and the homotopy theory of spaces should be equivalent-this is the "homotopy hypothesis." It is a proven theorem in the most successful formulations of higher category theory. So your example of an $\infty$-groupoid is, in an appropriate sense, the only possible example!
From a space, you can also derive a strict category of points and homotopy classes of homotopies. This is again a groupoid, the fundamental groupoid of the space. It's a well known generalization of the fundamental group. Similarly one level higher, Qiaochu was describing the fundamental 2-groupoid. It's harder to make the fundamental 2-groupoid strictly unital and associative than the fundamental groupoid, since as you say composition of paths is nonassociative as usually defined. But it can be done. However, it is actually known to be impossible, under reasonable hypotheses, to make a strict 3-groupoid of a space! So higher category necessarily studies weakly associative and unital structurures.
The availability of reasonably introductory books on this subject is getting better quickly. Cisinski: http://www.mathematik.uni-regensburg.de/cisinski/CatLR.pdf and Riehl and Verity: http://www.math.jhu.edu/~eriehl/ICWM.pdf both have draft books. These are two very different approaches to the subject, so you might take a look at either. Fair warning that things are probably more complicated than most people expect when first thinking about this topic, no matter which approach you take. In reverse chronological order, there are also books by Bergner, Simpson, Lurie, and Leinster, of which Lurie (higher topos theory, his first book) is the canonical reference. But even Leinster only goes back to '04. It's a young field, and not everything is close to settled.