I'll soon start my undergrad. studies in physics and because of that I picked up Spivak's Calculus a while ago to get a solid foundation in single-variable calculus before I start my studies. However, I have read that a book such as Spivak's Calculus isn't a particularly good choice for a physics student - not that one doesn't learn useful stuff from it but simply because of the fact that that amount of mathematical rigorousness probably isn't neccesary for a physics student (at this time in my career) and my time would be spent more wisely with a different book more relevant to physics. I've chosen to ask this question in the mathematics community too, since there probably are more of you who yourself have been through Spivak.
EDIT: In short, should I continue with Spivak or should I find a different book (perhaps entirely different subject) which is more relevant to physics?
I have nothing but pleasant memories of my time with Spivak, and I am nostalgically delighted that he is still used.
To some extent it depends on you and your personal taste and temperament. Some people want nothing but a box of tools to memorise, along with instructions for which to pick when. Others aren't satisfied unless they know how things work — indeed, they can't understand unless they do know that.
I am more mathematically inclined because I am lazy, hate memorising and hate facts. This is a great saving in brain capacity because I can invent the bits of maths I need as and when I need them. If a result comes out as $ \frac{0}{0} $, I don't apply l'Hôpital's Rule, I take the limit as $x$ tends to 0 (which is, of course, the same thing but doesn't involve magic incantations).
You need to know what style suits you best, and only you can discover that. I would say, try Spivak and see how you get on. If he takes time proving rigorously that this or that thing works, and you aren't interested, skip. It's allowed: indeed, it's compulsory.
If it all gets too much for you, drop it and find something else. But you will probably find that a well-skipped Spivak does all you need. And it's good to get thinking in terms of first principles early on, because a first-principles approach will save you a great deal of effort in your physical studies as well.