If you are integrating a function that is holomorphic except at a finite set of singularities around a contour, then you can deform the contour as long as it doesn't pass through any of the singularities. My lecture notes and wikipedia state that a contour has to have a continuous derivative on each piece and a finite number of pieces, and this is where my confusion comes in.
If we had a nasty contour (doesn't fit the definition above) then I don't see why we couldn't continuously deform it to be a nice contour. I see that piecewise continuously differentiable with a finite number of pieces means that we could always define one sided derivatives at each of the joining points, and this is needed when we substitute in the parametrisation of the curve, but surely the value of the integral doesn't depend on how we parameterise it.
I read this, but it still doesn't answer my question. If you had a function on a nice contour with a well defined value and continuously deformed the contour to a nasty contour, then according to my intuition the value shouldn't change. Some other thoughts I had was that the nice contours are connected and compact and these properties are needed, and continuous deformation would preserve these properties. If it was impossible to have a compact and connected curve that is not differentiable at infinitely many points, then this would answer my question as it means we wouldn't be able to continuously deform a nice contour to such a nasty a contour.
It is certainly possible to have a continuous curve $\gamma$ (whose image is therefore connected and compact) which is nowhere differentiable; see for instance https://en.wikipedia.org/wiki/Weierstrass_function. For such a curve, the contour integral $\int_{\gamma} f(z)\,dz$ has no meaning. Moreover, it is possible to continuously deform any smooth curve into such a $\gamma$; indeed, any given curve can be perturbed infinitesimally into one that is nowhere differentiable (just imagine adding very small sawteeth).
That's not all that can go wrong, though. For the following example, let $f$ be your favorite non-constant holomorphic function, and let $a,b \in \mathbb{C}$ be two points such that $f(a) \ne f(b)$. Let $\sigma : [0,1] \to \mathbb{C}$ be your favorite smooth curve joining $a$ to $b$. Then, as you know, $\int_{\sigma} f(z)\,dz = f(b) - f(a) \ne 0$.
Now let $c : [0,1] \to [0,1]$ be the Cantor function. It is continuous and satisfies $c(0)=0$, $c(1)=1$; and at almost every point $t$ its derivative $c'(t)$ exists and equals 0. (If you haven't studied measure theory, "almost every" means that the set of points where this does not happen is so small that it can't affect the value of any integral. It's analogous to the fact that modifying the value of a function on a finite set of points does not change its integral; here the set in question is infinite but still has "zero length".)
Let $\gamma(t) = \sigma(c(t))$. Then $\gamma$ is a reparametrization of $\sigma$: it is still continuous, and $\gamma(0)=a$, $\gamma(1)=b$. By the chain rule, for almost every $t$, $\gamma$ is differentiable at $t$ and satisfies $\gamma'(t) = \sigma'(c(t)) c'(t) = 0$. Thus we would have to say $$\int_\gamma f(z)\,dz = \int_0^1 f(\gamma(t))\,\gamma'(t)\,dt = 0$$ which is certainly not the same as $\int_\sigma f(z)\,dz$.
So your intuition that "surely the value of the integral doesn't depend on how we parameterise it" is wrong, unless some conditions are placed on which parametrizations are allowed. Continuity alone is not enough.
If you like, you can "continuously deform" $\sigma$ into $\gamma$ by letting $\gamma_s(t) = \sigma((1-s)t + s c(t))$. Then $\gamma$ is jointly continuous in $s,t$, with $\gamma_0 = \sigma$ and $\gamma_1 = \gamma$. Indeed, all the curves $\gamma_s$ have the same image, and are all just reparametrizations of each other.
So you definitely need some assumptions on the curve $\gamma$ in order to make things work reasonably. It can be weakened somewhat from piecewise $C^1$; a more general condition is that the curve $\gamma$ should be absolutely continuous. This is basically the necessary and sufficient condition for the fundamental theorem of calculus to work properly, which is the key ingredient in all your theorems about contour integration. The Cantor function is a classical example of a continuous function which is not absolutely continuous, and the curve $\gamma$ we constructed above is not absolutely continuous either.
This topic is not usually brought up in introductory calculus or complex variables courses. The definition of absolute continuity is complicated; one needs measure theory to prove the needed results in the absolutely continuous setting; and the stronger simpler assumption of piecewise $C^1$ covers the vast majority of cases that arise in practice. But it is pretty standard at a more advanced level.