From Griffiths, Introduction to Electrodynamics, pg. 48:
In the image above, the author makes the statement in the title of this question. But shouldn't it be
$$\pm \int_{-\infty}^{\infty} f(y/k)\delta (y)\frac{dy}{k} = \pm \frac{1}{k^2} f(0) \text{ ?}$$
Why can't you say that $f(y/k) = \frac{1}{k} f(y)$

As MichaelHardy notes, it is not true that $f(y/k) = f(y)/k$, in general.
Let $h(x) = x/k$ and notice that $h(0) = 0$. Then, when we change variables $kx = y$, we have that $k\,dx = dy$. Hence, when we make the substitution in the integral, we have the composition $f\circ h$ in the numerator: $$ \int_{-\infty}^\infty f(x)\delta(kx)\,dx = \int_{-\infty}^\infty \frac{f(y/k)}{k}\delta(y)\,dy = \int_{-\infty}^\infty \frac{(f\circ h)(y)}{k}\delta(y)\,dy= \frac{1}{\vert k\vert}(f\circ h)(0) = \frac{1}{\vert k\vert}f(0) $$ since $\int_{-\infty}^\infty g(t)\delta(t)\,dt = g(0)$, by property of the delta function. We have $\pm 1/k$ as a result of following through with our change of variables and noticing the cases where $k \lt 0$ and $k \gt 0$. I think you should see that we don't get $\pm 1/k^2$.