Let $f:\mathbb{R}^n\to\mathbb{R}$ be nice-enough function, say at least $C^1$. The argument I saw during my multivariable calculus course on why the gradient of $f$ is orthogonal to a given level set of $f$ was that if $\gamma:(-t,t):\mathbb{R}^n, t > 0$ is a differentiable curve in a level set $L_c\equiv \{x\in\mathbb{R}^n\mid f(x) = c\}$ such that $\gamma(0) = p\in L_c$, then by the chain-rule $f(\gamma(0)) = c \implies \nabla f(\gamma(0))\cdot \gamma'(0) = 0$. That is, the gradient of $f$ is orthogonal to the tangent of any curve passing through an arbitrary point $p\in L_c$.
This was not really discussed further and I got curious that is the reason on why then the gradient of $f$ is orthogonal the fact that the tangent vectors $\gamma'(0)$ of all possible path $\gamma$ certainly span the tangent space of $L_c$ at $p$. And since $\nabla f$ is orthogonal to all such vectors, then it is orthogonal to the tangent space and hence the level set itself? So in e.g. $n = 2$ case we'd have that $\nabla f$ is orthogonal to the tangent plane at every $p\in L_c$ and thus it has to be orthogonal to the level curve itself.
Edit: Question: Is the above reasoning correct, assuming that the gradient does not vanish at the said points?