Understanding implicit differentiation with concepts like "function" and "lambda abstraction."

Question

In high school, we learned to reason like so:

$$(*) \qquad \frac{d}{dx}(x^2+x) = \frac{d}{dx}(x^2)+\frac{d}{dx}(x) = 2x+1$$

Now that I know more, I can "reanalyze" this chain of reasoning using ideas that I have more faith in, like "function" and "lambda abstraction." We begin by defining $\nabla (f)$ as the derivative of $f$. Then the above chain of reasoning becomes:

$$\nabla\mathop{\lambda}_{x:\mathbb{R}}(x^2+x) = \nabla\mathop{\lambda}_{x:\mathbb{R}}(x^2)+\nabla\mathop{\lambda}_{x:\mathbb{R}}(x) = \left(\mathop{\lambda}_{x:\mathbb{R}}2x\right) +\left(\mathop{\lambda}_{x:\mathbb{R}}1\right) = \mathop{\lambda}_{x:\mathbb{R}}(2x+1)$$

So I can confidently say that I understand $(*)$, because I can reanalyze it in terms of ideas that I have a lot of faith in, like "function" and "lambda abstraction."

Onwards.

In high school, we also learned a pattern of reasoning that was referred to as "implicit differentiation." It looks a bit like so:

Suppose $y^2+x = x^2+y.$

Then: $$\frac{d}{dx}(y^2+x) = \frac{d}{dx}(x^2+y).$$

$$\therefore 2y \frac{dy}{dx}+1 = 2x+ \frac{dy}{dx}$$

$$\therefore (2y-1)\frac{dy}{dx} = 2x-1$$

$$\therefore \left(\frac{dy}{dx} = \frac{2x-1}{2y-1}\right) \vee (y = 1/2)$$

Unfortunately, I still have absolute no idea what any of this means.

Question. How can we reanalyze implicit differentiation using respectable concepts like "function" and "lambda abstraction" and "limit", and without using concepts such as as "dependent variable" and "independent variable" and "differential."

Edit. It seems to be unclear what I'm looking for. I'm not looking for handwaiving and intuition. I want something as formal as possible, with the minimum of handwaiving. For example, if you're going to "switch" semantics so that $y^2+x=x+y^2$ is no longer a condition on pairs of points $(x,y) \in \mathbb{R}^2$ but instead becomes a condition on smooth functions $(a,b) \rightarrow \mathbb{R}^2,$ you should make that completely clear, introduce notation for the set of paths, etc., and the rest of your answer should be phrased in terms of this notation. Write definitions. State theorems if relevant. Tell me the domains and codomains of all your functions. I want the absolute technical logical and/or set-theoretic nitty-gritty here.

Answer 1

The context of this is that we're looking at the subset of the plane $$ \{(x,y)\in\mathbb R^2 \mid y^2+x=x^2+y \}$$ which is a curve of a certain shape, and we're trying to find a parameterization of that curve -- or at least part of it -- that is, nontrivial functions smooth functions $x(t)$ and $y(t)$ such that $y(t)^2+x(t)=x(t)^2+y(t)$ for all $t$ in an interval.

In order to simplify things before we start, we decide to choose the parameterization such that $t$ is just the $x$-coordinate -- in other words, $x$ is now the identity function, so we can forget about the $t$ and $y$ is now secretly a function of $x$ (whereas $x$ itself is again just a number)!

With lambdas, the original equation is then $$ \tag{*} (\lambda x.(y(x))^2+x) = (\lambda x.x^2+y(x)) $$ where the unknown is now the function $y:\mathbb [a,b]\to\mathbb R$ on some yet-to-be-identified interval $[a,b]$.

Implicit differentiation now applies your $\nabla$ to both sides of $\text{(*)}$, and after some routine manipulations we find that $\text{(*)}$ implies is $$ \forall x\in[a,b]. \left((\nabla y)(x) = \frac{2\cdot x-1}{2\cdot y(x)-1} ~\lor~ y(x)=\frac12\right) $$ We can then use that to find $y'(x)$ -- and thus the tangent to the curve -- at a particular point that we already know is on it, without needing to derive an explicit expression for $y$. (Doing so would be an easy application of the quadratic formula in this particular case, but the technique also applies in cases where this is not easy).

Answer 2

The equation you wrote defines a curve, $y^2 + x = x^2 + y$.

On this curve (near any point where there is no vertical asymptote) $y$ is a differentiable function of $x$. This is the implicit function theorem, but without using that theorem, it is still a reasonable geometric assumption that is not directly related to the worries in the question. Assume it is true.

Because $y$ is a differentiable function, single-variable calculus, such as differentiation and integration with respect to $x$, apply as usual.

Replace $y$ by the notation $y(x)$ if it helps in accepting $y$ as an ordinary function of $x$. And then $\frac{dy}{dx}$ denotes $y'(x)$. If you want to think of differentiation as an operator then applying this operator to $y(x)$ yields $y'(x)$ (or $dy/dx$ when applied to $y$).

The calculations of "implicit differentiation" then are nothing more than an instruction to separately compute derivatives of two particular functions made from $x$ and $y(x)$. Or one particular function, if the equation of the curve was written as $F(x,y)=0$ instead of as LeftSide$(x,y)$ = Rightside$(x,y)$. It doesn't matter, in that single variable calculus works the same way whether we differentiate one function or a hundred. If the hundred satisfy some equations, that will lead to some equations between the differentiated functions, nothing mysterious there.

Keeping the same running example, the functions are $y(x)^2 +x$ and $x^2 + y(x)$, and they happen to be equal because $y(x)$ was selected to make that true. The end result is an equation relating $y'(x), x$ and $y$.

From this equation one can solve for $y'(x)$ in terms of $x$ and $y$. At any given point on the curve, $(x_0,y_0)$, the formula for $y'$, evaluated at $x=x_0$ and $y=y_0$, will correctly calculate the slope of the tangent line to the curve at that point.

Nowhere have mystery words like "(in)dependent variable" and "differential" appeared. Only one-variable calculus and the assumption that $y$ is expressible as a differentiable function of $x$ near almost all points.

The exceptional cases with vertical asymptotes actually will also work fine with the formalism of implicit differentiation, and only look like exceptions here because I used a notation that assumes lines have finite slope. The more general explanation that does not assume that is slightly longer, but it does not require the mystery words any more than what was just written down above. There are some genuine and unavoidable complications about self-intersections of the curve, or points where its graph is not smooth.

In the larger picture, there is a theory of "smooth manifolds", that provides the infrastructure for talking about almost-everywhere-smooth finite dimensional objects (like a curve) inside larger dimensional spaces (like the plane) and differential calculus on them. This is the context for the implicit function theorem and its generalizations and the words written above like "near any point".

Answer 3

Multi-variable differentiation

Let $f:\mathbb R^n\rightarrow \mathbb R$. Then we can define $\nabla_i$ to be "differentiation with respect to the $i$th argument". Formally:

$$\nabla_i f = \mathop{\lambda}_{(x_1,\dots,x_n):\mathbb R^n}\lim_{h\rightarrow 0}\frac{f(x_1,\dots,x_i+h,\dots,x_n)-f(x_1,\dots,x_i,\dots,x_n)}{h}$$

defined only where this limit exists.

If $U\subseteq \mathbb R^n$ is open and $f: U \rightarrow \mathbb R$ then $\nabla_i$ can still be defined with the same defintion.

$$\nabla_i f = \mathop{\lambda}_{(x_1,\dots,x_n):\mathbb U}\lim_{h\rightarrow 0}\frac{f(x_1,\dots,x_i+h,\dots,x_n)-f(x_1,\dots,x_i,\dots,x_n)}{h}$$

Say $f$ is continuously differentiable at $u\in U$ if for all $i$ we have that $\nabla_i f$ is defined and continuous for some neighbouhood of $u$.

Differentiation on a set with coordinates

Let $X$ be a set and let $c_1,\dots,c_n$ be functions $X\rightarrow \mathbb R$. We think of these as coordinates on $X$. Putting them together we get a map $c:X\rightarrow\mathbb R^n$, i.e. $c(x)=(c_1(x),\dots,c_n(x))$. Now if $c$ happens to be an injection with its image being an open set $U$, and we have a function $f:X\rightarrow \mathbb R$, then we may define "differentiation with respect to the coordinate $c_i$ holding the other $c_j$ constant" to be given by

$$\nabla^c_{c_i} f=\nabla_i (f\circ c^{-1})$$

One can of course verify that these $\nabla^c_{c_i}$ operators obey the rules you would expect of $\frac{\partial}{\partial c_i}|_{c_j}$. In particular since the $c_i$ are themselves functions $X\rightarrow \mathbb R$ we have defined $\nabla^c_{c_i} c_j$ and can check that it is equal to $\delta_{ij}$.

The implicit function theorem

This theorem tells us one particular set of circumstances where the above definitions apply.

Suppose we have a function $F:\mathbb R^{n+m}\rightarrow \mathbb R^m$, and we let $S\subseteq\mathbb R^{n+m}$ be the subset on which $F$ is zero. Note that the natural coordinates on $R^{n+m}$ give are $n+m$ functions $x_1,\dots,x_{n+m}:\mathbb R^{n+m}\rightarrow \mathbb R$. For $i=1,\dots,n$ let $c_i=x_i|_S$ and define $c:S\rightarrow \mathbb R^n$. Also, note that we may view $F$ as $m$ separate functions $F_1,\dots, F_m:\mathbb R^{n+m}\rightarrow \mathbb R$.

Now let $s\in S$ and suppose each $F_i$ is continuously differentiable at $s$. Then the IFT says that if the matrix with entries $\nabla_{n+i}F_j(s)$ for $i,j\leq m$ is invertible then there is an open neighborhood, $X$, of $s$ in the subspace topology on $S$ such that the map $c|_X:X\rightarrow \mathbb R^n$ is injective with image some open set $U$.

Let $\tilde S$ be the subset of $S$ where this condition holds. Then at any $s\in\tilde S$ and any function $f:S\rightarrow \mathbb R$ we may define $\nabla^c_{c_i} f=\nabla^{c|_X}_{c_i|_X} f|_X$.

The IFT also says that if $f$ is the restriction to $S$ of some continuously differentiable function on $\mathbb R^{n+m}$ then $f$ is continuously differentiable as a function on $\tilde S$.

Example

You give the example of implicitly differentiating $x^2+y=y^2+x$ with respect to $x$. Here $n=m=1$, $F=F_1=\mathop{\lambda}_{x,y:\mathbb R}x^2+y-y^2-x$, and our variables are $x_1=x$ and $x_2=y$. So

$$\nabla_{x_{1+1}}F_1=\mathop{\lambda}_{x,y:\mathbb R}1-2y$$

which is an invertible "matrix" whenever $y\neq 1/2$. So $\tilde S=S\setminus \{(x,1/2)|x\in\mathbb R\}$. Since $x^2+y$, $y^2+x$, $x$ and $y$ are all continuously differentiable $\mathbb R^2\rightarrow \mathbb R$ we therefore have that they are continuously differentiable on $\tilde S$ and so

$$\begin{align*} x^2+y&=y^2+x \Rightarrow\\ \nabla^c_c(x^2+y)&=\nabla^c_c(y^2+x) \Rightarrow\\ 2x+\nabla^c_cy&=2y\nabla^c_cy+1 \Rightarrow\\ \nabla^c_cy&=\frac{2x-1}{2y-1}\text{ or }y=1/2\\ \end{align*}$$

Where $c$ is the restriction to $S$ of $x$.