The question is rather general, but let me give a specific example.
In thermal physics we have the following identity involving differentials: $$T\,dS = dE + p\,dV$$ where $T$ = temperature, $S$ = entropy, $E$ = internal energy, $p$ = pressure, $V$ = volume of a system.
In many physics books on thermal physics one can find transformations like this: $$T\,dS = dE + p\,dV \iff dE = T\,dS - p\,dV$$
It's not clear for me why we can do such transformations. For me $$T\,dS = dE + p\,dV \iff dS = \frac{1}{T}\,dE + \frac{p}{T}\,dV$$ means that for each point $(E, V)$ $dS(E, V)$ is a linear functional with the matrix representation $$\begin{bmatrix}\frac{1}{T(E, V)} & \frac{p(E, V)}{T(E, V)}\end{bmatrix}$$ which maps a two-dimensional vector $\begin{bmatrix}\Delta E \\ \Delta V\end{bmatrix}$ to a scalar value $\Delta S$.
$dE, dV$ in this interpretation are just syntactic elements, so it's unclear how physicists do their tricks.
The question is: how can I rigorously interpret physics books tricks like $$T\,dS = dE + p\,dV \iff dE = T\,dS - p\,dV$$?
A few things get swept under the rug in these formal manipulations. The first thing is that, as you've mentioned, these expressions should be read as
$$\frac{\partial S}{\partial E}=\frac{1}{T} \\ \frac{\partial S}{\partial V}=\frac{p}{T}$$
where we hold $V$ and $E$ constant in the respective lines.
An important fact here is the inverse function theorem, which in differential notation looks like
$$\frac{dy}{dx}=\frac{1}{\frac{dx}{dy}}.$$
This is easy to remember, but it is a little bit misleading as written, because $\frac{dy}{dx}$ is a function of $x$, but $\frac{dx}{dy}$ is a function of $y$. In fact what is going on is:
$$\frac{dy}{dx}(x)=\frac{1}{\frac{dx}{dy}(y(x))}.$$
Monotonicity properties which arise from the physics imply that for each fixed $V$, $E \mapsto S(E,V)$ and $S \mapsto E(S,V)$ are both monotone functions, so they are invertible; in fact they are inverses of each other, as you would intuitively expect. So the inverse function theorem applies. As a result you find that
$$\frac{\partial E}{\partial S}=\frac{1}{\frac{\partial S}{\partial E}}=\frac{1}{\frac{1}{T}}=T.$$
where we are holding $V$ constant in all the partial derivatives. So that is the first part of the transformation shown.
The second part requires the use of the "triple product rule": you have
$$\frac{\partial E}{\partial V}=-\frac{\frac{\partial S}{\partial V}}{\frac{\partial S}{\partial E}}=-\frac{\frac{p}{T}}{\frac{1}{T}}=-p.$$
This rule has a classical proof which is based on the implicit function theorem. (To set it up, consider the equation $S(E,V)=S_0$ with the base point $(E_0,V_0)$, think of $E$ as the "y" variable and $V$ as the "x" variable, and differentiate according to the recipe from the implicit function theorem.) Now we know these two derivatives, from which the equation for $dE$ follows by definition.
The other main mathematical trick that you need for rigorous discussion of thermodynamics is the Legendre transform. For instance, the rigorous meaning of
$$dA=d(E-TS)=dE-SdT-TdS=TdS-pdV-SdT-TdS=-SdT-pdV$$
is that $A$ is the Legendre transform of $E$ with respect to the variable $S$. All free energies are of this sort, and there are also "free entropies" when you do the same thing on the entropy side. Most of the "free entropies" have no special name.
By the way, I really learned this in a class working out of Thermodynamics and an Introduction to Thermostatistics by Callen. I'd seen it 2-3 times previous in chemistry undergrad, but never with any rigor.