One thought I had in the shower today was that I've never encountered a differential equation with initial conditions designed to eliminate anything other than the arbitrary constant from the general solution, yet I don't see why this has to be the case.
Consider the ordinary differential equation for an account balance with $10$% interest and a periodic deposit of $2$ (both continuous), $\frac{dP}{dt} = .1P + 2$. The general solution is $P = Ce^{.1t} - 20$, which could be plotted as a surface with inputs $t$ and $C$, and output $P$. In other words, the general solution doesn't make much fuss over the conceptual difference between $t$ and $C$; the arbitrary constant may as well be a variable. In this problem, $C$ has the meaning of being the initial account balance, and it is certainly a quantity that could vary across different account holders.
The real difference is that we expect the initial condition to pin down $C$ and not $t$. Such an unremarkable initial condition might be $P(1) = 5$, which is compact notation for $t = 1, P = 5$. However, what if an initial condition was of the form $C = 3, P = 1$? Then one could solve for $t$ and obtain a "particular solution" representing account balance as a function of initial account balance, which also seems like useful information. This approach redefines particular solutions as constant-$t$ slices of the surface, rather than as constant-$C$ slices of the surface.
Extending this idea further, it should be possible to redefine particular solutions as diagonal slices parallel to $t = C$, or slices parallel to $t = 2C$, or slices of any other constant slope in the $tC$-plane. It should also be possible to introduce an initial condition of the form $t = 4, C = 7$, eliminating the dependent variable from the re-redefined particular solution, which would then express a relation between $t$ and $C$.
Why is this never explored in differential equations? Do we never find phenomena easiest to describe by a differential equation in certain independent and dependent variables, but empirical data points in terms of the integration constant (such as initial account balance)? Do we just have no reason to care about such alternative solutions? Does applying these sorts of initial conditions swapping the roles of the independent variable and the arbitrary constant invalidate the process used to solve the differential equation in the first place, instead turning it into a PDE? Does the answer change if we reformulate the question in terms of recurrence relations?
The arbitrary constant comes about solely to describe the fact that the given DE can be satisfied by a whole family of valid solutions. There are potentially many ways to parameterise those solutions, and the parameter you use may or may not correspond to a real-world quantity.
For example, you could parameterise the solution to $\frac{dx}{dt} = -kx$ as $x(t) = A_1 e^{-kt} = e^{-kt + A_2} = (e^k)^{A_3 - t}$ where the relationships between $A_1$, $A_2$ and $A_3$ are not hard to derive but all of them have different interpretations. That's why they're called arbitrary constants, because from a mathematical perspective they don't have any specific meaning until they're tied to a real-world system.
There's certainly nothing wrong in using something other than a zero-point as the "initial condition", and likewise transforming the solution to let you examine other relationships between quantities is valid. For example, motion under constant gravity is often represented by the DE $\frac{d^2 x}{dt^2} = -g$, the solutions of which can be written in a variety of ways depending on whether you know about position or velocity at a variety of possible times.