Given two multivariate functions $f(x), g(x) \in R$ where $x \in R^D$. I would like to check if we have the following bound (based on triangle inequality).
$$ \frac{\partial |f(x)|}{\partial x} - \frac{\partial |g(x)|}{\partial x} \preccurlyeq \frac{\partial |f(x) - g(x)|}{\partial x} $$
Where $ \preccurlyeq$ is element-wise operator. If not, please suggest a nice form for the upper bound of left-hand side.
Thanks,
Sorry, but this isn't true. As I noted in the comment, this is just the 1-dimensional inequality $$\frac {d|f(t)|}{dt}- \frac{d|g(t)|}{dt} \le \frac{d|f(t) - g(t)|}{dt}$$ repeated $D$ times (with $t = x_i$ and all the other variables held constant).
But that inequality is not true. If the absolute values were outside the derivatives, or the derivatives were not taken, it is just a form of the triangle inequality. But derivatives do not preserve inequalities.
A simple counter-example: $f(t) = t, g(t) = t -1$. When $t \in (0,1)$, the LH side is $2$, while the RH side is $0$.
As for what bound should be on the right side, that depends on what you are trying to do. There are many bounds that could be put on it, for example, the trivial and obviously useless bound $$\frac {d|f(t)|}{dt}- \frac{d|g(t)|}{dt} \le \frac {d|f(t)|}{dt}- \frac{d|g(t)|}{dt}$$ Without some understanding of what you hope to accomplish with this bound, I cannot say what would be adequate.