When is it useful to reduce mathematical objects to foundational levels and when it is not?
Let's say you work in the field of computer vision, or else. How can you claim your method is optimal if you don't prove it down to the lowest levels of mathematics?
Here are two cases when it helps to get things down to low levels and when it is not so useful:
1) A relation R is defined as set of ordered pairs (x,y), such that the relation x R y holds. You can think of a relation in two ways: the intuitive, where you think ok I have two objects x and y and there is relation between them, or you can think of it the set theory way where a relation is a set of pairs. In this case reducing the relation down to set theory, does add new knowledge or view on what a relation is and can be helpful. The idea of an empty relations is very useful, meaning that that relation does not hold for no two objects. In summary, it is useful to think about a relation both the intuitive and the set theory way.
2) On the other hand for some things it is not useful to get them down to the fundamental levels. For example, an ordered pair. Intuitively you know what it is. Then the question is, can I learn more about it from its set theory definition? My opinion is no, because a pair (x,y) is expressed as {{x}, {x,y}} and that definition in terms of sets does not really add any new understanding to the idea of an ordered pair. It is just a set theory code, cryptic code, that just puts that mathematical object in se
Depending on the situation, I wouldn't be so eager to get involved with the most fundamental levels of mathematics. I think it is important to work at the level you are operating at, occasionally moving up or down a level as needed.
For example, biologists may (and often do) need to get into the "nitty gritty" of chemistry to understand certain processes, chemists may (and often do) need to study up on physics to understand behaviors of substances, physicists may (and often do) need to get deep into some mathematics to work out some problems, and mathematicians may (but perhaps don't so often) need to examine things philosophically. (This last example is a bit contrived, but what else underlies mathematics?).
So all of the above makes sense, but it would likely be insensible if a massage therapist decided that, because the above chain terminates more or less at mathematics, she needs to study up on analysis and algebra before she can truly feel like she has mastered her craft.
It would be equally absurd for a mechanic to feel compelled to study metallurgy in order to fix a car.
As for mathematics, it is a tool which can be quite powerful, but the power is directly related to the accuracy of the model one builds, and one has to be careful to avoid spherical cows.
If you are hoping that it will make a concept more clear (vis-a-vis your example about relations) I would say that is possible if someone has already formalized the concepts you are working with, but if you are the one doing the formalizing then you would already have to have a pretty clear idea about the concept before formulating it precisely. The exercise may be what got you there though ...
In the case of computer vision that you mentioned, I think that is a special case of something that is relatively interdisciplinary (correct me if I'm wrong) and so you may be able to find members of a team to fill in/understand the "lower level" details, taking queues from the people who understand how things work at the "higher levels", and then give those people the Cole's notes version of what's happening.