I have vectors Y and W. I know constrain that Y should be sorted by W. So if Wi < Wn than Yi < Yn. And I need some function Z(preferably differentiated, maybe not everywhere) that returns how good the Y is sorted by W. If all elements are sorted Z is max if all are in incorrect order min, the more are sorted the more should be Z.
I want to use it in the deep learning as a loss to make a monotonicity constraint using it. Any idea or hint?
A function that does what you want that can be computed in quadratic time is to loop over all unordered pairs of elements and add $1$ to a sum if they're not inverted and add $0$ if they are. There is a way to also compute this in $O(n\log n) $ time similar to merge sort.
This is simply the complement of the number of inversions of the array, and it satisfies your conditions.