Is the signature of inverse images of diffeomorphic submanifolds (along a homotopy equivalence) the same?

Question

Suppose it is given an orientation preserving homotopy equivalence $h:N\to M$ between closed oriented connected manifolds. Let $X$ and $Y$ be diffeomorphic submanifolds of $M$, and assume $h$ to be transverse to both $X$ and $Y$. Define $A:=h^{-1}(X)$ and $B:=h^{-1}(Y)$. I would like to know if

sign($A$)=sign($B$) ?

To avoid triviality, assume dim($A$)=dim($B$) to be a multiple of 4. Is there a way to show that (maybe) $A$ and $B$ are oriented cobordant? Any example/counterexample can be useful.

Answer 1

This turned out to be false. By Example 3.1 in the paper of James Davis "Manifold aspects of the Novikov conjecture", there is a homotopy equivalence $h:S(E')\to S^4\times S^4$ such that $\sigma(h^{-1}(pt\times S^4))=16$, but $\sigma(h^{-1}(S^4\times pt))=\sigma(S^4)=0$ since $h$ preserves the fibers.