question 1: Are there mathematical definition of the semi-direct product between manifolds $$ M^{d_1} \rtimes V^{d_2}? $$
For example, is it defined as a fibration such that $M^{d_1}$ is the fiber and the $V^{d_2}$ is the base, so the total space is a bundle with the following relation $$ M^{d_1} \hookrightarrow M^{d_1} \rtimes V^{d_2} \to V^{d_2} $$
question 2: Can $M^{d_1} \rtimes V^{d_2}$ be a mapping torus?
question 3: What is the mapping class group of $M^{d_1} \rtimes V^{d_2}$?
Partial answers are very welcome! Thanks!
I don't know any standard way to define this semidirect product of manifolds, and I don't see how we can invent one.
Your intuition is correct that a fiber bundle of manifolds is like a product manifold but with more complex structure, roughly in the same way that a semidirect product of groups is like a product group but with more complex structure.
However, I think the analogy breaks down. In the case of groups, when we have $G = N \rtimes_\phi H$, we are also given an action $\phi$ of $H$ on $N$. In the case of manifolds $M$ and $V$, in order for $V$ to act on $M$, $V$ needs some algebraic structure. We could require $V$ to be a Lie group (basically, a group which is also a manifold), but I still don't see how to define the total space.
A standard mapping torus on $M$ is a fiber bundle with fiber $M$, but its base space is a circle whereas you want it to be a more general $V$.