I'm not sure I understood properly when we need to add "something" before the dVariable. For example, in a practice problem where I was asked to calculate flux out of a cylinder, I parametrized the solid as follows:
$0 < s < 2\pi$, $-1< t < 1$, $0 < r < 1$, $x = r\cos s$, $y = t$, $z = r\sin s$, in order to sweep "around it" using s, from top to bottom using t, and inside to outside using r. I then simply evaluated the divergence of the field in an integral with dr dt ds.
However, my result was a bit off - And I think it might be because I should have evaluated $\nabla\cdot F\ dr\,dt\, r\ ds$, however I'm never quite sure when I need to put something in front of my $dVar$'s or not...
The answer is you do unless you have good reason to believe you don't. And technically, even the "you don't" cases are really "you do" cases in which the factor you have to insert is $1.$
The thing about Cartesian coordinates is that if you take a chunk of space defined by $(x,y,z)$ coordinates such that $x_1 < x < x_1 + \Delta x$, $y_1 < y < y_1 + \Delta y$, and $z_1 < z < z_1 + \Delta z$ -- that is, a rectangular box of length $\Delta x$, width $\Delta y,$ and height $\Delta z$ -- the volume of that chunk of space will always simply be $(\Delta x)(\Delta y)(\Delta z)$; the same volume regardless of where you found that chunk of space (that is, independent of $x_1,$ $y_1,$ and $z_1$).
In your cylindrical coordinates, the relevant chunk of space is defined by $r_1 < r < r_1 + \Delta r$, $s_1 < s < s_1 + \Delta s$, and $t_1 < t < t_1 + \Delta t.$ The volume of this chunk is not the same regardless of where you found it. It is smaller for smaller values of $r_1,$ larger for larger values of $r_1.$
So when you integrate over $dr\,dt\,ds$, you're just not covering as much space for a given change in $r,$ $t,$ and $s$ when $r$ is small as you are when $r$ is large. And importantly, the amount of space you cover is generally not the same as you would get from the same amount of change in $x,$ $y,$ and $z$ in standard Cartesian coordinates, where the amount of space really can be measured just by multiplying length, width, and height.
So the general formula for integrating any function $f$ over any coordinates other than standard Cartesian coordinates, let's say the coordinates are named $u,$ $v,$ $w,$ is
$$ \iiint f(u,v,w)\, |J| \,du\,dv\,dw, $$
where $J$ is the Jacobian of the transformation from coordinates $(u,v,w)$ to coordinates $(x,y,z)$:
$$ J = \left | \begin{matrix} \dfrac{\partial x}{\partial u} & \dfrac{\partial x}{\partial v} & \dfrac{\partial x}{\partial w} \\ \dfrac{\partial y}{\partial u} & \dfrac{\partial y}{\partial v} & \dfrac{\partial y}{\partial w} \\ \dfrac{\partial z}{\partial u} & \dfrac{\partial z}{\partial v} & \dfrac{\partial z}{\partial w} \end{matrix} \right |. $$
See this page for how this works with cylindrical and spherical coordinates.
The cases where $|J|=1$ are the only ones where you "don't" have to insert something in front of the $du\,dv\,dw$ notation.
The way I remember the values of the Jacobian for cylindrical and spherical coordinates is to visualize a chunk of space defined by some range of values of each coordinate, like the chunks of space described above. For your cylindrical coordinates I would take the chunk defined by $r_1 < r < r_1 + \Delta r$, $s_1 < s < s_1 + \Delta s$, and $t_1 < t < t_1 + \Delta t,$ but with values of $\Delta r,$ $\Delta s,$ and $\Delta t$ so small that this chunk is almost a rectangular box; then I ask myself what the approximate volume of that box is. The answer I get is $r_1 (\Delta r)(\Delta t)(\Delta s).$ And that translates to $$\iiint f(r,t,s)\,|J|\,dr\,dt\,ds = \iiint f(r,t,s)\,r\,dr\,dt\,ds.$$
Just plug in the appropriate $\nabla\cdot F$ in place of $f(r,t,s).$
By the way, $$ \iiint \nabla\cdot F\,dr\,dt\,r\,ds$$ does not make sense; $r$ is the variable we integrate over when we do the $dr$ integration, so it should be placed in such a way as to make that clear. The usual practice is to put it to the left of all the "$d$" symbols.