In this example,
If we use the relation $F'(x) = f(x)$, this consequence of the fundamental theorem may be written in the form
$$ f(b) - f(a) = \int_a^b F'(x) dx = \color{#c66}{\boxed{\color{black}{ \int_a^b \frac{dF(x)}{\color{red}{dx}}\color{red}{dx} = \int_a^b dF(x)}}}, $$
In the last equation of this example, obviously, the two $dx$ are canceled in the left before we get the right, this indicates that the relationship between $f(x)$ and $dx$ within $\int_a^b f(x)\,\mathrm{d}x$ is multiplication , but I cannot find a reasonable explanation for this. Using infinitely small quantities to explain it is wrong since infinitely small has no place in modern mathematics saying here .
Anyone can explain why the relationship between $f(x)$ and $dx$ in $\int_a^b f(x)\,\mathrm{d}x$ is multiplication? what does $dx$ mean here ?
It's not really multiplication. It's actually largely just notational.
$dx$ can be interpreted as the differential of $x$.
$dF(x)$ can be interpreted as the differential of $F(x)$, which is actually just $F'(x)\, dx.$ This comes from the fact that $\dfrac{dF(x)}{dx} = F'(x).$ We can move the $dx$ to the other side of the equation to get $dF(x) = F'(x) \, dx.$ This is known as separating the variables. This looks like we're multiplying both sides by $dx$ but we're not. We can't multiply by $dx$ because $dx$ doesn't actually represent a number. It represents an infinitely small quantity, somewhat similar to how $\infty$ represents an infinitely large one.
So, when we simplify $\dfrac{dF(x)}{dx} \, dx$ to just $dF(x)$ it's just a matter of convenient notation.
Here's a concrete example following the same pattern as (31) in the provided image, where in this example I use $F(x) = \sin x.$ So then $F'(x) = \cos x$ and we have:
$$\sin 2 - \sin 1 = \int_1^2 \cos x \, dx = \int_1^2 \frac{d \sin x}{dx} \, dx = \int_1^2 d\sin x$$