According to Orthogonality condition, it is true that:
$\int_0^b xJ_0(\lambda_nx)J_0(\lambda_mx)dx = 0$ if $m \not=n $
What is the result for :
$\int_0^b xJ_0(\lambda_nx)Y_0(\lambda_mx)dx = ?$
For two cases:
if $m \not=n $
if $m=n$
According to Orthogonality condition, it is true that:
$\int_0^b xJ_0(\lambda_nx)J_0(\lambda_mx)dx = 0$ if $m \not=n $
What is the result for :
$\int_0^b xJ_0(\lambda_nx)Y_0(\lambda_mx)dx = ?$
For two cases:
if $m \not=n $
if $m=n$
On
For any $\mu, \nu > 0$, let $\tilde{J}(z) = J_0(\mu z)$ and $\tilde{Y}(z) = Y_0(\nu z)$. They satisfies the ODE: $$ \frac{d}{dz}\big[z\tilde{J}'(z)\big] + \mu^2 z \tilde{J}(z) = 0 \quad\text{ and }\quad \frac{d}{dz}\big[z\tilde{Y}'(z)\big] + \nu^2 z \tilde{Y}(z) = 0 $$ Mutiply $1^{st}$ ODE by $\tilde{Y}$, the $2^{nd}$ by $\tilde{J}$ and subtract them, we get: $$\frac{d}{dz}\big[z\tilde{J}'(z)\big]\tilde{Y}(z) - \frac{d}{dz}\big[z\tilde{Y}'(z)\big]\tilde{J}(z) + (\mu^2-\nu^2) z\tilde{J}(z)\tilde{Y}(z) = 0 $$ This implies $$\begin{align} z \tilde{J}(z)\tilde{Y}(z) = & \frac{1}{\nu^2-\mu^2}\frac{d}{dz}\left[z\left( \tilde{J}'(z)\tilde{Y}(z) - \tilde{Y}'(z)\tilde{J}(z)\right)\right]\\ = & \frac{1}{\mu^2-\nu^2}\frac{d}{dz}\Big[\mu z J_1(\mu z) Y_0(\nu z) - \nu z Y_1(\nu z)J_0(\mu z)\Big] \end{align} $$ Notice $$\lim_{z\to 0} z Y_0(z) = 0, \quad \lim_{z\to 0} z Y_1(z) = -\frac{2}{\pi}$$ and $J_0(z), J_1(z)$ are regular at $z = 0$, we get: $$\begin{align} &\int_{0}^{b} z J_0(\mu z)Y_0(\nu z) dz\\ = & \frac{1}{\mu^2-\nu^2}\lim_{\epsilon\to 0}\Big[ \mu z J_1(\mu z) Y_0(\nu z) - \nu z Y_1(\nu z)J_0(\mu z) \Big]_{\epsilon}^b\\ = & \frac{1}{\nu^2-\mu^2}\left[ \frac{2}{\pi} - b \left( \mu J_1(\mu b) Y_0(\nu b) - \nu Y_1(\nu b)J_0(\mu b) \right)\tag{*1} \right]\end{align} $$ If $\lambda_m$ and $\lambda_n$ are distinct roots of $J_0(\lambda b)$, this reduces to:
$$\color{firebrick}{\int_{0}^{b} z J_0(\lambda_n z)Y_0(\lambda_m z) dz = \frac{1}{\lambda_m^2 - \lambda_n^2}\left[\frac{2}{\pi} - \lambda_n b J_1(\lambda_n b)Y_0(\lambda_m b)\right]} $$
For the remaining $\lambda_m = \lambda_n$ case, we need to use the fact: $$J_1(z) Y_0(z) - Y_1(z) J_0(z) = Y_0'(z)J_0(z) - J_0'(z)Y_0(z)$$ is a Wronskian for the pair of solutions $J_0(z)$, $Y_0(z)$ of the $k = 0$-th Bessel ODE:
$$\frac{d^2\phi(z)}{dz^2} + \frac{1}{z}\frac{d\phi(z)}{dz} + ( 1 - \frac{k^2}{z^2} ) \phi(z) = 0$$ General theory of $2^{nd}$ order ODE tell us it is proportional to $\frac{1}{z}$. Evaluate it at $z \sim 0$ shows that it is equal to $\frac{2}{\pi z}$. This means in $(*1)$, we can take the limit $\mu \to \nu$ to obtain:
$$\begin{align}\int_{0}^{b} z J_0(\nu z)Y_0(\nu z) dz = & \frac{b}{2\nu} \left( \frac{d}{d\mu} \big[\mu J_1(\mu b)\big]_{\mu\to\nu} Y_0(\nu b) - \nu Y_1(\nu b)\frac{d}{d\mu}\big[J_0(\mu b)\big]_{\mu\to\nu} \right)\\ = & \frac{b^2}{2}\big( J_0(\nu b)Y_0(\nu b) + J_1(\nu b)Y_1(\nu b)\big) \end{align} $$ When $\lambda_m$ is a root of $J_0(\lambda b)$, this simplifies to:
$$\color{firebrick}{\int_{0}^{b} z J_0(\lambda_m z)Y_0(\lambda_m z) dz = \frac{b^2}{2} J_1(\lambda_m b)Y_1(\lambda_m b)}$$
Presumably $\lambda_j$ are the zeros of $J_0(\lambda b)$. The answer is not $0$. For example, with $b=1$, Maple says $$\int_0^1 x J_0(\lambda_1 x) Y_0(\lambda_2 x)\ dx \approx 0.042926296743804723657$$ I doubt that there is a closed-form formula.