Electric potential at a point outside the charge distribution is:
$\displaystyle \psi (\mathbf{r})= \int_{V'} \dfrac{\rho (\mathbf{r'})}{|\mathbf{r}-\mathbf{r'}|} dV'$
where:
$\mathbf{r}=(x,y,z)$ is coordinates of field point
$\mathbf{r'}=(x',y',z')$ is coordinates of source point
$\rho (\mathbf{r'})$ is the density of charge distribution
PART:$(1)$ If we move from one point $\mathbf{r_1}$ to another point $\mathbf{r_2}$ (through a path outside the charge distribution), $|\mathbf{r}-\mathbf{r'}|$ varies continuously. There is no variation in $\rho (\mathbf{r'})$ and limits of integration w.r.t $V'$. So intuitively $\psi (\mathbf{r})$ varies continuously. Using these arguments, is there a way to mathematically prove this?
PART:$(2)$ Is there a simple mathematical way to prove $\dfrac{\partial \psi (\mathbf{r})}{\partial x}=\lim\limits_{h \to 0} \dfrac{\psi(x+h,y,z)-\psi(x,y,z)}{h}$ exists?
The easiest thing to prove is this. If $f$ is a continuous function of $(x,y)\in\Bbb R^3\times\Bbb R^3$, and $y$ varies over a compact (closed and bounded) domain $V'\subset\Bbb R^3$, then the function $$F(x) = \int_{V'} f(x,y)\,dy$$ is continuous.
Fix a point $x_0$ and a closed ball $\bar B(x_0,R)$ of radius $R$. Then the set $\bar B(x_0,R)\times V'$ is a compact subset of $\Bbb R^3\times\Bbb R^3$. A continuous function on such a compact set is uniformly continuous. This means that given any $\epsilon>0$, there is $\delta>0$ so that whenever $\|(x,y)-(x',y')\|<\delta$, we'll have $|f(x,y)-f(x',y')|<\epsilon$. This means, in particular, that whenever $\|x-x_0\|<\delta$, we'll have $|f(x,y)-f(x_0,y)|<\epsilon$ for every $y\in V'$. Thus, whenever $\|x-x_0\|<\delta$, we have \begin{align*} |F(x)-F(x_0)| &= \left|\int_{V'} f(x,y)\,dy - \int_{V'} f(x_0,y)\,dy\right| \\ & = \left|\int_{V'} \big(f(x,y)-f(x_0,y)\big)dy\right| \\ &\le \int_{V'} |f(x,y)-f(x_0,y)|dy < \epsilon\,\text{vol}(V'), \end{align*} so $F$ is continuous at $x_0$.
Here's a sneaky way to show that you can differentiate under the integral sign (there are fancier ways, of course, with Lebesgue integration). For now, I'll assume $x$ is just a single variable, rather than a vector, but you can apply this to any component function. We assume $\dfrac{\partial f}{\partial x}$ is continuous (which certainly holds in the case you're interested in). Let $$\phi(t) = \int_{V'} \frac{\partial f}{\partial x}(t,y)\,dy \quad\text{and}\quad \Phi(x) = \int_{x_0}^x \phi(t)\,dt.$$ Note that, $\phi$ is continuous by our first result. Then, interchanging the order of integration, we get \begin{align*} \Phi(x) &= \int_{V'}\int_{x_0}^x \frac{\partial f}{\partial x}(t,y)\,dt\,dy \\ &= \int_{V'} \big(f(x,y)-f(x_0,y)\big)\,dy = F(x)-F(x_0). \end{align*} It follows from the (first part of the) Fundamental Theorem of Calculus that $F'(x) = \Phi'(x) = \phi(x) = \displaystyle{\int_{V'} \frac{\partial f}{\partial x}(x,y)\,dy}$, as desired.