Let $f:\mathbb{R} \rightarrow \mathbb{R}$ such that $f \in C_{1}(\mathbb{R})$
The Caputo fractional derivative is written as:
\begin{equation} {C}{}{D}^{\alpha}_{a} f(x) = \frac{1}{\Gamma(1-\alpha)} \int_{a}^{x} \frac{f'(t)}{(x-t)^\alpha} \, dt \end{equation} Since we can write the ordinary derivative of $f$ as:
\begin{equation} f'(x) = \lim_{{h \to 0}} \frac{f(x + h) - f(x)}{h} \end{equation}
How can we write the Caputo fractional derivative in a similar way that involve the limits?
This is more of a comment than an answer as I’m not addressing the “Caputo” specific concern
The fractional-difference is a much simpler operator to understand than the fractional-derivative. We begin by considering a somewhat arbitrary two term linear operator such as below (we will later show how to transform any two term linear operator into this form)
$$ f(x) - a(x)f(\theta(x)) $$
Here $\theta(x)$ is any function that you like (preferably invertible) and same with $a(x)$. We now make a formal variable substitution $O = a(x)f(\theta(x)), I = f(x)$ so that this equation now reads
$$ a(x)f(\theta(x)) - f(x) = I - O $$
From here suppose we want to find the "half" iterate of this operator. We simply consider
$$\left( I - O \right)^{\frac{1}{2}} $$
this can be viewed as a formal power series in $O$. So much like how we evaluate (from here)
$$ \sqrt{1 -x} = 1 - \frac{x}{2} - \frac{x^2}{8} - \frac{x^3}{16} - \frac{5x^4}{128} ... $$
We can say that:
$$ \left( I - O \right)^{\frac{1}{2}} = I - \frac{O}{2} - \frac{O^2}{8} - \frac{O^3}{16} - \frac{5O^4}{128} ... $$
Here $O^n$ indicates applying $O$ a total of $n$ times. So for example if $Of(x+1)$ then $O^n = f(x+n)$ (for a very rigorous reader this might be hard to pattern match. Just drop a comment and I can make it more explicit whats being said here).
So putting this into practice suppose we wish to evaluate the half fractional iterate of $T[f] = \frac{f(x+2)-f(x)}{2}$ then we have that $-2T[f] = f(x) - f(x+2)$. So if we let $O=f(x+2)$ and $I = f(x)$ we have:
$$ (-2T[f])^{\frac{1}{2}} = (I-O)^{\frac{1}{2}} $$
So
$$ \pm i \sqrt{2} T[f]^{\frac{1}{2}} = (I-O)^{\frac{1}{2}} $$
From here we then just reuse our earlier result
$$ \pm i \sqrt{2} T[f]^{\frac{1}{2}} = I - \frac{O}{2} - \frac{O^2}{8} - \frac{O^3}{16} - \frac{5O^4}{128} ... $$
Substituting back $O=f(x+2)$ we have that the square root of the finite difference: $\frac{f(x+2)-f(x)}{2}$ or its "$\frac{1}{2}$-difference" like $\frac{1}{2}$-derivative: is given by:
$$ T[f]^{\frac{1}{2}}[f(x)] = \mp \frac{i}{\sqrt{2}} \left( f(x) - \frac{1}{2}f(x+2) - \frac{1}{8}f(x+4) - \frac{1}{16}f(x+6) - \frac{5}{128}f(x+8) ... \right) $$
In general any difference operator (and generally arbitrary 2-term linear operator) you want to raise to any (even complex) power is then covered by this case. Actually we can do even better than that. We consider any power in any algebra where we can embed a copy of the natural numbers. So a question as difficult as it is vague as find me a natural $\begin{pmatrix} 1 & 0 \\ \frac{1}{3} & -5\end{pmatrix}^{\text{th}}$ iterate of $T[f] = \frac{f(3x^2) - f(x)}{2x}$ becomes possible to answer. If the the $\frac{1}{2}$ the derivative wasn't strange enough then this must seem really bizarre to say the least!
$$$$
What's funny is the only time this becomes difficult is when looking at the derivative. See the problem is the derivative is given by $\frac{f(x+dx) - f(x)}{dx}$ where $dx$ means a quantity that limits absolutely close to 0. So when you look at an expression like $f(x+dx)$. If we compose this operator with itself we get $f(x+2dx)$. But in practice $\lim_{dx \rightarrow 0} f(x+2dx) = \lim_{dx \rightarrow 0} f(x+dx) = f(x)$ so our entire setup here falls apart and we get just one gigantic and complicated looking $\frac{0}{0}$ when using these techniques. That is why the case of the derivative is handled very differently than the other operators and moreover there exists so many different ways to define the fractional derivative (depending on which properties you want to preserve).