Coste's notes on semialgebraic geometry have the question:
If $f:U \to \mathbb{R}$ is semialgebraic, with $U$ an open semialgebraic set, then each partial derivative $\dfrac{\partial f}{\partial x_{i}}$ is semialgebraic
Once, a teacher told me how to answer this question. He said to write the graph of $\dfrac{\partial f}{\partial x_{i}}$ as
$$\left\{(x,y) \in U \times \mathbb{R}: \forall \varepsilon > 0, \exists \delta > 0 \left(\|t\|^{2} \geq \delta \lor \left|\left|y - \dfrac{f(p+te_{i})-f(p)}{t}\right|\right|^{2} < \varepsilon \right) \right\}$$
and use Tarski-Seidenberg to finish the proof. However, upon reviewing the proof, it is wrong, since this set is not defined using the first-order language of ordered fields. $f$ is a semialgebraic function, not a polynomial.
I don't know how to proceed here, and I appreciate any help.
Let's get this off the unanswered list.
Your main concern seems to be about using semi-algebraic functions to define other semi-algebraic functions - this is totally legal! If we've previously defined some semi-algebraic function $f:\Bbb R^m\to \Bbb R^n$, we can define a semi-algebraic function $g:\Bbb R^s\to\Bbb R^t$ using $f$ in the formula if we'd like. If you're concerned that this is somehow bad because we're not using polynomials, you can simply expand the definition of $f$ in the formula for $g$ so that all the polynomial conditions are in there. But that's a huge hassle and frequently incomprehensible to readers, so we don't typically do that.
The correct way to show that the directional derivative of a semi-algebraic function is semi-algebraic (where this derivative is defined) is to note that the statement we use to define the directional derivative at a point uses language we have available to us in the first order language of real closed fields: given a function $f:U\to \Bbb R$ where $U\subset\Bbb R^n$ is open, the derivative at $p\in U$ along the vector $v\in\Bbb R^n$ is defined to be the limit $\lim_{t\to 0} \frac{f(p+tv)-f(p)}{t}$. We can rephrase this in a manner similar to your post: the derivative is the number $y$ such that for all $\epsilon >0$ there exists a $\delta >0$ such that for all $t$ with $0<|t|<\delta$ we have $\left|\frac{f(p+tv)-f(p)}{t}\right|<\epsilon$. All of those constructors are available in the language we're working with, and so as long as $f$ can be written in the language we're working with, we're good!
Wrapping it all up, given a semi-algebraic function $f:U\to \Bbb R$ defined on a semi-algebraic open set $U\subset\Bbb R^n$, we can define the graph of the directional derivative of $f$ along a vector $v\in \Bbb R^n$ (where the derivative exists) as follows: $$\left\{ (x,y)\in U\times\Bbb R \mid \forall \epsilon > 0, \exists \delta >0, \forall t\in (0,\delta), \left|y-\frac{f(x+tv)-f(x)}{t}\right| < \epsilon \right\}.$$