I'm looking for a double answer to this question: a mathematical one (say, if the statement is correct or not) and a philosophical one (say, why we do expect this to be true, or not).
Let $k$ be a field, that we may or may not assume to be algebraically closed. Let $A = k[x_1, \ldots, x_n]$ be the polynomial ring in $n$-variables and coefficients in $k$. Consider an embedded curve $C$ in $\mathbb{A}^n$, regular at a point $P$ (say, the origin). Let $I = (f_1, \ldots, f_r)$ be the ideal defining the curve. By the Jacobian criterion, we know that not all the derivatives of $f_i$ are $0$ in $P$.
Let $\mathcal{O}_{C,P} = A_P/I_P$ be the local ring of $C$ at $P$: it is a discrete valuation ring and we can choose a local parameter $t$ for $C$ at $P$, i.e. a generator for the maximal ideal. We can therefore write down a local parametrization of the curve:
$$ x_i=x_i(t)= t^{c_i}g_i(t), c_i\in \mathbb{Z}, v_t(g_i(t))=0 $$ where $v_t$ denotes the valuation. The Jacobian criterion recalled above tells us that, in particular, there is one $x_i$ (say $x_n$) such that $c_i=1$.
By passing to the completion $\hat{\mathcal{O}}_{C,P}$ we can absorb the unit term in the local parameter. This gives in particular that we could have chosen directly $x_n\in \mathcal{O}_{C,P}$ as parameter and we could have written down
$$x_i =h_i(x_n), h_i(x_n)\in k[[x_n]]$$
Now the question is: is it true that we have a canonical isomorphism
$$\hat{\mathcal{O}}_{C,P} \cong k[[x_1, \ldots, x_n]]/(x_i-h_i(x_n))$$ (hopefully one should not complete again on the right)?
For $n=2$, one can invoke Hensel's lemma, in the form that reminds the implicit function theorem: let $f(x,y)\in k[[x]][y]$ and suppose $f(0,0)=0$, $\partial f/\partial y(0,0)\neq 0$. Then there exists $g(x)\in k[[x]]$ such that $g(0)=0$ and $f(x, g(x))=0$. If I understand correctly, this statement gives that, after passing to the completion, one has $$k[[x,y]]/(f(x,y)) \cong k[[x,y]]/(y-g(x))\cong k[[x]]/(g(x)).$$
Side question: we know that the implicit function theorem does not hold in algebraic geometry, in the sense that Zariski topology has too large open sets to expect that an étale map is a local isomorphism. On the other hand, we generally say that "this is true étale-locally". So I ask: is the above statement correct after passing to the Henselianization of the ring $\mathcal{O}_{C,P}$? Is this true after passing to the strict Henselianization?
I think I can try to make a partial answer to my question. And the answer is yes, it is indeed true that one has the claimed isomorphism $\hat{\mathcal{O}}_{C,P}\cong k[[x_i]]/(x_i-h_i(x_n))$, and this can be shown either abstractly or "geometrically".
Approach 1: the ring on the right is clearly isomorphic to $k[[t]]$, power series ring in $1$-variable. On the other hand, the left hand side is the completion of a discrete valuation ring that contains a field (equi-characteristic case of Cohen structure theorem). Thus its completion is isomorphic to $k[[t]]$. Ok, this is not very enlightening, but at least it answers to the question.
Approach 2: more interesting. First, since $x_n$ is a local parameter at $P$, one can write down equations for the other variables even before passing to the completion. We don't need to absorb the unit (as I was saying in the question). This gives a canonical morphism between the local ring $\mathcal{O}_{C,P}$ and $k[x_1, \ldots, x_n]_{P}/I$, where $k[x_1, \ldots, x_n]_{P}$ is the localization of the polynomial ring at the origin and $I$ is the ideal given by the equations $x_i$=some function of $x_n$. This is true simply because every regular function on the curve at $P$ must satisfy those equations.
Now: the non singularity condition shows that one has indeed (at least) an injection between the tangent space to the local ring $k[x_1, \ldots, x_n]_{P}/I$ and the tangent space to the curve $C$ at $P$, or, in other words, that we have an étale morphism between two local rings with the same residue field (in the example $P$ was the origin: in general we can do this for a $k$-rational point).
But then (see e.g. Liu Quing's book 4.3.26) the morphism induces an isomorphism between the formal completions of the 2 local rings, that was exactly what I was asking for. In the end, the fact is that the implicit function theorem does not hold for the Zariski topology (no way to get the statement for the local rings), but it holds formally after completion.
I'm still curious to see/read improvements and/or corrections!