Exercise $8.4.G$ of Vakil's algebraic geometry notes asks us to prove:
If a locally closed embedding $\pi:X\rightarrow Y$ of locally Noetherian schemes is a regular embedding at $p$, then it is a regular embedding in some neighborhood of $p$ in $X$.
Where being a regular embedding at $p$ means that the stalk of the ideal sheaf $(I_{X/Y})_p$ is generated by a regular sequence for $\mathcal{O}_{Y,p}$.
There is a hint, which roughly says that we can reduce to the affine case with $\pi$ a closed immersion, say $Y = \operatorname{Spec}(B)$ and $X = \operatorname{Spec}(B/I)$. Moreover if $I_p$ is generated by the image of elements of $B$, $x_1,\dots,x_r$, then there is an open neighbourhood about $p$ where $I_q$ is also generated by the images of $(x_1,\dots, x_r)$ for all $q$ in this neighbourhood. I understood all this, but I don't see why the images of the $x_i$ should form a regular sequence at each stalk. I get that if they form a regular sequence for $B$, then they form a regular sequence at the stalk of any point in the open set in $X$, but I'm not sure if this is helpful.
Ah, I just figured it out:
First observe that $x_1,\dots, x_r$ is a regular sequence at $q$ iff for each $i$, the multiplication by $x_i$ map from $A_q/(x_1\dots,x_{i-1})_q$ to itself is injective (and the ideal generated by these elements isn't all of $A_q$). But this is the same as saying that the stalk of the kernel of the multiplication by $x_i$ map from $A/(x_1,\dots,x_{i-1})$ to itself is $0$ (call this kernel $J_i$) at $q$ vanishes.
Thus the condition that the images of $x,\dots,x_r$ form a regular sequence in $A_q$ is equivalent to saying that $(J_i)_q=0$ for each $i$ (plus the condition that the ideal generated by the elements is strictly less than $A_q$). But since this holds at $p$, by the hint there is some open neighbourhood on $(J_i)_q =0$ for all $q$ in the neighbourhood, and so intersecting these neighbourhoods with the neighbourhood where the images of the $x_i$ generate the ideal $I$ gives the desired open set.