By higher-order differential (forms?) I mean things like $d^2 y$ which can be found here on Wikipedia. We have the formula $$ d^2y = d(dy) = d\left( \frac{dy}{dx} dx\right) = \frac{d^2 y}{ dx^2} dx + \frac{dy}{dx} d^2 x. $$
The question is simple: Do they have a formal definition?
If we think in the usual way and take $d$ as the exterior derivative, then $d^2 = 0,$ which is clearly not what's going on here. The definition in differential geometry about $df$ as a linear map between tangent bundles does not apply as well since second order differential may not be linear (may involve $(dx)^2$).
You can think of $ \mathrm d ^ 2 y $ as a map on a higher-order tangent bundle (a jet bundle), although (as you've noticed) it's not a linear one.
At each point $ p $ on a differentiable manifold $ M $ of dimension $ n $, you have the tangent space $ \mathrm T _ p M $, a vector space of dimension $ n $. If $ M $ is twice differentiable, then the second-order jet space at $ p $ is is a vector space $ \mathrm J ^ 2 _ p M $ of dimension $ 2 n $, which you can think of as a direct sum $ \mathrm T _ p M \oplus \mathrm T _ p M $. But the two halves of this vector space play different roles; one represents first-order changes (or velocities), while the other represents second-order changes (or accelerations). If you think of a vector in $ \mathrm T _ p M $ as an equivalence class of curves through $ p $, where two curves are equivalent if they have the same first derivative at $ p $, then you can think of a vector in $ \mathrm J ^ 2 _ p M $ as a smaller equivalence class of curves, where two curves are now only equivalent if they have the same first and second derivatives through $ p $.
So if $ y $ is a differentiable real-valued map on $ M $, then $ \mathrm d y $ is a real-valued map from the tangent bundle $ \mathrm T M = \biguplus _ p T _ p M $, which takes a point $ p $ and a vector $ v $ to the number $ ( y \circ c ) ' ( 0 ) $, where $ c $ is any differentiable curve with $ c ( 0 ) = p $ and $ \dot c ( 0 ) = v $. And if $ y $ is a twice differentiable real-valued map on $ M $, then $ \mathrm d ^ 2 y $ is a real-valued map from the second-order jet bundle $ \mathrm J ^ 2 M = \biguplus _ p \mathrm J ^ 2 _ p M $, which takes a point $ p $, a vector $ v $, and a vector $ a $ to the number $ ( y \circ c ) ' ' ( 0 ) $, where $ c $ is any twice differentiable curve with $ c ( 0 ) = p $, $ \dot c ( 0 ) = v $, and $ \ddot c ( 0 ) = a $.
If $ ( x _ 1 , \ldots , x _ n ) $ are coordinates on an open set $ U $ in $ M $ and $ y | _ U = f ( x _ 1 , \ldots , x _ n ) $ for some differentiable real-valued partial function $ f $ of $ n $ variables, then you can write $$ \mathrm d y = \sum _ { i = 1 } ^ n \mathrm D _ i f ( x _ 1 , \ldots , x _ n ) \, \mathrm d x _ i \text , $$ or $$ \mathrm d y = \sum _ { i = 1 } ^ n \frac { \partial y } { \partial x _ i } \, \mathrm d x _ i $$ to suppress mention of $ f $. When evaluating this at a point $ p $ and a vector $ v $, you put in the coordinates of $ p $ for $ x _ 1 , \ldots , x _ n $ and the components of $ v $ for $ \mathrm d x _ 1 , \ldots , \mathrm d x _ n $, so you can see that this is linear in $ v $. If $ f $ is twice differentiable, then you can also write $$ \mathrm d ^ 2 y = \sum _ { i = 1 } ^ n \sum _ { j = 1 } ^ m \mathrm D _ { i , j } f ( x _ 1 , \ldots , x _ n ) \, \mathrm d x _ i \, \mathrm d x _ j + \sum _ { i = 1 } ^ n \mathrm D _ i f ( x _ 1 , \ldots , x _ n ) \, \mathrm d ^ 2 x _ i \text , $$ or $$ \mathrm d ^ 2 y = \sum _ { i = 1 } ^ n \sum _ { j = 1 } ^ m \frac { \partial ^ 2 y } { \partial x _ i \, \partial x _ j } \, \mathrm d x _ i \, \mathrm d x _ j + \sum _ { i = 1 } ^ n \frac { \partial y } { \partial x _ i } \, \mathrm d ^ 2 x _ i \text . $$ When evaluating this at a point $ p $, a vector $ v $, and a vector $ a $, you put in the coordinates of $ p $ for $ x _ 1 , \ldots , x _ n $, the components of $ v $ for $ \mathrm d x _ 1 , \ldots , \mathrm d x _ n $, and the components of $ a $ for $ \mathrm d ^ 2 x _ 1 , \ldots , \mathrm d ^ 2 x _ n $, so you can see that this is linear in $ a $ but quadratic in $ v $.
When $ M $ is the real line and $ y = f ( x ) $, these simplify to $ \mathrm d y = f ' ( x ) \, \mathrm d x $ and $ \mathrm d ^ 2 y = f ' ' ( x ) \, \mathrm d x ^ 2 + f ' ( x ) \, \mathrm d ^ 2 x $ (where $ \mathrm d x ^ 2 $ means $ ( \mathrm d x ) ^ 2 $), as it says on Wikipedia. If you want to suppress $ f $, then you can write these as $$ \mathrm d y = \frac { \mathrm d y } { \mathrm d x } \, \mathrm d x \text , $$ which justifies the notation $ \mathrm d y / \mathrm d x $ for a derivative, and as $$ \mathrm d ^ 2 y = \frac { \mathrm d ^ 2 y } { \mathrm d x ^ 2 } \, \mathrm d x ^ 2 + \frac { \mathrm d y } { \mathrm d x } \, \mathrm d ^ 2 x \text , $$ which unfortunately does not justify the notation $ \mathrm d ^ 2 y / \mathrm d x ^ 2 $ for a second derivative, so I prefer to write $ ( \mathrm d / \mathrm d x ) ^ 2 y $ instead (or even $ \partial ^ 2 y / \partial x ^ 2 $, which has some charm but might confuse people since it's not standard in the one-variable case).