A tetrahedron is self-dual, so it is no surprise that in both the tetrahedron and its dual the middle cross section is the same shape (a square).
A cube and an octahedron are dual, and the middle cross sections are regular hexagons.
A dodecahedron and an icosahedron are dual, and the middle cross sections are regular decagons.
Is there some reason for this?
You seem to be getting at Petrie polygons. These are polygons formed by following edges on a polyhedron, so that every two adjacent edges are on the same face, but no three are. You sort of zig-zag around the whole thing.
As you say, the Petrie polygon for a tetrahedron is a square (well, it doesn't lie in a plane, so we might call it a skew quadrilateral instead); for a cube or octahedron, it's a hexagon, and for a dodecahedron or icosahedron, it's a decagon.
Here is a picture from Coxeter's Regular Polytopes of each Platonic solid projected into its Petrie polygon:
They are closely related to the equatorial polygons of the quasi-regular polyhedra. These polyhedra can be formed by truncating the vertices of each regular polyhedron to the edge midpoints. If you do this to either the cube or the octahedron, you get a cuboctahedron; if you do it to either the dodecahedron or icosahedron, you get an icosidodecahedron; if you do it to the tetrahedron, you get an octahedron.
If you look at the cuboctahedron, you will notice that there are four different ways that it's split in half by an "equator" of edges running around it; these equators are all regular hexagons. Each vertex of the cuboctahedron is a midpoint of an edge of the cube you formed it from, and taking the edges of the cube corresponding to the vertices of an equatorial hexagon gives you a Petrie polygon in the cube.
As for why they are always the same for a polyhedron and its dual, it has to do with the fact that the quasiregular polyhedra are halfway in a process of transforming a regular polyhedron into its dual:
It has the same relationship to each. You can make a cuboctahedron, for instance, as the intersection of a cube and its dual octahedron, with the same center and the same radius of the edge midpoints. Then the vertices of the cuboctahedron are the midpoints of the edges of both the cube and the octahedron.
You can read all about it in Chapter 2 of Coxeter's Regular Polytopes.