In this universe lets say everything exists in a 2d plane. There exists a 2d planet and hurdling towards it (at some velocity), is a 2d asteroid. Luckily, the asteroid isn't going to collide with the planet, however, the asteroid is going to get caught by the planets gravity and begin orbiting the planet.
Given 3 variables:
- Magnitude of the asteroid
- Position of the asteroid (in x, y coordinates)
- Position of the planet (in x, y, coordinates)
How can I determine if the asteroid will orbit clockwise or counter-clockwise?
Here are examples of the asteroid approaching the planet and the direction they would orbit.
You need the initial velocity of the asteroid (relative to a frame reference centered on the planet). If the asteroid were initially at rest in that frame, since gravity acts in the direction to the center of mass of the planet, the asteroid would follow a direct collision trajectory (a free fall on the planet).
If you have the velocity vector for the asteroid in the reference frame centered at the planet, then sign of the third component of the vector product of the position and velocity vectors (taken as vectors in $\mathbb R^3$) predicts the orientation of the orbit. If you want to keep it two dimensional, think of the determinant of the matrix whose columns are the position and the velocity... again, the sign defines the orientation (which orientation is + and which is - depends on the detailed definition used for those vectors and the order in which they're taken for the calculations.)