The question and its hint is given below:
But I could not understand what the question is trying to teach me, could anyone explain this for me please?
Also I could not understand how the hint could be used in the solution, could anyone explain this for me please?
Thank you!


The problem is testing your understanding of the definition of a group acting on a set by having you work on a specific example: the group $GL_2(\mathbb R)$ acting on the set $\hat{\mathbb{R}}$. If you can see what the connection is to the suggested transformation is and the group, then all you're doing is verifying group action axioms.
The hint is in case you fail to see the connection between the proposed action and $GL_2(\mathbb R)$. (Yes, there are four numbers $a,b,c,d$ in the original proposition, but what do they have to do with matrices? Well, here's a hint...)
It explicitly outlines a correspondence between elements in $\hat{\mathbb R}$ and one dimensional subspaces of $\mathbb R^2$ plus the vertical line through $(0,0)$, and notes that the proposed action is simply matrix multiplication plus the extra rule that assigns meaning to the edge cases when $\infty$ appears for $x$ and $0$ appears in the denominator.