Let we write a tensor, $$P_\mu A^\mu = - P_ \mu A_\mu $$ Where, P= momentum and A is vector potential.
My query is, when we interchange the covariant and contraviant tensor, we get a negative sign, why is that? Is there any physical or geometrical significance it it?
While the left hand side of your equation is a scalar (which is a tensor of rank 0), the right hand side is not invariant under coordinate transformations; it is neither a tensor nor a scalar.
Therefore the answer is: Because you happened to choose values of $A$ and $P$ so that the expression on the right hand side happens to be, in your current coordinate system, the negative of the expression on the left hand side.
If you want to have meaningful expressions (where "meaningful" means "independent from your coordinate system", or "physical" when you're doing physics calculations), always remember the rule:
On your right hand side, you contract two lower indices.
Note that if you have two lower indices, you can raise the index using the metric, by the equation $$P^\mu = g^{\mu\nu}P_\nu$$ and then you've got an upper index to contract with the other, lower index. Of course, in your case you then recover the left hand side expression:
$$g^{\mu\nu}P_\nu A_\mu=P^\mu A_\mu$$