Assume that matrix $A$ and $B$ are square with same dimensions.
prove by mathematical induction that $(ABA^{-1})^k=AB^kA^{-1}$
if I let $k=2$ then I think I can do
$$ (ABA^{-1})^k=AB^kA^{-1}\\ (ABA^{-1})^2=AB^2A^{-1}\\ (ABA^{-1})(ABA^{-1})=ABBA^{-1}\\ ABBA^{-1}=ABBA^{-1}\\ $$
From here I can see a repeating pattern where I can manually make $k$ any positive integer and do the above but I'm not sure how to do this formally for the general case.
You have already formed a basis for induction.
Assuming that $n=k$ is true, we have $$(ABA^{-1})^k=AB^kA^{-1}.$$ Hence if we consider $n=k+1$, we get $$(ABA^{-1})^{k+1}=(ABA^{-1})^kABA^{-1}=AB^kA^{-1}ABA^{-1}=AB^{k+1}A^{-1}$$ which completes the induction step.