I am currently learning about confidence intervals and the following question keeps bugging me.
How come that when we find a confidence interval I can just use the $Z$ or $t$ table? Let's say that I want to find a $95 \%$ confidence interval for the mean of a population, $\mu$. The variance, $\sigma$ is usually not available, so we just approximate it with $s$. In order for us to find the $95 \%$ confidence interval we have to find the margin of error. So we multiply the standard error (which is equivalent to the variance of the distribution of the sample means, I think?) by $1.96$, where we got that $1.96$ from the $Z$ table. What I don't understand is how come we can use that value from the $Z$ table. I mean, our distribution of the sample means, even if it is normal, it is not the standard normal. Going $1.96$ standard deviations tot the left and to the right of the mean would result in encompassing $95 \%$ of the area of the standard normal, not just any normal. So how come I can just use that value from the $Z$ table, which describes the standard normal, even though my distribution is indeed normal, but it is not the standard normal, it has a different mean and a different variance/standard deviation. It seems to me that this shouldn't work.
This thing seems really complicated and illogical to me. I probably misunderstood a lot of thing and that is why I have this dilemma. I would really appreciate it if you could explain this in detail, as I am just starting with statistics and my knowledge and understanding is not advanced.
I believe you are talking about a confidence interval for estimating the mean of a normal distribution $\mathrm{N}(\mu,\sigma^2)$, when the variance is unknown and has to be estimated.
But let us first look at the simpler case where we know the true variance $\sigma^2$. Then, we know that the sample mean $\bar{X}$ is normally distributed with mean $\mu$ and variance $\sigma^2/n$. Hence, $$\frac{\bar{X}-\mu}{\sigma/\sqrt{n}}\sim\mathrm{N}(0,1).$$ That is the reason why you have to look up the quantiles in the table for the standard normal distribution.
The 95-percent confidence interval is supposed to overlap the true mean $\mu$ with probability 0.95, i.e., $$P\left( -z_{0.975} \leq \frac{\overline X-\mu}{\sigma/\sqrt{n}} \leq z_{0.975} \right) =0.95. $$ Rearranging gives you the confidence interval $$\left[ { \overline x-z_{0.975}\frac {\sigma}{\sqrt{n}}; \overline x+z_{0.975}\frac{\sigma}{\sqrt{n}}} \right]$$
Now, in the case where we have to estimate $\sigma^2$ using the sample variance $s^2$, everything works the same, just that $$\frac{\bar{X}-\mu}{s/\sqrt{n}}$$ does not follow a normal distribution, but a t-distribution with $n-1$ degrees of freedom.