I know I am missing one of the concepts here, but to calculate the standard error, we use the standard deviation of the population. Because getting the data of population was not possible, therefore we used samples in the first place. Then what does the population standard deviation here mean?
For example: I am working with the ages of people in the US. I take 50 samples of 100 people every time and I get a sampling distribution. I get the mean which can be approximated to the mean of the population. Now, what if I want to calculate the standard error? I would need the standard deviation of the population. But there is no way I can get the data of whole population and calculate it's standard deviation. That's the whole point of using samples. What is it that I am missing here?
You do not need to use the population standard deviation to calculate the sample standard deviation. The sample standard deviation has the formula $$s = \sqrt{\frac{\sum_{i=1}^n (x_i-\bar{x})^2}{n-1}}$$ where $\bar{x}=\frac{1}{n}\sum_{i=1}^n x_i$ is the sample mean. This is an unbiased estimator of the population standard deviation $\sigma$, meaning that when you take a random sample from your population and calculate $s$, its expected value is $\sigma$.