I am not quite familiar with the concept of correlation. The Pearson's correlation coefficient is defined as:
$$\rho_{X,Y}=\mathrm{corr}(X,Y)={\mathrm{cov}(X,Y) \over \sigma_X \sigma_Y} ={E[(X-\mu_X)(Y-\mu_Y)] \over \sigma_X\sigma_Y}$$
which makes use of Mean and Standard deviation. But, is it strict to the normal distributed data ? Since Gaussian distribution is configured by mean and variance.
I currently have some which is apparently not following normal distribution. When assessing the correlation between them, is correlation appropriate here ?
The Pearson's correlation is often used for normal distributions. It doesn't need to assume normality, although it assumes finite variances and finite covariance. When the variables are bivariate normal, it represents a reliable and exhaustive measure of association.
However, if you have some non-normal distribution, you can consider other measures, such as the Spearman's rho (particularly if you are interested in monotonic rather than linear association), or the Kendall's tau.