I'm taking Stanford's CS229 ML course and while studying about "parametric algorithms", Prof. Andrew Ng says that this class of algorithms has a fix number of parameters (parameters are also called as "weights") that fit the data. Example is the thetas in linear regression.
Okay, I got it as $\theta0+\theta1X$ contains two parameters or weights and it is fixed as two.
But in his notes, page no. 15, he writes that (unweighted) linear regression algorithm is an example.
Now this led me to confusion. What is weights in linear regression and what is weighted and unweighted linear regression?
The weights in a linear regression are applied to each observation. The base case of an unweighted linear regression correspond to giving each observation a weight of $1$, so all have the same weight.
Using weights allows you to make some observations more important than others in the linear regression. Giving one observation a weight of $2$ is the same as if you use this observation twice but in general you can put any positive numbers as weights. The linear regression will then try to fit the high weight observations especially well.
Edit: Some formulas of where the weights are applied. You are given a list of observations $(x_i, y_i)$ for $i \in 1, .., n$ and you are trying to find parameters $a,b$ for the model $y \sim a\cdot x+ b$. In the unweighted case you look for the values $a,b$ that minimize the error $\sum_i (ax_i+b - y_i)^2$. In the weighted case you also have a list of weights $w_i$ for $i \in 1, .., n$ and then look for the values $a,b$ that minimize the error $\sum_i w_i(ax_i+b - y_i)^2$.