In the above picture, I am trying to find the smallest distance from points on the line segment to the origin.
Now, I can see that it must be half the square root of 2, but this is not exactly my question.
How can I use calculus to find the smallest distance from points on a line to a point(the origin)? In my previous calculus courses we have studied optimization but it does not apply to this scenario.

You want to minimize $$ d^2 = (x-0)^2+(y-0)^2$$
The equation of your line is $$y=1-x$$
Thus you are minimizing $$d^2 = x^2+(1-x)^2$$
Taking derivative you get $$2x-2(1-x)=0$$ which results in $x=1/2$
The point on the line is then $(1/2,1/2)$ with the minimum distance from the origin $\sqrt 2 /2$ as expected.