I have a set of discrete points (at most a single $y$ value for a given $x$) and I need to find two parallel lines which contain all of these points and minimize the distance between them. Note that the lines do not have to be parallel with the $x$ axis as in the picture, they can have arbitrary angle. Is there a well known way of solving this?
Prediction Interval Bands in Simple Linear Regression.
Following from my Comment, here is a plot of $n = 20$ pairs $(x, Y)$ with a regression line of $Y$ on $x,$ showing 95% 'prediction interval bands'. For a new $x_p$ (not used to make the regression line), the corresponding predicted $Y$-value is between the two curves (nearly linear here) just above $x_p$ (with 95% confidence).
Depending on the data, it is possible for a few points to lie outside the bands. If this is undesirable, one can use a higher level of confidence for the predictions interval (perhaps 99% instead of 95%).
If you will look at the plot very closely, you will see that the 'bands' are curves. They are a little closer to each other at $\bar X = 10.5$ than anywhere else.
This is a standard procedure. Formulas for the prediction interval are given in almost any basic statistics text containing a treatment of simple linear regression.
Notes: (1) The data were simulated according to the model $Y_i = 10 + 2x_i + e_i,$ where $e_i \stackrel{indep}{\sim} \mathsf{Norm}(\mu=0,\sigma=2).$ (2) The plot is from Minitab 17 software.