The meaning of realizable case in ML

Question

When learning the Chapter 3 of 'Foundations of Machine Learning', I saw the 'realizable case' and 'non-realizable case' which has never mentioned before.

Wish someone to tell me if you know the meaning.

Thanks for your time.

Answer 1

These terms are defined in terms of a true target function $\tilde{f}$ you’re trying to identify and a finite hypothesis class of functions $\mathcal{C}$ in which you’re searching.

The realizable case is that $\tilde{f} \in \mathcal{C}$; the target is within the set that you’re searching.

The non-realizable case is more frequent in machine learning. The true function is not within your hypothesis space.

In both cases, your goal is to identify a function with minimal error on the data. In the realizable case, this function will be $\tilde{f}$. In the non-realizable case, it cannot be; consequently, you will have some nonzero error.