Properties of VC dimension

4.3k Views Asked by At

I have some difficulties with understanding the notion of VC dimension.

The following is the number of question I want to answer.

Q: If there is a set $|S|=k$ such that hypothesis space $H$ doesn't shatter it, does it mean that $VC(H)<k$?

Yes. According to the definition if H doesn't shatter the set of size k, it doesn't shatter any possible set of size k, however here we have some set $S$, so there are might other set $S'$ of size $k$ which is shatter by $H$.

Q:if $H_1$ and $H_2$ are two hypothesis spaces and $H_1 \subseteq H_2$ then $VC(H_1) \leq VC(H_2)$?

Yes. Every hypothesis that belongs to $H_1$ are in $H_2$ so the shattered subset of $H_2$ is at least as of $H_1$.

Did I answer it correctly?

1

There are 1 best solutions below

0
On BEST ANSWER

"According to the definition if H doesn't shatter the set of size k, it doesn't shatter any possible set of size k"

-> No, e.g. say your hypothesis space is all straight lines. Then there exist sets of 3 points that can be shattered using this model (any 3 points that are not collinear can be shattered), but some that cannot (3 collinear points).

$H$ = all straight lines:

enter image description here

enter image description here

Keep in mind that the VC dimension of a hypothesis set $H$ is the most points $H$ can shatter.

Your answer to the second question sounds good.

If you have difficulties with understanding the notion of VC dimension, I strongly recommend CalTech's free machine Learning online course by Yaser Abu-Mostafa Learning from Data (the first 7 lectures).