$\underline{\text{Overview}}$
This is a self-answer re-posting of this currently active MathSE question, which will likely be closed without an answer, since it violates MathSE protocol. I consider the question worthwhile, so I am re-posting, with a solution.
For what it's worth, I did try to check if the question/answer was a duplicate. I did find this answer, but I had trouble determining how to modify the answer to fit my problem.
$\underline{\text{The Problem}}$
If you have n books and m consecutive places, where m is bigger than n, how many combinations are there so that no 3 books sit next to each other? 2 books can be adjacent but not three in a row.
I am interpreting this question as intending that the books are indistinguishable from each other. For the alternative assumption, you can simply apply the $~n!~$ scalar to the answer.
Part of the reason that I value this question is because I find it much more difficult than the simpler question of no 2 books in a row. The simpler question immediately yields to Stars and Bars, which is also discussed here:
- $x_1 + x_2 + \cdots + x_n = m.$
- $x_1, x_n \in \Bbb{Z_{\geq 0}}.$
- $x_2, x_3, \cdots, x_{n-1} \in \Bbb{Z_{\geq 1}}.$
Simply apply the change of variables $~y_i = x_i - 1 ~: ~i \in \{2,3,\cdots, n-1\},~$ and everything falls into place.
That approach won't work with the current problem. Also, I think that Inclusion-Exclusion would be a nightmare, because you would have to make an attempt to diagnose varying patterns in the intersections of $~r~$ triplet subsets. A separate diagnosis would be needed for each $~r~$ in $~\displaystyle \left\{1,2,\cdots,(m-2)\right\}.$
Since I am ignorant of generating functions, I have no idea whether such an approach is viable.
For me, that leaves recursion. Solution follows.
Disclaimer
This answer does not provide a closed form solution, but hopefully provides some additional insights.
Set Up
Put the books into small and large boxes. Each small box stores one book and takes one place. Each large box stores two books and takes two places. There is at most one box between two consecutive empty places, at most one box before the first empty place, and at most one box after the last empty place.
Calculation
Put the books into $r$ large boxes and $n-2r$ small boxes, so we have a total of $n-r$ boxes. Since the books occupy $n$ places, there are $m-n$ empty places. The number of ways to put the box among the empty places is:
$$ \binom{m-n+1}{n-r} $$
After deciding where to put the box, we need to choose which boxes are the large boxes. The number of possibilities:
$$ \binom{n-r}{r} $$
Therefore, the total number of ways is the product of the two binomial coefficients above, summed over all possible $r$ values:
$$ f(m,n)= \sum_{r=0}^{\left\lfloor n/2\right\rfloor} \binom{m-n+1}{n-r}\binom{n-r}{r} $$