In the attempt to answer Binomial identity involving Harmonic numbers we stumbled on the following problem. Let $i\ge 0 $ and $k \ge i+2 $ and $p \ge 1$ be integers. Consider a following sum:
\begin{equation} {\mathfrak S}^k_{i,p} := \sum\limits_{j=2}^{k-i} \binom{k}{i+j} \frac{(-1)^j}{(j-1)^p} \quad (i) \end{equation}
By using the usual trick $1/j^p = \int\limits_0^\infty \frac{t^{p-1}}{(p-1)!} e^{-j t} dt$ and then by summing up the resulting binomial series and finally by using the Faa di Bruno formula we have arrived at the following:
\begin{eqnarray} {\mathfrak S}^k_{i,p} &=& \lim\limits_{\theta \rightarrow 0} \left[ \frac{(-1)^{p-1+i}}{(p-1)!} \frac{d^{p-1}}{d \theta^{p-1}} \left[ B(k+1,\theta-i-1) \right] - \sum\limits_{j=0}^{i+1} \binom{k}{j} (-1)^{j+i} \frac{1}{(\theta-i+j-1)^p} \right] \\ &=& \lim\limits_{\theta \rightarrow 0} \left[(-1)^{p-1+i} B(k+1,\theta-i-1) \sum\limits_{\sum\limits_{\xi=1}^{p-1} \xi m_\xi =p-1} \frac{\prod\limits_{\xi=1}^{p-1} \left( \psi^{(\xi-1)}(\theta-i-1) - \psi^{(\xi-1)}(\theta-i+k)\right)^{m_\xi}}{\prod\limits_{\xi=1}^{p-1} (\xi!)^{m_\xi} m_\xi!} - \sum\limits_{j=0}^{i+1} \binom{k}{j} (-1)^{j+i} \frac{1}{(\theta-i+j-1)^p} \right] \quad(ii) \end{eqnarray} In the second line $\psi^{(j)}$ is the polygamma function .
The code below verifies the result:
{i} = RandomInteger[{1, 5}, 1]; p = RandomInteger[{3, 6}];
l1 = Table[
Sum[ Binomial[k, j + i] (-1)^j/(j - 1)^p, {j, 2, k - i}], {k, i + 2,
10}]
N[l1]
th =.;
l2 = Table[
Normal[Series[(-1)^(p - 1 + i)/(p - 1)! D[
Beta[k + 1, th - i - 1] -
Sum[ Binomial[k, j] (-1)^j 1/(th - i + j - 1), {j, 0,
i + 1}], {th, p - 1}], {th, 0, 0}]], {k, i + 2, 10}] // N
l3 = Table[
Normal[Series[
Beta[k + 1, th - i - 1] Total[
DeleteCases[
Flatten[Table[
With[{mm = (p - 1 - Sum[xi m[xi], {xi, 1, p - 2}])/(p -
1)}, If[
Element[mm,
Integers], (-1)^(p - 1 + i)/
Product[
m[xi]!, {xi, 1, p - 1}] Product[((
PolyGamma[xi - 1, th - i - 1] -
PolyGamma[xi - 1, th - i + k])/xi!)^
m[xi], {xi, 1, p - 1}] /. m[p - 1] :> mm, {}]],
Evaluate[
Sequence @@
Table[{m[q], 0,
Floor[(p - 1 - Sum[xi m[xi], {xi, 1, q - 1}])/q]}, {q,
1, p - 2}]]], p - 3], {}]] -
Sum[ Binomial[k, j] (-1)^j (-1)^(i)/(th - i + j - 1)^p, {j, 0,
i + 1}], {th, 0, 0}]], {k, i + 2, 10}];
FullSimplify[l3]
In particular we have: \begin{eqnarray} {\mathfrak S}^{k}_{i,1} &=& \binom{k}{i+1} \frac{1}{1!} \left( H_{k-i-1} - H_{i+1} \right) - \sum\limits_{j=0}^i \binom{k}{j} \frac{(-1)^{j+i}}{(j-i-1)^1} \\ {\mathfrak S}^{k}_{i,2} &=& \binom{k}{i+1} \frac{1}{2!} \left( (H_{k-i-1}-H_{i+1})^2 + H_{i+1}^{(2)} + H_{k-i-1}^{(2)} \right) - \sum\limits_{j=0}^i \binom{k}{j} \frac{(-1)^{j+i}}{(j-i-1)^2} \\ {\mathfrak S}^{k}_{i,3} &=& \binom{k}{i+1} \frac{1}{3!} \left( (H_{k-i-1}-H_{i+1})^3 + 3 (H_{i+1}^{(2)} + H_{k-i-1}^{(2)}) \cdot (H_{k-i-1}-H_{i+1}) + 2 (H_{k-i-1}^{(3)} - H_{i+1}^{(3)}) \right)- \sum\limits_{j=0}^i \binom{k}{j} \frac{(-1)^{j+i}}{(j-i-1)^3} \end{eqnarray}
In[127]:= Clear[H];
H[i_] := HarmonicNumber[i];
H[i_, p_] := HarmonicNumber[i, p];
l1 = Table[
Sum[ Binomial[k, j + i] (-1)^j/(j - 1)^1, {j, 2, k - i}], {i, 0,
5}, {k, i + 2, 10}];
l2 = Table[
Binomial[k,
i + 1] (HarmonicNumber[k - i - 1] - HarmonicNumber[i + 1]) -
Sum[ Binomial[k, j] (-1)^(j + i)/(j - i - 1), {j, 0, i}], {i, 0,
5}, {k, i + 2, 10}];
l1 - l2
l1 = Table[
Sum[ Binomial[k, j + i] (-1)^j/(j - 1)^2, {j, 2, k - i}], {i, 0,
5}, {k, i + 2, 10}];
l2 = Table[
Binomial[k, i + 1] 1/
2 ((H[1 + i] - H[k - i - 1])^2 + H[1 + i, 2] +
H[-1 - i + k, 2]) -
Sum[ Binomial[k, j] (-1)^(j + i)/(j - i - 1)^2, {j, 0, i}], {i, 0,
5}, {k, i + 2, 10}];
l1 - l2
l1 = Table[
Sum[ Binomial[k, j + i] (-1)^j/(j - 1)^3, {j, 2, k - i}], {i, 0,
5}, {k, i + 2, 10}];
l2 = Table[
Binomial[k, i + 1] 1/
6 ((-H[1 + i] + H[k - i - 1])^3 +
3 (H[1 + i, 2] + H[-1 - i + k, 2]) (-H[1 + i] +
H[-1 - i + k]) + 2 (-H[1 + i, 3] + H[-1 - i + k, 3])) -
Sum[ Binomial[k, j] (-1)^(j + i)/(j - i - 1)^3, {j, 0, i}], {i, 0,
5}, {k, i + 2, 10}];
l1 - l2
Out[132]= {{0, 0, 0, 0, 0, 0, 0, 0, 0}, {0, 0, 0, 0, 0, 0, 0, 0}, {0,
0, 0, 0, 0, 0, 0}, {0, 0, 0, 0, 0, 0}, {0, 0, 0, 0, 0}, {0, 0, 0, 0}}
Out[135]= {{0, 0, 0, 0, 0, 0, 0, 0, 0}, {0, 0, 0, 0, 0, 0, 0, 0}, {0,
0, 0, 0, 0, 0, 0}, {0, 0, 0, 0, 0, 0}, {0, 0, 0, 0, 0}, {0, 0, 0, 0}}
Out[138]= {{0, 0, 0, 0, 0, 0, 0, 0, 0}, {0, 0, 0, 0, 0, 0, 0, 0}, {0,
0, 0, 0, 0, 0, 0}, {0, 0, 0, 0, 0, 0}, {0, 0, 0, 0, 0}, {0, 0, 0,
0}}
Unfortunately the result $(ii)$ is highly singular and it is hard to carry out the limit $\theta \rightarrow 0$ in full generality. My question would therefore be how to perform this limit and obtain a closed form result for the sum in question.

Now let us assume that $p \ge 2$ is an arbitrary integer. We also define coefficients as below:
\begin{equation} {\mathcal S}^{\pm}_p(i):= \frac{1}{p!} \left. \frac{d^p}{d x^p} e^{\sum\limits_{q=1}^\infty (\mp 1)^{q-1} \frac{H^{(q)}_i}{q} x^q } \right|_{x=0} \end{equation}
We compute the first term in the right hand side as follows:
\begin{eqnarray} &&\frac{(-1)^{p-1+i}}{(p-1)!} \frac{d^{p-1}}{d \theta^{p-1}} B(k+1,\theta-i-1) =\\ &&\frac{(-1)^{p-1+i}}{(p-1)!} \frac{d^{p-1}}{d \theta^{p-1}} \left(\frac{1}{\prod\limits_{p=1}^{i+1} (\theta-p)} \cdot \frac{k!}{\theta} \cdot \frac{1}{ \prod\limits_{p=1}^{k-i-1} (\theta+p)} \right) = \\ &&\frac{(-1)^{p}}{(p-1)!} \binom{k}{i+1} \frac{d^{p-1}}{d \theta^{p-1}} \left(\frac{1}{\prod\limits_{p=1}^{i+1} (-\frac{\theta}{p}+1)} \cdot \frac{1}{\theta} \cdot \frac{1}{ \prod\limits_{p=1}^{k-i-1} (\frac{\theta}{p}+1)} \right) = \\ &&\frac{(-1)^{p}}{(p-1)!} \binom{k}{i+1} \frac{d^{p-1}}{d \theta^{p-1}} \left( \frac{1}{\theta} \cdot \frac{1}{\left[1+ \sum\limits_{q_1=1}^\infty (-\theta)^{q_1} {\mathcal S}^{+}_{q_1}(i+1) \right]} \cdot \frac{1}{\left[1+ \sum\limits_{q_2=1}^\infty (+\theta)^{q_2} {\mathcal S}^{+}_{q_2}(k-i-1) \right]} \right) = \\ && \frac{(-1)^{p}}{(p-1)!} \binom{k}{i+1} \frac{d^{p-1}}{d \theta^{p-1}} \left( \frac{1}{\theta} \cdot \sum\limits_{q_1=0}^\infty (+\theta)^{q_1} {\mathcal S}^{-}_{q_1}(i+1) \cdot \sum\limits_{q_2=0}^\infty (-\theta)^{q_2} {\mathcal S}^{-}_{q_2}(k-i-1) \right) = \\ && \frac{(-1)^{p}}{(p-1)!} \binom{k}{i+1} \frac{d^{p-1}}{d \theta^{p-1}} \left( \frac{1}{\theta} + (-1)^q \sum\limits_{q=1}^\infty \theta^{q-1} \sum\limits_{q_1+q_2=q} (-1)^{q_1} {\mathcal S}^{-}_{q_1}(i+1) \cdot {\mathcal S}^{-}_{q_2}(k-i-1) \right) = \\ && \binom{k}{i+1} \left[ \frac{(-1)^1}{\theta^p} + \left(\sum\limits_{q_1+q_2=p} (-1)^{q_1} {\mathcal S}^{-}_{q_1}(i+1) \cdot {\mathcal S}^{-}_{q_2}(k-i-1) \right) + O\left(\theta^{p+1} \right) \right] \end{eqnarray}
Now we can see that the first term in the square brackets cancels the last term , i.e. the $j=i+1$ term in the last sum in $(ii)$ . The singularities cancel out nicely and now it is straightforward to take the limit . So the result reads:
\begin{eqnarray} {\mathfrak S}^k_{i,p} &=& \binom{k}{i+1} \left[\sum\limits_{q_1=0}^p (-1)^{q_1}{\mathcal S}^{-}_{q_1}(i+1) \cdot {\mathcal S}^{-}_{p-q_1}(k-i-1)\right] - \sum\limits_{j=0}^i \binom{k}{j} \frac{(-1)^{j+i}}{(j-i-1)^p} \\ &=& \binom{k}{i+1} \cdot \left( \sum\limits_{\sum\limits_{j=1}^p j \cdot m_j = p} \frac{1}{\prod\limits_{j=1}^p m_j!} \prod\limits_{j=1}^p \left[ \frac{(-1)^j H^{(j)}_{i+1} + H^{(j)}_{k-i-1}}{j} \right]^{m_j} \right) - \sum\limits_{j=0}^i \binom{k}{j} \frac{(-1)^{j+i}}{(j-i-1)^p} \end{eqnarray}
Update: Here we also partially evaluate the remaining sum. Take $i \ge 0 $ and $\xi =0,\cdots, i$ be integers then let us firstly define some coefficients:
\begin{eqnarray} {\mathcal A}^{(\xi)}_{i,p} &:=& \sum\limits_{j=0}^\xi \binom{i+1}{\xi-j} \frac{1}{(\xi-j-i-1)^{p-1}} \binom{-i+\xi-1}{j} \\ &=& \left\{ \begin{array}{rrr} 1 & \quad \mbox{if $p=1$}\\ H_{i-\xi}- H_{i+1} & \quad \mbox{if $p=2$} \\ \frac{1}{2} \left( (H_{i+1}^{(2)} - H_{i-\xi}^{(2)}) + (H_{i+1} - H_{i-\xi})^2 \right) & \quad \mbox{if $p=3$} \\ \vdots \end{array} \right. \end{eqnarray}
now our remaining sum reads:
\begin{eqnarray} \sum\limits_{j=0}^i \binom{k}{j} \frac{(-1)^{j+i}}{(j-i-1)^p} &=& -\binom{k}{i+1} \cdot \sum\limits_{\xi=0}^i \frac{1}{(k-\xi)} \cdot {\mathcal A}^{(\xi)}_{i,p} \quad (i) \end{eqnarray}
Equation $(i)$ comes from factoring out the the binomial factor and then decomposing the rest into partial fractions in the variable $k$ and then all the similar terms together and simplifying.