I have a fact which I haven't been able to prove. I briefly touched upon it in The first Kirby move and $\mathbb{C}P^2$.
Let $M$ be a manifold-with-boundary, and let $N$ be a closed manifold, both of same dimension.
We have $M\#N$, the connect sum of both.
We have also $M\natural (N-B^4)$, which is done by removing an open 4-ball in $N$, and instead of puncturing the interior of $M$ as in the connected sum we attach $N-B^4$ to $M$ along an embedded $D^3$ of the boundaries of $M$ and $N-B^4$ (which are $\partial M$ and $S^3$).
Claim: The manifolds $M\# N$ and $M\natural (N-B^4)$ are diffeomorphic.
I drew some pictures and it's pretty believable, but I'd like to see some rigorous proof of this.
I have the following pictures:
Let the manifolds have dimension $4$. Then we can do connect sum of $M$ and $N$ by cutting out a 4-ball from both, taking a cylinder $S^3\times I$, then gluing to $M$ at $S^3\times\{0\}$ and to $N$ at $S^3\times\{1\}$. This is Fig.1.
Now we restrict ourselves to $M$. We find a $D^3\times I$ in $M$ (a band) such that $D^3\times\{0\}$ lies in the boundary of $M$ and $D^3\times \{1\}$ lies in the $S^3$ left in the interior of $M$ after removing an open 4-ball. In fact, we can think of this open ball as the same open 4-ball removed to perform a connect sum with $N$. (This band exists because we can take the open 4-ball to lie in the interior of a boundary chart of $M$, so we are doing this in $\mathbb{R}^4_+$).
We now remove this band $D^3\times I$, or more specifically, $\mathring{D^3}\times I$, leaving $S^2\times I$ behind. This is done in Fig.2.
Claim: Removing this open 4-ball and then the band gives a manifold diffeomorphic to $M$.
Again, this is easy to see by performing these removals within a boundary coordinate neighborhood of $M$.
Using the claim, we can perform boundary connected sum of $M$ and $N-B^4$ by going from Fig.2 to Fig.3. In Fig.3 the red arcs in both $M$ and $N-B^4$ are the $D^3$'s that were chosen to perform the boundary connected sum.
It is represented as taking a $D^3\times I$ and gluing $D^3\times\{0\}$ to $M$ along the left $D^3$, and gluing $D^3\times\{1\}$ to $N$ along the right $D^3$.
The boundary in Fig.3 is composed of the boundary of $M$, $\partial D^3\times I= S^2\times I$ (the red lines going across), the boundary of $N-B^4$, but with the two chosen red arcs (the $D^3$'s) removed.
We note this boundary connected sum operation above changed the boundary of $M\sqcup (N-B^4)$ by connect summming the boundary of $M$ with that of $N-B^4$ which is $S^3$. In particular, the boundary is diffeomorphically the same as $\partial M$ after the operation.
We can now isotope this resulting manifold by pulling the part of the boundary lying in the copy of $N-B^4$ towards the copy of $M$, resulting in Fig.4, at least for the initial stage.
If we keep isotopying the manifold like that, we can make sure that its boundary is pulled completely into $M$, and then finally back up towards the original boundary of $M$, following the trace of the band $S^2\times I$ we removed in Fig.2.
This shows that the boundary connected sum $M\natural (N-B^4)$ (Fig.3) is diffeomorphic (through isotopies!) to $M\# N$ (Fig.1), as claimed.