The concepts of bi-definability and bi-interpretability in model theory give precise meanings to what we mean by two structures being essentially the same.
Some structures, however, are almost literally identical. For instance, a complete Boolean algebra $B$ , and a poset $\mathbb P$ obtained from it by removing the top element (or the bottom element, depending on whether you are Saharon Shelah). Now, $\Bbb P$ is trivially definable in $B$, but I don't know if $B$ is definable or even interpretable in $\Bbb P$, even though the two structures are essentially the same (I believe), and they are almost identical.
Given this, my question is: can a structure $M$ that differs from another $N$ only by lacking an element named by a constant symbol (or, presumably equivalently, finitely many elements named by finitely many constants) have important model-theoretic properties, such as interpretability strength, that are different from those of $N$?
To make the setting precise, suppose $B$ is an $L$-structure containing a constant symbol $c$. Let $L' = L\setminus \{c\}$, and let $A = B\setminus \{c^B\}$. Assume further that $A$ is an $L'$-substructure of $B|_{L'}$ (this is trivial when $L'$ is a relational language, but in the general case this amounts to assuming that $c^B$ is not the interpretation of any constant symbols other than $c$, and that it is not the output of the interpretation of any function symbol, when evaluated on inputs from $A$).
As you say in the quesiton, it is clear that $A$ is interpretable (even definable) in $B$. And we can ask whether $B$ is interpretable in $A$ (and stronger, whether they're bi-interpretable).
In general, the answer is no. For example, consider the structure $(Q,c,R)$, where $Q = \mathbb{Q}\cup \{c\}$, $c$ is interpreted as $c$, and $R(x,y,z)$ holds iff $x\leq y$ and $z = c$. Then if you remove $c$, you get $\mathbb{Q}$ equipped with a ternary relation which holds of no triples. This extremely trivial structure obviously does not interpret $Q$. For example, it is stable, while $(Q,c,R)$ is unstable.
There's another silly obstruction in finite structures: If $|B| = 1$ or $2$, then $|A| = 0$ or $1$, and then $A$ cannot interpret $B$. Indeed, if $|A| = 0$, then $|A^k| = 0$, so an empty structure cannot interpret a non-empty structure. And if $|A| = 1$, then $|A^k| = 1$, so a structure of size $1$ cannot interpret a structure of size greater than $1$.
But sometimes the answer is yes. If $A$ has at least two elements, then we can try to interpret $B$ in $A$ as follows: The domain of the interpreted structure will be $A^2$, and equality will be defined by $(a_1,a_2)\equiv (a_1',a_2')$ iff either (1) $a_1 = a_2 = a_1' = a_2'$ or (2) $a_1\neq a_2$ and $a_1'\neq a_2'$. The equivalence classes consist of $C_a = \{(a,a)\}$ for each $a\in A$, together with one other class $C = \{(a_1,a_2)\mid a_1\neq a_2\}$ (the existence of this class is where we use the assumption $|A|\geq 2$).
Now we turn $A^2/\equiv$ into an $L$-structure as follows. We interpret $c$ as $C$. For each symbol in $L'$, we definite it by cases: on a tuple which does not include $C$, e.g. $(C_{a_1},\dots,C_{a_k})$ we interpret the symbol just as we do on $(a_1,\dots,a_k)$ in $A$. For tuples including $c$, break into cases according to which positions in the tuple $c$ appears in. But here we can potentially run into trouble. We need conditions like the following: if $R$ is a binary relation symbol, then $\{a\in A\mid B\models R(a,c)\}$ is definable in $A$.
In your example of removing the top element from a poset, these conditions are satisfied. That is, the way the top element relates to every other element is trivially definable: it's greater than every other element and not less than any other element. The same will be true, for example, of the identity element of a monoid (though the monoid had better have no non-identity units, in order to make removing the identity make sense). When the construction above goes through, it's easy to see that it gives a bi-interpretation between $A$ and $B$, so they share all model-theoretic properties that are invariant under bi-interpretation.