I'm reading this and am trying to unpack the definition of $Alg_2(k)$ found at the bottom of page 21 so I'm wondering if the following is a good example of what's being described.
Let $R= M_{n\times n}(\mathbb{R})$ where $M_{n\times n}(\mathbb{R})$ is the ring of $n\times n$ matrices. Now let $R$ be a module by considering it as a module over itself. Similary, let $S$ be the module of $m\times m$ matrices. Now, it's easy to show that $M_{n\times m}(\mathbb{R})$ is an $R$-$S$ bimodule. We also have the following map. Take $A\in M_{n\times m}(\mathbb{R})$, then $A$ defines a map from $ M_{n\times n}(\mathbb{R})\to M_{m\times m}(\mathbb{R})$ by $x\mapsto A^Tx A$.
My question is, is the map $x\mapsto A^Tx A$ an element of $\operatorname{Maps}_{Alg_2(k)}(R,S)$?
Edit: Fixed some typos and tried to make my notation more clear. Also, other examples of elements of $\operatorname{Maps}_{Alg_2(k)}(R,S)$ are welcome.
Edit2: Incorporated notation suggestions.
Have a look at the nLab article; it should clear up any confusion.
Briefly, the 2-category they are describing has $k$-algebras as the objects, and a 1-morphism from a $k$-algebra $A$ to a $k$-algebra $B$ is an $(A,B)$-bimodule $V$. Bimodules are composed using the tensor product. So given a $(B,C)$-bimodule $W$, you can compose with $V$ by tensoring over $B.$ The $(A,C)$-bimodule $V \otimes_B W$ is the composite of $V$ and $W$. Thus, the identity 1-morphism on a $k$-algebra $A$ is just $A$ itself viewed as an $A$-bialgebra. Indeed, tensoring $A \otimes_A V \cong V.$
Since bimodules are the 1-morphisms, 2-morphisms must take us between bimodules, and the obvious notion here is the correct one: 2-morphisms are just bimodule homomorphisms. The vertical composition is just the usual composition of bimodule homomorphisms.
If it still doesn't make sense to you, you probably need to review tensor products and bimodules in your favourite abstract algebra text (such as Aluffi, for example).
Note: I have used the opposite of the usual composition convention for the 1-morphisms, because it is easier to follow in this situation.