So I was wondering:
If we have a structure that has the universe $A$ and only relations in it, then any structure with universe $B \subseteq A$ and the same relations, IS a substructure?
I feel like this is true. If this is true, does it follow that any 2 structures that have the same relations and functions, are each others substructures/superstructures only if the function in the superstructure is closed in its universe.
Yes, any subset of a relational structure is (the underlying set of) a substructure. Re: your second question, the answer is also yes if I'm interpreting it correctly: if $\mathcal{A}$ is a structure with underlying set $A$, and $B\subseteq A$ is a subset which is closed under the function symbols of $\mathcal{A}$, then $B$ is (the underlying set of) a substructure of $\mathcal{A}$.
Put another way: the only way "substructure" differs from "subset" is by requiring closure under the function symbols. (Note that I'm thinking of constants as functions, here - a constant symbol is a $0$-ary function symbol.)