Suppose I have a subset $\mathcal{A}\subset \mathcal{P}(n)$ for some $n$.
Are there extension of the notion of colexicographic order that apply to ordering of these sets of different sizes?
I have only ever seen a colexigraphic ordering applied to sets of the same sizes, but if one uses the definition $A<B$ iff $\text{max}\{A\triangle B\} \in B$ then it can apply to sets of different sizes too.
Yes.
The one extension I know about is the following (I borrowed it from On the trace of finite sets by Peter Frankl):
$A < B$ iff $A \subset B$ or $\max(A \triangle B) \in B$.
One of the applications is a theorem by Rudolf Ahlswede and Gyula O. H. Katona (which is a generalization/corollary of the Kruskal-Katona theorem). Define $L(n,m)$ as the first $m$ subsets of $\{1,\ldots,n\}$ in colexicographic order.
Theorem. Let $\mathcal{F}$ be a downward-closed (sometimes called hereditary) family of subsets of $\{1,\ldots,n\}$, $\left|\mathcal{F}\right| = m$. Then for every monotone non-increasing function $f\colon \mathbb{N} \to \mathbb{R}$ we have $$ \sum_{F \in \mathcal{F}} f(\left|F\right|) \ge \sum_{F \in L(n, m)} f(\left|F\right|). $$