Given a point $x$ and a bounding box $B$ - let's say we have the unit normals $N_i$ of the sides (pointing inwards) and one point on each side $P_i$ - we can check if $x$ is inside $B$ as follows:
$\forall{i}:(x - P_i)^T N_i > 0$
If, for use in quadratic programming, we want to write this constraint in the form of $Ax > b$ (let's say in 2D) we can write
$A = \begin{bmatrix} N_1 & N_2 \end{bmatrix}, \quad b = \begin{bmatrix} P_1^T N_1 \\ P_2^T N_2 \end{bmatrix}$.
This works for checking if a given point is inside a given bounding box.
If I want to constrain a given point to be outside of the bounding box, things seem to be more difficult. The problem is that the constraint is now as follows:
$\exists{i} : (x - P_i)^T N_i < 0.$
This change from a for-all operator to an exists-operator means I somehow have to write these constraints using an or-operator. Is there still some way for me to write these constraints in the form of $Ax > b$? Or should I look elsewhere entirely for solving a problem of this form?
I am not sure I get the point but in my understanding your problem may be formulated as
$min ||d||_2 $
$ x_i -d \notin B, \quad \forall i=1,...,n$
that is minimizing a displacement $d$ such that a set of $n$ points $x_i$ in $R^k$ are outside a box
$B= \Pi_{j=1}^k [l_j,b_j]$
You can require also the points to be in a larger container if you want. This is a not convex problem in general. Although a feasible solution may be easy to find, in general there are multiple local optima. Moreover, you cannot explicitly handle a $ \notin B$ constraints, unless you convert in a function that measure how much this constraint is violated.
I suggest you to look at this paper I published some years ago
Cassioli, Andrea, and Marco Locatelli. "A heuristic approach for packing identical rectangles in convex regions." Computers & Operations Research 38.9 (2011): 1342-1350.
In there you can find references and a way to measure the how deep a point is in a rectangle. Such measure can be used to define a constraint like $f(y)<=0$ that you can include in your optimization model with some tolerance:
$min ||d||_2 $
$ f(x_i -d) <=\epsilon, \quad \forall i=1,...,n$
It is still non convex but then you can apply local searches, heuristics or a deterministic global optimization method.
Note also that the case
$min ||d||_2 $
$ x_i -d \in B, \quad \forall i=1,...,n$
is a convex quadratic problem that does not need any quadratic term for the box inclusion. Any quadratic programming solver (as MOSEK, CPLEX,...) will handle directly in this form.