We need to find a maximal value of $k$, for which the inequality
$$x^2+xy+3y^2-x-4y+1\geq k$$ is true for all reals $x$ and $y$ such that $x+y\geq0$ and $x\leq1.$
We have $$3y^2+(x-4)y+x^2-x+1-k\geq0,$$ for which we need
$$(x-4)^2-12(x^2-x+1-k)\leq0$$ or
$$11x^2-4x-4-12k\geq0,$$ for which we need
$$4-11(-4-12k)\leq0$$ or
$$k\leq-\frac{4}{11}.$$
The equality occurs for $(x,y)=\left(\frac{2}{11},\frac{7}{11}\right)$ and we see that the condition holds.
Thus, $-\frac{4}{11}$ is a minimal value of our expression.
We need to find a maximal value of $k$, for which the inequality $$x^2+xy+3y^2-x-4y+1\geq k$$ is true for all reals $x$ and $y$ such that $x+y\geq0$ and $x\leq1.$
We have $$3y^2+(x-4)y+x^2-x+1-k\geq0,$$ for which we need $$(x-4)^2-12(x^2-x+1-k)\leq0$$ or $$11x^2-4x-4-12k\geq0,$$ for which we need $$4-11(-4-12k)\leq0$$ or $$k\leq-\frac{4}{11}.$$ The equality occurs for $(x,y)=\left(\frac{2}{11},\frac{7}{11}\right)$ and we see that the condition holds.
Thus, $-\frac{4}{11}$ is a minimal value of our expression.