TL;DR: I'm trying to find the first-order condition (FOC) for an optimization problem with two state variables and one control variable. I don't want the value function $V$ to appear in the FOC but can't get rid of it. Help is highly appreciated. $\def\d{\delta} \def\b{\beta} \def\g{\gamma} \def\G{\Gamma}$
A decision-maker needs to choose $a$ in every stage, payoffs are given according to a function $w$ that depends on $a$ and on the state, which is a vector with two elements $s_1,s_2$. Only the first if directly payoff relevant, but they both affect some nice function $b$ which determines the state tomorrow. Payoffs are discounted, time is discrete. The goal is to find the first-order condition in terms of all these parameters to make optimal choices.
Here is a formal statement of the problem and the calculation up until the moment where I'm stuck.
Let $s^t=\begin{pmatrix} s^t_1 \\ s^t_2 \end{pmatrix}$ be the state vector at time $t$, $b:\mathbf{R}^2\to \mathbf{R}$ a known "nice" function, $\b, \d \in (0,1)$ constants. $w(a^t,s^t_1)$ is the single stage payoff that depends on the action $a^t$ and the first element of the state. The states evolve according to $s^{t+1}=\begin{pmatrix} s^{t+1}_1 \\ s^{t+1}_2 \end{pmatrix}=\d \begin{pmatrix} s^t_1 \\ s^t_2 \end{pmatrix}+\G \begin{pmatrix} a^t \\ b(s^t) \end{pmatrix}$ where $$ \Gamma= \begin{pmatrix} \gamma_{11} & \gamma_{12}\\ \gamma_{21} & \gamma_{22} \end{pmatrix}$$
The optimization problem is therefore $$V\begin{pmatrix} s_1 \\ s_2\end{pmatrix}=\max\limits_{a\in[0,1]} \left[(1-\beta)w(a,s_1)+\beta V\begin{pmatrix} \d s_1+ \g_{11} a + \g_{12}b(s) \\ \d s_2+ \g_{21} a + \g_{22} b(s) \end{pmatrix}\right]$$
Denote the $[\ldots]$ by $L^t$ (the Lagrangian at time $t$). The First-order condition is $\tfrac{\partial L^t}{\partial a^t}=0$: \begin{equation}\label{foct} (1-\b)w_1(a^{t},s_1^t)+\b \left[\tfrac{\partial V}{\partial s_1^{t+1}}\g_{11}+\tfrac{\partial V}{\partial s_2^{t+1}}\g_{21} \right]=0 \end{equation} and the similar expression at time $t+1$ is: \begin{equation}\label{foct+1} (1-\b)w_1(a^{t+1},s_1^{t+1})+\b \left[\tfrac{\partial V}{\partial s_1^{t+2}}\g_{11}+\tfrac{\partial V}{\partial s_2^{t+2}}\g_{21} \right]=0 \end{equation} According to the envelope theorem: \begin{eqnarray}\label{envt} \tfrac{\partial V}{\partial s_1^t}&=&(1-\b) w_2(a^t,s_1^t) + \b \left[\tfrac{\partial V}{\partial s_1^{t+1}}(\d+\g_{12}b_1(s^t))+\tfrac{\partial V}{\partial s_2^{t+1}}\g_{22}b_1(s^t) \right] \\ \tfrac{\partial V}{\partial s_2^t}&=&\b \left[\tfrac{\partial V}{\partial s_1^{t+1}}\g_{12}b_2(s^t)+\tfrac{\partial V}{\partial s_2^{t+1}}(\d+\g_{22}b_2(s^t)) \right] \end{eqnarray} and for time $t+1$: \begin{eqnarray}\label{envt+1} \tfrac{\partial V}{\partial s_1^{t+1}}&=&(1-\b) w_2(a^{t+1},s_1^{t+1}) + \b \left[\tfrac{\partial V}{\partial s_1^{t+2}}(\d+\g_{12}b_1(s^{t+1}))+\tfrac{\partial V}{\partial s_2^{t+2}}\g_{22}b_1(s^{t+1}) \right] \\ \tfrac{\partial V}{\partial s_2^{t+1}}&=&\b \left[\tfrac{\partial V}{\partial s_1^{t+2}}\g_{12}b_2(s^{t+1})+\tfrac{\partial V}{\partial s_2^{t+2}}(\d+\g_{22}b_2(s^{t+1})) \right] \end{eqnarray}
We can sum these two equations to obtain \begin{eqnarray} \gamma_{11}\tfrac{\partial V}{\partial s_1^{t+1}}+\gamma_{21}\tfrac{\partial V}{\partial s_2^{t+1}}&=& (1-\b) w_2(a^{t+1},s_1^{t+1})\gamma_{11} \nonumber \\ &+&\b\gamma_{11} \left[\tfrac{\partial V}{\partial s_1^{t+2}}(\d+\g_{12}b_1(s^{t+1}))+\tfrac{\partial V}{\partial s_2^{t+2}}\g_{22}b_1(s^{t+1}) \right] \nonumber \\ &+&\b\gamma_{21} \left[\tfrac{\partial V}{\partial s_1^{t+2}}\g_{12}b_2(s^{t+1})+\tfrac{\partial V}{\partial s_2^{t+2}}(\d+\g_{22}b_2(s^{t+1})) \right] \nonumber \end{eqnarray} The $\delta$ part of this is $$\beta\delta \left[\gamma_{11}\tfrac{\partial V}{\partial s_1^{t+2}}+\gamma_{21}\tfrac{\partial V}{\partial s_1^{t+2}} \right]=-\delta (1-\beta) w_1(a^{t+1},s_1^{t+1})$$ The rest is \begin{eqnarray} \b\left[\gamma_{11}b_1(s^{t+1})+\gamma_{21}b_2(s^{t+1}) \right]\left[\gamma_{12}\tfrac{\partial V}{\partial s_1^{t+2}}+ \gamma_{22} \tfrac{\partial V}{\partial s_2^{t+2}}\right] \end{eqnarray} Hence the FOC has the following form (also divide by $1-\b$): \begin{eqnarray}\label{foct_neet} w_1(a^{t},s_1^t)+\b \left[w_2(a^{t+1},s_1^{t+1})\gamma_{11} -\delta w_1(a^{t+1},s_1^{t+1})\right]\nonumber \\ +\tfrac{\b}{1-\b}\left[\gamma_{11}b_1(s^{t+1})+\gamma_{21}b_2(s^{t+1}) \right]\left[\gamma_{12}\tfrac{\partial V}{\partial s_1^{t+2}}+ \gamma_{22} \tfrac{\partial V}{\partial s_2^{t+2}}\right]=0 \end{eqnarray} Where the first line is fine but the second line is the problematic one. How one can get rid of $V$ from this equation? Any conditions or equations that I can write that I'm missing?
(Any references about this kind of problems are also welcome)
Added in edit: I have here 6 linear equations in 6 variables so in theory I can solve it and finish with it. However, I want to generalize it to $n$-dimensional state space, and such approach won't work there.