I have such stochastic process with which I struggle all day, finally I found 2 mistakes, however answer is still unsatisfying. $$X_t = atW_t^2 - \int_0^t(W_s^2+s)ds,$$ I need to check if it is a martingale. I simply write Ito formula for $X_t(t,W_t,S_t)$, where I denote by $S_t = \int_0^tW_s^2ds$. So I apply it, and get: $$dX_t= (aW_t^2 - t -W_t^2 +at )dt + 2atW_tdW_t,$$ I pick function $f_t$, which should be equal to zero if the process is a martingale. $$aW_t^2 - t -W_t^2 +at = 0,$$ so my $a$ is $a = 1$.
BUT! I have decided to check it substituting the $a$ into the $X_t$ and calculating expectation(using martingale property ($\mathbb{E}(X_t \mid \mathcal{F_s}) = X_s$), and I get problems. As far as I understand I should get: $$\mathbb{E} \left(tW_t^2- \int_0^t(W_s^2+s)ds \mid \mathcal{F_s} \right) = sW_s^2 - \int_0^s(W_u^2+u)du$$ But what I get is: $$\mathbb{E}(tW_t^2 \mid \mathcal{F_s})=tW_s^2 + t^2 -ts$$ $$\mathbb{E} \left( \int_0^tW_s^2ds \mid \mathcal{F_s} \right)= \int_0^sW_u^2du +st -s^2$$ $$\mathbb{E} \left(t- \int_0^tsds \mid \mathcal{F_s} \right) = t - t^2/2$$ Substituting: $$\mathbb{E} \left(tW_t^2 - \int_0^t(W_s^2+s)ds \mid \mathcal{F_s} \right) = tW_s^2 + t^2 -ts - \int_0^sW_u^2du - st + s^2 -t^2/2$$ which is does not look like needed: \begin{align} \mathbb{E} \left(tW_t^2- \int_0^t(W_s^2+s)ds \mid \mathcal{F_s} \right) &= sW_s^2 - \int_0^sW_u^2du - s^2/2\\ & = tW_s^2 - \int_0^sW_u^2du+ s^2 - 2st +t^2/2 \end{align}
Without your (detailed) calculations it is hard to say what you did wrong. So let's do it step by step:
It is not difficult to check that $(W_t^2-t)$ is a martingale. Therefore, we have
$$\begin{align*} \mathbb{E}(t W_t^2 \mid \mathcal{F}_s) &= t \mathbb{E}((W_t^2-t)+t) \mid \mathcal{F}_s) \\ &= t (W_s^2-s+t) = t W_s^2 + t^2-ts. \end{align*}$$
That agrees with your result for this term. Now the next one.
$$\begin{align*} \mathbb{E} \left( \int_0^t W_u^2 \, du \mid \mathcal{F}_s \right) &= \int_0^s W_u^2 \, du + \mathbb{E} \left( \int_s^t W_u^2 \, du \mid \mathcal{F}_s \right) \\ &= \int_0^s W_u^2 \, du + \mathbb{E} \left( \int_s^t ((W_u-W_s)+W_s)^2 \, du \mid \mathcal{F}_s \right) \\ &= \int_0^s W_u^2 \, du + \int_s^t \mathbb{E}((W_u-W_s)^2) \, du + 2W_s \int_s^t \underbrace{\mathbb{E}(W_u-W_s)}_{0} \, du \\ &\quad + W_s^2 (t-s) \\ &= \int_0^s W_u^2 \, du + \int_s^t (u-s) \, du + W_s^2 (t-s) \\ &= \int_0^s W_u^2 \, du + \frac{(t-s)^2}{2} + W_s^2 (t-s). \end{align*}$$
This is different from your result. The third one is again correct. Adding all up, we get
$$\begin{align*} \mathbb{E}(X_t \mid \mathcal{F}_s) &= t W_s^2 + t^2-ts - \left( \int_0^s W_u^2 \, du + \frac{(t-s)^2}{2} + W_s^2 (t-s) \right) - \frac{t^2}{2} \\ &= s W_s^2 - \int_0^s (W_u^2-u) \, du = X_s \end{align*}$$