I have an operator $U=x(t) + \int_0^1 x(st) \, ds$. Goal is to find $U^2$ without iterated integrals.
I start with: $$U(Ux(t))=x(t) + \int_0^1 x(st)\,ds+\int_0^1 (x(qt) + \int_0^1 x(sqt)\,ds )\, dq= \\ =x(t) + 2\int_0^1 x(st) \, ds + \int_0^1 \int_0^1 x(sqt) \, ds\,dq.$$ I have a problem in transformation $\int_0^1 \int_0^1 x(sqt) \, ds\,dq$ . I have used integration by parts for the internal integral, but i do not know what to do next.
\begin{align} y(t) & = x(t) + \int_0^1 x(st) \, ds = x(t) + \int_0^t x(u)\,\left( \frac{du} t \right) = x(t) + \frac 1 t \int_0^t x(u)\,du \\[10pt] & = (\text{the present value of } x) + (\text{the average value of $x$ from time 0 until now}). \end{align} Now do it again: \begin{align} z(t) & = y(t) + \int_0^1 y(st) \, ds = y(t) + \int_0^t y(u)\,\left( \frac{du} t \right) = y(t) + \frac 1 t \int_0^t y(u)\,du \\[10pt] & = (\text{the present value of } y) + (\text{the average value of $y$ from time 0 until now}) \\[10pt] & = \left( \begin{array}{l} \text{the present value of $x$} \\ {} + \text{the average value} \\ \phantom{{}+{}}\text{ of $x$ until time $t$} \end{array} \right) + \frac 1 t \int_0^t \Big( x(u) + \left( \begin{array}{l} \text{the average} \\ \text{value of $x$} \\ \text{until now} \end{array} \right) \Big) \, du \\[10pt] & = \left( \begin{array}{l} \text{the present value of $x$} \\ {} + \text{the average value} \\ \phantom{{}+{}}\text{ of $x$ until time $t$} \end{array} \right) + \frac 1 t \int_0^t x(u) \, du + \frac 1 t \int_0^t \left( \begin{array}{l} \text{the average} \\ \text{value of $x$} \\ \text{until time $u$} \end{array} \right) \, du \\[10pt] & = \left( \begin{array}{l} \text{the present value of $x$} \\ {} + \text{the average value} \\ \phantom{{}+{}}\text{ of $x$ until time $t$} \end{array} \right) + \left( \begin{array}{l} \text{the average value} \\ \text{of $x$ until time $t$} \end{array} \right) + \frac 1 t \int_0^t \left( \begin{array}{l} \text{the average} \\ \text{value of $x$} \\ \text{until time $u$} \end{array} \right) \, du \\[10pt] & = \left( \begin{array}{l} \text{the present value of $x$} \\ {} + (2\times\text{the average value} \\ \phantom{{}+{}(}\text{ of $x$ until time $t$}) \end{array} \right) + \frac 1 t \int_0^t \left( \begin{array}{l} \text{the average} \\ \text{value of $x$} \\ \text{until time $u$} \end{array} \right) \, du. \end{align} The last expression is of course an iterated integral.
Does the question mean
I will assume the former since I don't know how to do the latter.
$$ \frac 1 t \int_0^t \left( \begin{array}{l} \text{the average} \\ \text{value of $x$} \\ \text{until time $u$} \end{array} \right) \, du = \frac 1 t \int_0^t \left( \frac 1 u \int_0^u x(s) \, ds \right) \, du = \frac 1 t \int_0^t \left( \int_0^u \frac{x(s)} u \, ds \right) \, du. $$ Now we are integrating over the region $\left\{ (u,s) : 0 \le s \le u \le t \right\}.$
We did this by letting $u$ run from $0$ to $t$, and then for each fixed value of $u$, letting $s$ run from $0$ to $u.$
But we could do it by first letting $s$ run from $0$ to $t$, and then for each fixed $s$, letting $u$ run from $s$ to $t.$
Then we would have this: $$ \frac 1 t \int_0^t \left( \int_s^t \frac{x(s)} u \, du \right) \, ds. $$ As $u$ goes from $s$ to $t$, the factor $x(s)$ does not change; therefore it can be pulled out: $$ \frac 1 t \int_0^t \left( x(s) \int_s^t \frac 1 u \, du \right) \, ds. $$ Now we can evaluate the inner integral: \begin{align} & \frac 1 t \int_0^t \left( x(s) \int_s^t \frac 1 u \, du \right) \, ds = \frac 1 t \int_0^t x(s) (\log t - \log s) \, ds \\[10pt] = {} & \frac 1 t \int_0^t x(s)\log t \, ds - \frac 1 t \int_0^t x(s) \log s\, ds. \end{align} In the first integral, as $s$ goes from $0$ to $t$, the factor $\log t$ does not change, so it can be pulled out and we get $$ \frac {\log t} t \int_0^t x(s) \, ds - \frac 1 t \int_0^t x(s) \log s\, ds. $$ That is an expression for $(U^2 x)(t)$ that does not use iterated integrals. (But we used iterated integrals to find it.)
So $$ (U^2 x) (t) = (\log t)\cdot(\text{average value of $x$ on $[0,t]$}) - (\text{average value of $(x\cdot\log)$ on $[0,t]$}). $$