I am given the following differential equation $$\ddot{x}+\lambda\dot{x}=x-x^3.$$
I have difficulty interpreting the question (which is translated to English by me):
Rewrite this differential equation as a system of first order differential equations and calculate the potential energy $U:\mathbb{R}\to\mathbb{R}$ with $U(0)=0$ for which the for $\lambda$ equal to zero first order system has the Hamiltonian shape $\dot{x}=y$ and $\dot{y}=-U'(x)$.
Where does the $\lambda=0$ come into play? This is what I thought: Let $y=\dot{x}$, then $\dot{y}=-\lambda\dot{x}+x-x^3$. We then define $U'(x)$ as $-x+x^3$. (So the $\lambda\dot{x}$ is not in the $u'(x)$ function.) Then we have the first order system: \begin{align*} \dot{x}&=y\\ \dot{y}&=-\lambda\dot{x}-U'(x) \end{align*} which is equal to the desired system when $\lambda=0$.
But then I am asked to determine how the Hamiltonian function $H(x,y)=\frac{1}{2}y^2+U(x)$ changes under the time development given by the very first differential equation in this exercise, and they mention that I have to do this for random $\lambda\in\mathbb{R}$. In other words, calculate $\dot{H}$. But as I interpreted the question, the function $H(x,y)=\frac{1}{2}y^2+U(x)=\frac{1}{2}\dot{x}^2-\frac{1}{2}x^2 + \frac{1}{4}x^4$ does not depend on $\lambda$. And therefore neither does $\dot{H}$. Or does it?
So I think I've misinterpreted the question, but if the $\lambda\dot{x}$ term were to be inside of $U'(x)$, then I don't understand the extra for $\lambda$ equal to zero portion of the block quote, since no matter what $\lambda$ would be, the system would then ultimately look like the desired shape anyway, if the equation for $\dot{y}$ were to look like $\dot{y}=-U'(x)$ with $U'(x)=\lambda\dot{x}-x+x^3$.
Then for background: I am then asked to conclude that the second order differential equation at the top of this question has no periodic trajectories for $\lambda\neq 0$.
You should get $$ \dot H=y\dot y+U'(x)\dot x=y(\dot y+U'(x))=-λy^2. $$ For $λ=0$ this means that the value of $H$ is constant on solution curves, or in turn, that solution curves lie inside level curves of $H$.
This also shows that there are no periodic solutions for $λ\ne0$, as integrating over a period would give zero on the left side and an $λ$ times an integral over a non-negative integrand on the right side.
You can not get $U'(x)=λ\dot x−x+x^3$ as $U$ is a function of the position only, it does not contain the velocity.