Let $J \subseteq \mathbb{R}^2$ be a continuous, closed curve (closed in the sense that $j(a) = j(b)$ for some continuous parameterization $j:[a,b]\rightarrow\mathbb{R}^2$ of $J$ with $a,b \in \mathbb{R}$, $a < b$), and let $p \in \mathbb{R}^2$ be a point in the unbounded component of $\mathbb{R}^2\setminus J$. Can a circle through $p$ be drawn whose interior contains $J$?
What if $J$ is the boundary of a convex set, or more generally if $J$ is contained in a half plane whose boundary includes $p$? I guess the question can be reduced to the following plane geometric problem: Given a rectangle $R$ and a point $p$ in its exterior, is it possible to draw a circle through $p$ encompassing $R$? Intuitively the answer is "yes", but I don't know how to prove it.
I will only address the question raised in the second paragraph of the original question.
Let $\ell$ be a line through $p$, and suppose that $J\cap\ell = \emptyset$. Since $J$ is compact (being the continuous image of the compact set $[a,b]$) and $\ell$ is closed, there are $x \in J$ and $y \in \ell$ such that the distance $t$ between $x$ and $y$ is positive and less than or equal to the distance between any $x' \in J$ and $y' \in \ell$.
Denote by $b$ the line parallel to $\ell$ that passes through $x$. Then $\ell$ is contained in one of the two connected components of $\mathbb{R}^2\setminus\ell$; denote the other component by $h$. $h$'s closure $\overline{h}$ has $b$ as boundary, and therefore includes $x$. Then $J \subseteq \overline{h}$. (Explanation: Denote by $h'$ the mirror image of $h$ in $\ell$. Then $J \subseteq \overline{h}\cup\overline{h'}$, since every point of $J$ is removed from $\ell$ by a distance at least $t$. $\mathbb{R}^2\setminus\ell$ is the disjoint union of two open, connected sets, one of whom contains $\overline{h}$, and the other one contains $\overline{h'}$. So $J \subseteq \mathbb{R}^2\setminus\ell$, and therefore $J$ is entirely contained in one of the two components of $\mathbb{R}^2\setminus J$. Therefore either $J \subseteq \overline{h}$ or $J \subseteq \overline{h'}$. Since $x \in J\cap\overline{h}$, it must be that $J \subseteq \overline{h}$.)
$J$ is compact, and therefore, in particular, bounded. Let $d$ be a closed disc (of finite radius) centered at the origin and containing $J$, with radius sufficiently big that $b$ intersects $d$'s boundary at two points.
Denote by $s$ the interior of the square circumscribing $d$ whose sides are parallel, resp. perpendicular, to $b$. Denote $s$'s closure by $\overline{s}$. Then $J$ is contained in the set $\overline{s}\cap\overline{h}$ whose boundary $r$ is a rectangle one of whose sides is a segment of $b$.
If needed, we can "stretch" those sides of $r$ that are parallel to $b$ so that the line through $p$ that is perpendicular to $b$ bisects these sides.
The problem has now been reduced to the one described in this post.
Remarks
The construction above did not use the assumption given in the original question that $J(a) = J(b)$.
The construction above did not use the assumption that $J$ is contained in one of the half planes determined by $\ell$; we simply assumed that $J$ does not intersect $\ell$, and we proved that this implies that $\ell$ must lie entirely in one of the half planes determined by $\ell$.
The circle resulting from the above construction is tangent to $\ell$. To see this, note that $p \in \ell$, and that $\ell \parallel b$. Therefore, the line through $p$ that is perpendicular to $b$ and bisects two of $r$'s sides is also perpendicular to $\ell$. The construction of the circle places the center of the circle on this perpendicular.
We can now use Theorem 8.1.7 (Tangent Line Theorem) from p. 197 of Venema, Gerard A., Foundations of Geometry, 2nd edition, Pearson, 2012: Let $t$ be a line, $\gamma$ a circle centered at $O$, and $P$ a point of $t\cap\gamma$. The line $t$ is tangent to the circle $\gamma$ at the point $P$ iff $op \perp t$.