Consider a random figure. Let's say a regular hexagon ($6$- sided polygon) of area $6$. Chip off the $6$ corners to get a dodecagon ($12$ sided polygon). And repreat this chipping process forever. Basically the number of sides will get twice the previous one. (Just for enumeration, let's clarify, the $n^\text{th}$ step yields a $3\cdot 2^n$ sided polygon). If this continues forever, we get a circle I guess. But how do we calculate its (circle's) area?
This gives rise to another question actually. What if it wasn't a hexagon but an eight sided polygon (aka octagon)? It would've resulted into a circle as well, right? So what would be the area?
Do we get an unique circle for polygons of different area? Or maybe the area is same at the beginning (for two different polygons) but changes when they result into a circle?
Short answer
For regular polygons, the area of the resulting circle is $$\boxed{A = \frac{P^2}{4} \frac{\pi}{n^2}\cot^2{\left(\frac{\pi}{n}\right)} }$$
where $P$ and $n$ are the perimeter and number of sides of the original regular polygon.
Furthermore, this chipping away process multiplies the original area and the perimeter of the figure by a factor of $(\pi/n) \cot{(\pi/n)}$. (This is a factor that starts at around 0.60 for triangles, and gets closer and closer to 1 for figures with more sides.) Because this factor depends on $n$, different-sided polygons with the same initial area will end up as circles of different area.
Chipping away reveals a circle
For simplicity, let's consider what happens with regular polygons. You begin with a regular polygon, and chip away at its sides, producing another regular polygon with twice as many sides. You repeat the process many times; in the limiting case you obtain a circle.
The dimensions of the circle are special: it is the largest circle that fits inside of the original polygon. It is the incircle of the original polygon.
Computing the area of the incircle
We still have to prove that the result of the chipping process is the incircle of the original polygon. But suppose we take this result as an assumption— what will be the area of the incircle?
We can find the area of the circle by finding its radius. We divide the regular $n$-sided polygon into $n$ triangles, one per side. The orange angle is, in general, $2\pi/n$. By trigonometry, we know that if the side length is $s$, the height of the triangle is $\frac{s}{2}\cot{(\pi/n)}$. This height is also equal the radius of the circle. If we like, we can express the radius in terms of the perimeter $P=n\cdot s$:
$$r = \frac{s}{2}\cot{(\pi/n)} = \frac{P}{2n}\cot{(\pi/n)}$$
The area of the circle (and the answer to this question) is therefore
$$\boxed{A_\infty = \pi r^2 = \frac{P^2}{4} \frac{\pi}{n^2}\cot^2{\left(\frac{\pi}{n}\right)} }$$
where $P$ and $n$ are the perimeter and number of sides of the original regular polygon. ■
Interestingly, the original area of the regular polygon was:
$$A_0 \equiv \frac{P^2}{4} \cot(\pi/n) \times \frac{1}{n}$$
Hence the cutting process multiplied its area by a factor of $$\frac{\pi}{n}\cot{\left(\frac{\pi}{n}\right)}.$$
and in fact the perimeter of the figure was multiplied by exactly the same factor (!). See below for details.
Showing that the incircle is the limit
Let's formalize the chipping process by saying that you divide each edge into three partitions with two symmetrically-spaced marks. Then those marked divisions become the vertices of the new polygon. Because each vertex participates in two edges, this transforms a polygon with $n$ vertices into a polygon with $2n$ vertices.
If we choose the spacing of the marks wisely, we can ensure that this chipping process preserves regularity. This means that the process will transform an $n$-sided regular polygon with sides of length $s$ into a $2n$-sided regular polygon with sides of length $p\times s$—some factor $p$ shorter.
In the next step, we'll compute the factor $p$ which ensures regularity. Here, let's note that if the side length $s$ becomes $p\times s$, then the perimeter changes from $ns$ to $(2n)(ps) = (2p)(ns)$. In other words, the perimeter becomes multiplied by $2p$.
Using some triangle geometry, we find that this factor of $p$ actually depends on $n$, the number of sides: $$p(n) = \frac{\tan{(\pi/2n)}}{\tan{(\pi/n)}}$$
(See below for details.)
Hence suppose we have a certain regular polygon. Initially, it has $n$ sides, and a perimeter of $P$. After cleaving it several times, our formula in step (3) tells us that it will have perimeter: $$P_k \equiv P\times 2p(n) \times 2p(2n) \times 2p(4n) \times 2p(8n) \times \ldots \times 2p(2^k n)$$
Our formula from step (4) allows us to write this in a form which simplifie: the perimeter after $k$ chipping iterations becomes
$$P \times \frac{2 \tan{(\pi/2n)}}{\tan{(\pi/n)}}\times \frac{2 \tan{(\pi/4n)}}{\tan{(\pi/2n)}}\times \frac{2 \tan{(\pi/8n)}}{\tan{(\pi/4n)}}\times\ldots \times \frac{2 \tan{(\pi/2^kn)}}{\tan{(\pi/2^{k-1}n)}}$$ which cancels significantly: $$P_k \equiv P \times \frac{2^k \tan{(\pi/2^k n)}}{\tan{(\pi/n)}}$$
Miraculously, the limit of this process as $k\rightarrow \infty$ is simply: $$P_{\infty} = P\cdot \frac{\pi/n}{\tan{(\pi/n)}} = P\,\frac{\pi}{n}\cot{(\pi/n)}$$
(Indeed, $\lim_{b\rightarrow \infty} b\tan{(a/b)} = a$ by l'Hospital's rule: $\lim_{b\rightarrow\infty}b\tan{(a/b)}=\lim_{t\rightarrow 0}\tan{(at)}/t = \lim_{t\rightarrow 0} a\sec^2(at)/1 = a\cdot 1/ 1 = a$).
A circle with such a circumference would have a radius of $(P/2n)\cot{(\pi/n)}$ and a corresponding area of $$A = \pi\;\frac{P^2}{4n^2}\cot^2(\pi/n)$$
This is exactly the dimensions of the incircle of the original polygon, and we are done.
Proof of the formula for p(n)
In the previous section, we made use of a formula for a value of $p$ which ensure that the polygon remained regular when cleaved. In this section, we derive this formula.
$$p(n) = \frac{\tan{(\pi/2n)}}{\tan{(\pi/n)}}$$
Let's take the edge of a regular $n$-sided polygon and form a triangle with the center of the polygon as the third vertex. The angle at that vertex is $2\pi/n$ because the polygon is regular.