If $ABC$ is an equilateral triangle. $P$ is a variable point inside the triangle or on its sides. We need to find the maximum product of $AP \cdot BP \cdot CP$.
This question is in a Complex Analysis homework. I don't know how can I use the complex analysis (maybe, maximum modulus theorem ) to find the maximum product.
Updated work
Let $z_1$, $z_2$, $z_3$ and $z$ correspond to the vertices $A$, $B$, $C$, and variable point $p$, respectively, then the product function is given by $f(z)=|z-z_1||z-z_2||z-z_3|$, clearly, $(z-z_1)(z-z_2)(z-z_3)$ is nonconstant holomorphic function in $\mathbb{C}$ and so in $\overline{T}$, where $T$ represents the region inside the equilateral triangle. Then by the Maximum modulus Theorem, $f$ attains the maximum on the boundary $\partial T$. To this point, is there a way to find $z$ explicitly that maximize $f$?
Up to a similitude $s(z)=az+b$, we can reduce the issue to equilateral triangle $T$ with vertices:
$$z_1=1, \ \ z_2=\omega, \ \ z_3=\omega^2=\overline{\omega} \ \ \text{with} \ \ \omega=e^{2i \pi/3} $$
Therefore: AP.BP.CP is the absolute value of :
$$(z-1)(z-\omega)(z-\omega^2)=z^3-1$$
The maximum value of this holomorphic function, call it $f(z)$, is reached for a $z$ belonging to the boundary (Maximum Modulus Theorem, as you have noticed). Let us parametrize the line segment from $1$ to $\omega$ in the following way:
$$z=(1-t)+t \omega, \ \ \ \ 0\le t \le 1\tag{1}$$
The image by $f$ of this line segment is the red "drop-like" cubic curve (see equation in the remark below).
The two other sides
$$z'=(1-t)\omega+t \omega^2, \ \ z''=(1-t)\omega^2+t$$
can be written under the form:
$$z'=\omega z, \ \ z''=\omega^2 z \tag{1}$$
Therefore, because $\omega^3=1$ and $(\omega^2)^3=1$, the image by $f$ of these two other sides obtained by taking to the third power expressions (1) gives the same curve!
This is the image $f(\partial T)$. The image by function $f$ of $T$ itself is the inside of the drop. The sister curve described as the image of $\partial T$ by $g(z):=z^3$ is magenta curve.
Considering the red curve, $f(z)=z^3-1$ reaches its maximal value for $z_m^3-1=I'$, which corresponds to:
$$z_m^3=-\tfrac18 \ \ \text{(point I')} \tag{2}$$
Cubic equation (2) has three solutions. As the argument of $I$ is $\pi+k 2 \pi$, the arguments of solutions $z_m$ must be $\pi/3 + k 2 \pi/3$. As the maximum must occur on the boundary, taking $k=0,1,2$ gives points $J_0, J_1, J_2$ resp., i.e., the midpoints of the sides of triangle $T$.
Fig. 1: The three maximizing solutions $J_0, J_1, J_2$.
Remark: The parametric equations of the "drop" obtained by taking the real and imaginary parts of $((1-t)+t \omega)^3$ are
$$(x=1-\tfrac92 t(1-t), \ \ \ y=\tfrac{3\sqrt{3}}{2}t(1-t)(1-2t))$$