I want to optimize the following function $U = ...$ using KKT. I am still learning KKT. However, I cannot understand how I can find if the following function is a convex function?
$$U = \frac{(A-1)B\ln(B) - A\ln(A)B + A\ln(A)}{(A-1)(B-1)(B-A)}.$$
$$A = \Big(\frac{r}{p}\Big)^{\alpha}\Big(\frac{P_a}{P_x}\Big)$$and
$$B = \Big(\frac{r}{q}\Big)^{\alpha}\Big(\frac{P_{b}}{P_x}\Big)\varepsilon$$.
where $\varepsilon \in[0,1], \frac{r}{p},\frac{r}{q} <1$ and $r,p,q \in \mathbb{R}$ and are determined by an algirthm I have developed. $\alpha = [2,4]$, lets assume $\varepsilon = 0.1,\alpha = 2$ for the current scenario (If this info in important).
$P_a,P_b,P_x \leq 1$.
Note that $P_a$ and $P_b$ are constants and I have to control $P_x$ to optimize.
Not a complete answer, but to help you investigate yourself further:
Let's simplify the notation. Define
$$ c_A\equiv\bigg(\frac{r}{p}\bigg)^\alpha P_a $$
$$ c_B\equiv\bigg(\frac{r}{q}\bigg)^\alpha P_b\varepsilon$$
Then your specified function $U$ is
$$ U(x)=\frac{\big(\frac{c_A}{x}-1\big)\frac{c_B}{x}\log\big(\frac{c_B}{x}\big)-\big(\frac{c_B}{x}-1\big)\frac{c_A}{x}\log\big(\frac{c_A}{x}\big)}{\big(\frac{c_A}{x}-1\big)\big(\frac{c_B}{x}-1\big)\big(\frac{c_B-c_A}{x}\big)} $$
which can be simplified to
$$ U(x)=\frac{x}{c_B-c_A}\bigg(\frac{c_B\log(c_B/x)}{c_B-x}-\frac{c_A\log(c_A/x)}{c_A-x}\bigg) $$
where I have replaced your notation $P_x$ with the letter $x$, to make things a little clearer. The function may be easier to investigate in this form.