I'm a data analyst, no mathematician. Mostly I do 'standard' stuff such as linear mixed-effects regressions or generalized additive mixed models. But now I need to determine the maximum log-likelihood of a very specific model. I wanted to let you know in case my mathematical formulas are awkward or unclear.
The model $f(x, a_{sg}, a_{pl}, p)$ predicts a response latency given an input word with properties $x, a_{sg}, a_{pl}$. My dataset contains $N=2663$ observation. Each word is either a singular or a plural. Singulars and plurals are assumed to be processed via different mechanisms, thus the model is a piecewise-defined function.
Note also that, if $x\in Plurals$, there are two mechanisms again, whereby the one with the lower prediction will determine the prediction.
$p$ is an unknown variable, that is, a parameter that should be optimized. $f(x, a_{sg}, a_{pl}, p)$ = \begin{cases} \frac{1}{500} \sum\limits_{k=1}^{500} y_{LKPk} \sim \mathcal{N}(\frac{1}{1+\log_{n}(a_{sg})},\,0.25)\, & , x\in Singulars \\ min[\frac{1}{500} \sum\limits_{k=1}^{500} y_{LKPk} \sim \mathcal{N}(\frac{1}{1+\log_{n}(a_{pl})},\,0.25), (\frac{1}{500} \sum\limits_{k=1}^{500} y_{DCMPk} \sim \mathcal{N}(\frac{1}{1+\log_{n}(a_{sg})},\,0.25))+(\frac{1}{500} \sum\limits_{k=1}^{500} z_{k} \sim \mathcal{N}(p,\,0.25))]\, & , x\in Plurals \end{cases}
My questions are:
- What is the log-likelihood function of $f(x, a_{sg}, a_{pl}, p)$
- What is the maximum log-likelihood of $f(x, a_{sg}, a_{pl}, p)$?
Lemme me know if you need more information!
Edit 24-08-21: Edited function w.r.t. comment by Abdoul Haki
Edit 25-08-21: The answer of Abdoul Haki made clear to me that I used ambiguous names for the variables. Because of that, here is a reformulation of the model:
$f(x, a_{sg}, a_{pl}, p)$ = \begin{cases} \frac{1}{500} \sum\limits_{k=1}^{500} y_{SGk} \sim \mathcal{N}(\frac{1}{1+\log_{n}(a_{sg})},\,0.25)\, & , x\in Singulars \\ min[\frac{1}{500} \sum\limits_{k=1}^{500} y_{PLk} \sim \mathcal{N}(\frac{1}{1+\log_{n}(a_{pl})},\,0.25), (\frac{1}{500} \sum\limits_{k=1}^{500} y_{SGk} \sim \mathcal{N}(\frac{1}{1+\log_{n}(a_{sg})},\,0.25))+(\frac{1}{500} \sum\limits_{k=1}^{500} z_{k} \sim \mathcal{N}(p,\,0.25))]\, & , x\in Plurals \end{cases}
Question reformulation : (Tel me if I misunderstand the problem somewhere)
If I understand well you problem, you want to predicts a response latency (let's say $y$) given an input word with properties $x,\,a_{sg},\,a_{pl}$ which are supposed to be known. Then you proceed by the following :
1 - You first draw $500$ observations of $3$ variables $y_{SG},\, y_{PL}$ and $z$ which are suppose to respectively follow $$y_{SGk}\sim\mathcal{N}\left(\frac{1}{1+\log_n(a_{sg})},0.25\right),\quad y_{PLk}\sim\mathcal{N}\left(\frac{1}{1+\log_n(a_{pl})},0.25\right),\quad z_k\sim\mathcal{N}(p,0.25)\,.$$
2 - Thus, you predict $y$ by the process : $$\displaystyle\left\{\begin{array}{lcl} y=\frac{1}{500}\sum_{k=1}^{500}y_{SGk} & if & x\in Singulars\\ y=\min\left\{\frac{1}{500}\sum_{k=1}^{500}y_{PLk}, \frac{1}{500}\sum_{k=1}^{500}(y_{SGk} + z_k) \right\} & if & x\in Plurals\\ \end{array}\right.$$
Answer proposed : (I suppose the reformulation is good)
if $x\in Singulars$ we know that as the $500$ sampling of $y_{SG}$ are independent, $$\frac{1}{500}\sum_{k=1}^{500}y_{SGk} \sim \mathcal{N}\left(\frac{1}{1+\log(a_{sg})}, 0.25\right)$$
if $x\in Plurals$ we also know that as the $500$ sampling of $y_{PL}$ are independent, $$\frac{1}{500}\sum_{k=1}^{500}y_{PLk} \sim \mathcal{N}\left(\frac{1}{1+\log(a_{pl})}, 0.25\right)\,\, \text{and}\,\, \frac{1}{500}\sum_{k=1}^{500}z_{k} \sim \mathcal{N}\left(p, 0.25\right)$$ and also $$\frac{1}{500}\sum_{k=1}^{500}y_{SGk} + \frac{1}{500}\sum_{k=1}^{500}z_{k} \sim \mathcal{N}\left(\frac{1}{1+\log(a_{sg})} + p, 0.5\right)$$
So to simplify, we can write $y = \left\{\begin{array}{lcl} V_1 & if & \alpha = 1\\ V_2 & if & \alpha = 0 \end{array}\right.$ where $V_1=X_1\,,$ $V_2=\min\{X_2,X_3\}$, $$X_1\sim \mathcal{N}\left(\frac{1}{1+\log(a_{sg})}, 0.25\right),\,\, X_2\sim \mathcal{N}\left(\frac{1}{1+\log(a_{pl})}, 0.25\right)\,,$$ $$X_3\sim \mathcal{N}\left(\frac{1}{1+\log(a_{sg})}+p, 0.5\right)$$ and $\alpha=1$ when $x\in Singulars$, $\alpha=0$ when $x\in Plurals$.
The distribution of $X_1$, $X_2$ and $X_3$ are knowned. There are the one of a normal distribution (let's call them respectively $\phi_1$, $\phi_2$ and $\phi_3$) : $$\phi_1(x) = \frac{1}{\sqrt{2(0.25)\pi}}\exp\left(-\frac{\left(x-\frac{1}{1+\log(a_{sg})}\right)^2}{2(0.25)}\right)$$ $$\phi_2(x) = \frac{1}{\sqrt{2(0.25)\pi}}\exp\left(-\frac{\left(x-\frac{1}{1+\log(a_{pl})}\right)^2}{2(0.25)}\right)$$ $$\phi_3(x) = \frac{1}{\sqrt{2(0.5)\pi}}\exp\left(-\frac{\left(x-\frac{1}{1+\log(a_{sg})}-p\right)^2}{2(0.5)}\right)$$
Then the distribution of $V_1$ and $V_2$ can be found easily.
The one of $V_1$ (let's call it $f_1$) is $\phi_1$.
The one of $V_2$ can also be found. In fact $$\begin{align*} P(V_2\leq x) &= P(X_2\leq x \text{ and } X_3\leq x)\\ &= P(X_2\leq x)\,P(X_3\leq x)\\ &= \Phi_2(x)\,\Phi_3(x)\,, \end{align*}$$ where $\Phi_2(x)$ and $\Phi_3(x)$ are the cumulative distribution function of $X_2$ and $X_3$. Then the density function of $V_2$ is $$f_2(x) = \phi_2(x)\Phi_3(x) + \Phi_2(x)\phi_3(x)$$
$f_2$ can not be computed by hand because it involve an integral ($\Phi$) that can not be computed. But, it can be computed using any software such as R, Python etc.
Now, if we suppose that, we have $n$ observations (words), the likelihood function of $y$ is : $$\mathcal{L}(y_1,\,y_2,\,\dots,\,y_n\mid p,\,a_{sg},\,a_{pl}) = \prod_{i=1}^n\left[f_1(y_i)\right]^{\alpha_i}\left[f_2(y_i)\right]^{1-\alpha_i}$$ and the log-likelihood is $$\ell=\log \mathcal{L}(y_1,\,y_2,\,\dots,\,y_n\mid p,\,a_{sg},\,a_{pl}) = \sum_{i=1}^n\left[\alpha_i\log\left(f_1(y_i)\right) + (1-\alpha_i)\log\left(f_2(y_i)\right)\right]\,.$$
Proposition : I propose you to compute the log-likelihood using a software and then you maximize it the find the maximum log-likelihood because it can not be computed by hand.