I have already posted this question on Stats stackexchange so I am unsure whether I am allowed to post it here to.
I am asking what would be a good/suitable investigation/test to use when trying to compare 2 models.
So for my model in question, I am looking at the Lac Operon model and I add in substace $Y$ into the model and it produces enzyme $X$. The objective of adding $Y$ into the system is the produce as much $X$ as possible. I want to investigate if it is better to add all my $Y$ into the system in one big go (let's say I add 10,000 in all together), or to add say 2,000 $Y$ in at 5 even time intervals, or to add 1,000 of $Y$ in in 10 even time intervals.
I am wondering what sort of statistical analysis should i be using here, or what sort of tests should I be performing. I know I could just look at the amount of $X$ produced for each variation of my model, but I am looking for more intricate tests to perform.
Now my models aren't linear models so I am unable to do anova using R, they are actually stochastic models. They are a series of chemical equations but I am basically just interested in the value of $X$, so i doubt the specifics of the model are needed to be known.
Edit: May I propose another question to you. So to follow on from my earlier point, basically I have a system of chemical equations, and the rate constants from these equations follow a gamma distribution. Should I be testing rate constants against each other, or should I be testing the data against each other?
Edit: Maybe I could test the comparison of the rate constants for my 2 models by using Bayesian Inference? I was wondering what would be a suitable test to compare to posterior/prior distributions rather than just using Bayes Factors??
Suppose you have Methods A and B of producing $X.$ If you have $n_1$ independent runs with Method A, and independently of that you have $n_2$ runs with method B. Also suppose that the amount of $X$ produced by each method is approximately normal. Say $X_{ai} \sim \mathsf{Norm}(\mu_a, \sigma_a)$ and $X_{bi} \sim \mathsf{Norm}(\mu_b, \sigma_b).$
Then you want to test $H_0: \mu_a = \mu_b$ against $H_1: \mu_a \ne \mu_b.$ The appropriate test would be the Welch (separate-variances) two-sample t test.
Supposing that data are far from normal, but that the distributions for A and B are of the same shape, then the Mann-Whitney-Wilcoxon rank-sum test could be used to see if there is a shift in the location of the median $\eta_a$ of Method A to $\eta_b \ne \eta_a$ for Method B. If the assumptions for this test are not met (or if there are many tied values among the two samples of $X$'s), then you could see if a permutation test is useful.
Not knowing how many observations you would have for each method, and not knowing the shapes of the population distributions I could not give a useful opinion which kind of test to use.
Note: If 'batches' to which methods are applied differ greatly, then it might be a good idea to split each batch in two, and to use Method A on one half and Method B on the other. Then you might have $n$ pairs of of outputs $X$. For this experimental design, a paired t test, a Wilcoxon signed-rank test, or a paired permutation test might be more likely to find any difference between methods.