How do I define a cost function to trade latency and throughput?

108 Views Asked by At

I have a system that is a pipeline of three functions. Function P is a message producer. Function PA is a pre aggregate of data. Function C is a consumer of data. The pipeline is P -> PA -> C. I have some known parameters:

  • the capacity of function C to process messages given in message/sec is fixed.
  • the capacity of the channel PA -> C to handle messages given in message/sec (bandwidth) is fixed.
  • the throughput of messages coming to the function C given in message/sec I can monitor.

the unknown variable is:

  • the throughput that the function P generates messages.

My goal is to have a maximum throughput of messages going to function C without exceeding the capacities that I said before. This is my objective function. Can I say that?

I do that by adjusting the latency on the PA function. I cannot always set the latency to 0 because it can exceed the capacity of the channel PA -> c or the function C. And besides that, the workload of messages on the function P varies with respect to time.

I have the insights in steps to adjust the latency:

  1. adjust the latency on the PA function only with the capacities of the function C of the channel PA->C is almost ~100%.
  2. measure the latency of the message and if the current latency is bigger than a previous measurement (when the capacities were not exceeded) we know that messages are queueing.
  3. calculate how many times the latency increased on the function C and increase the latency on the function PA to the same degree (e.g.: if increased 3 times on the C I increase the latency 3 times on the PA). Thus, the throughput decreased and it is affordable by the capacity of the channel PA->C and to the function C.

My question is, how do I define a cost function to such a system? I believe that I described all the dynamics of my system, didn't I? I was studying Little`s Law and Single Server model in queueing theory. It gave a lot of insights into my step-by-step method. But How should I formulate a cost function for that?