Finding effect of inputs on output (shapely values)

23 Views Asked by At

I've developed a neural network which takes in n inputs returning m outputs.

I want to see which inputs contribute most with each output. One idea I had is for all inputs/output combinations, lock the other inputs to some values, and vary the target input, analysing the target output. I'd then have a list of outputs (the same amount as the sample of inputs). I don't know if this is correct but I was thinking I could use the standard deviation of the outputs to measure how much this input affects the output. But then I remembered that all the other inputs are only locked to a very specific value. So I would then have to redo this whole process with every combination of other inputs. And then (again I don't know if this is right) average all of the standard deviations for an overall measure of how much this target input affects the target output

I..e. Let's say I use 1000 samples for my target input, I would have to do this for 1000^(n-1) input combinations. Essentially, this process has a complexity of s^n where s is the sample count. This seems very high, is this necessary? Also would this whole process, in general, really provide a good way of seeing which inputs affect each output?