For some cases of spherical functions, like BRDF or a dot product between a sample direction and a surface normal, projecting them to spherical harmonics coefficients gives values over 1 or below -1, e.g. when the dot product is always 1 for any point on the sphere (any direction).
Thus my question - are such values (larger than 1 and/or less than -1) valid values for spherical coefficients? Should the spherical function be prepared somehow before projection, by doing some kind of normalization? Am I simply forgetting something when implementing "the thing" in my program?
Yes it's normal, SH are not conservative of the original signal, it is a compression of sorts. They have artefacts in the form of ringing which can give values you never saw in the original signal. For example the infamous negative little 2 lobes in the back of what should be a cosine lobe.
There are ways to mitigate the issue with somekind of prefiltering like when applying fourier. Or rescale down after projection etc. If you get values that are way way beyond anything present in your original signal, like 3 where ought to get something around 1, you might miss a Pi factor I don't know. Refer to "stupid SH tricks" paper by PP Sloan I guess.