Thursday, February 27, 2014

Thursday: never ending statistics

There is no fun math here.  This dataset is where fun statistics go to die.  The best algorithms I can come up with to smooth out the noise in a unquantized manner is of order O(N^4).  That sucks.  So instead I'm using median-based histograms, which means all the high frequency information larger than the box size is eliminated.  I guess I should be doing sigma clipping on that box to remove the discrepant points, and then feed that filtered set into a better smoothing function that now doesn't need to deal with the nonsense data.

And then I need to iterate the thing to identify the three different signals I'm trying to calibrate out.

I wanted to say "dogtender," but that's obviously wrong, and "bartendog" seems like trying too hard.

1 comment: