Friday, July 26, 2013

Friday: Fucking integers, am I right?

Short version: libc "abs" function returns the integer absolute value of the integer argument.  This is fundamentally useless, as if you're taking an absolute value, you're probably working with floating point numbers, as those are the ones you use for "actual fucking math."

Long version: approximation A isn't very good, and I've been truncating it at y = 1.0 because it's garbage beyond that.  This week has led me to the conclusion that the garbage point is actually closer to y = 0.7 (potentially y = sqrt(0.5), but that's not important here).  Therefore, to deal with values of y > 1.0, I needed to find a different approximation.  I've been playing with approximation B for the past week or so, but it kept having a strange quantization error at high values of x.  I found approximation C, which uses ~30 order polynomials in x and y, and so should be pretty decent.  Nope.  Exact same quantization issue (plus there must be a typo in region 2, as that just falls apart).  So, it seemed like the only solution was to find midpoints of each quantized region, interpolate along them, and define the function that way.  That led to hunting down the midpoints.  It turns out that each midpoint was located at y = sqrt(0.5) + 0.5 * sqrt(0.5).  Huh.  That's basically the definition of "mathematically suspicious."  Does approximation B use sqrt(0.5) anywhere?  No.  So where does it come from?

Way up top, I define y = abs(z * s / sqrt(2)).  And because that's abs and not fabs, it quantizes the values of y in steps of 1/sqrt(2) = sqrt(0.5).  Fucking fuck fuck.

This seems like an appropriate place to put this largely unrelated note from my notebook:

It's still partially broken, as I only have a decently working implementation in the test code that I've been using to compare approximations A,B, and C (plus D,E,F,manual calculation, and mostly lies).


I wonder if they drew it going over Hawaii just to be dramatic.

No comments:

Post a Comment