shithub: libvpx

Info  •  Files  •  Log  •  Branches

fix a small bug in 16 point forward dct

The commit fixes a minor error in 16 point fdct where in a rotation can
produce result of -1 instead of 0.

Change-Id: I45aac4a52bcd06225c6d04e643547a13e1c1aade

--- a/vp9/encoder/vp9_dct.c

+++ b/vp9/encoder/vp9_dct.c

@@ -1288,7 +1288,7 @@

     output[15] = (intermediate[11] + intermediate[12] + output_rounding)

         >> output_shift;

-    output[ 1] = -(intermediate[11] - intermediate[12] + output_rounding)

+    output[ 1] = (intermediate[12] - intermediate[11] + output_rounding)

         >> output_shift;

     output[ 7] = (2 * (intermediate[13] * C8) + final_rounding) >> final_shift;

-- 

⑨