shithub: openh264

Info  •  Files  •  Log  •  Branches

Avoid reading iCountMbNumInSlice out of bounds on slice realloc

Prior to 7bcb3ba4f4abf18a,
pCurLayer->sLayerInfo.pSliceInLayer[uiSliceIdx].iCountMbNumInSlice
was read after setting pCurLayer->sLayerInfo.pSliceInLayer to
the newly allocated, larger array. After this commit, it is read
before the array has been switched, and thus is read from the
old array (which only holds elements up to iMaxSliceNumOld, not
up to iMaxSliceNum).

This fixes reads out of bounds, and crashes in the test suite.

--- a/codec/encoder/core/src/encoder_ext.cpp

+++ b/codec/encoder/core/src/encoder_ext.cpp

@@ -4877,7 +4877,7 @@

     pSliceIdx->sSlicingOverRc.iTotalQpSlice         = 0;

     pSliceIdx->sSlicingOverRc.iTotalMbSlice         = 0;

     pSliceIdx->sSlicingOverRc.iTargetBitsSlice      = WELS_DIV_ROUND (kiBitsPerMb *

-                                                      pCurLayer->sLayerInfo.pSliceInLayer[uiSliceIdx].iCountMbNumInSlice,

+                                                      pSlice[uiSliceIdx].iCountMbNumInSlice,

                                                       INT_MULTIPLY);

     pSliceIdx->sSlicingOverRc.iFrameBitsSlice       = 0;

     pSliceIdx->sSlicingOverRc.iGomBitsSlice         = 0;