[prev in list] [next in list] [prev in thread] [next in thread] 

List:       sqlite-dev
Subject:    [sqlite-dev] ext/rtree/rtree.c array bounds read violation for op = RTREE_MATCH
From:       timeless <timeless () gmail ! com>
Date:       2011-03-18 15:04:18
Message-ID: AANLkTin-=FvgVhTn-yQjKZWjWVZdZjTGLwxHJG3-zO6F () mail ! gmail ! com
[Download RAW message or body]

I'm looking at http://sqlite.org/sqlite-src-3070500.zip -- I have no
intention of compiling this code, Coverity has already done so (but it
used some amalgamation, I'm trying to report the problem based on the
normal sources).

There was a change at some point such that RTREE_MATCH changed from
0x45 to 0x46, with that change, this code probably changed its
behavior...

/* Possible values for RtreeConstraint.op */
247:#define RTREE_EQ    0x41
248:#define RTREE_LE    0x42
249:#define RTREE_LT    0x43
250:#define RTREE_GE    0x44
251:#define RTREE_GT    0x45
252:#define RTREE_MATCH 0x46

1389:      static const unsigned char compatible[] = { 0, 0, 1, 1, 2, 2 };

compatible has 6 slots (0..5)

1354:static int rtreeBestIndex(sqlite3_vtab *tab, sqlite3_index_info *pIdxInfo){

1390:      u8 op = 0;
1391:      switch( p->op ){
1397:        default:
1399:          op = RTREE_MATCH;
1400:          break;

there's no assert here ensuring that the values all fit in the array compatible:
1413:      assert( compatible[RTREE_EQ & 7]==0 );
1414:      assert( compatible[RTREE_LT & 7]==1 );
1415:      assert( compatible[RTREE_LE & 7]==1 );
1416:      assert( compatible[RTREE_GT & 7]==2 );
1417:      assert( compatible[RTREE_GE & 7]==2 );
(and no assertion about RTREE_MATCH at all)

1419:      opmsk = compatible[op & 7];

RTREE_MATCH & 7 = 6 which exceeds that slot count and thus this reads
past the end of compatible[].

[prev in list] [next in list] [prev in thread] [next in thread]