[prev in list] [next in list] [prev in thread] [next in thread] 

List:       sqlite-users
Subject:    Re: [sqlite] Tsan Bug in WAL mode
From:       Ben Asher <benasher44 () gmail ! com>
Date:       2018-05-23 18:33:27
Message-ID: CAFejQ_SYRdPiNUZe4zsfFUws+66GTbb3Uw+wMDvzh0=_NZAK1w () mail ! gmail ! com
[Download RAW message or body]

Got it. I'll add that one to our blacklist as well then. Thanks again!

Ben

On Wed, May 23, 2018 at 11:29 AM, Dan Kennedy <danielk1977@gmail.com> wrote:

> On 05/24/2018 01:01 AM, Ben Asher wrote:
>
>> I see. I think that makes sense! I've gone ahead and added these 2
>> functions to our local Tsan blacklist then. In researching this further, I
>> found one more Tsan issue here in walTryBeginRead:
>>
>> - One thread is here:
>> http://www.sqlite.org/src/artifact/aa9cffc7a2bad?ln=2558
>> - One thread is here:
>> http://www.sqlite.org/src/artifact/aa9cffc7a2bad?ln=2571
>>
>> Is this one expected as well? If so, I'll also add this function to our
>> blacklist.
>>
>
> It is expected, yes.
>
> The writer at line 2571 is holding an EXCLUSIVE posix lock when it writes
> the aReadMark[] entry. The reader at 2558 is not holding any lock, but
> later on in the same function it takes a SHARED lock and checks that the
> value is as it was earlier. And retries everything if it is not.
>
>   http://www.sqlite.org/src/artifact/aa9cffc7a2bad?2589..2591
>
> Dan.
>
>
>
>
>
>
>
>> Thanks again!
>>
>> Ben
>>
>>
>> On Wed, May 23, 2018 at 1:24 AM, Dan Kennedy <danielk1977@gmail.com>
>> wrote:
>>
>> On 05/23/2018 06:21 AM, Ben Asher wrote:
>>>
>>> Hi there! I believe I've found a SQLite bug that triggers TSAN. I'm
>>>> hoping
>>>> to either confirm this bug or learn what I'm doing wrong. Some
>>>> background:
>>>> in our code, we make sure to synchronize writes to our database such
>>>> that
>>>> only one write can happen at any given time for the same database in our
>>>> application (i.e. there's one synchronized writer connection). However,
>>>> we
>>>> allow any number of reads (readers create their own connections, which
>>>> are
>>>> similarly not shared with other threads) to happen on the same database
>>>> (with a cap on the number of connections to respect resource limits).
>>>> Additionally, the database has PRAGMA journal_mode=WAL; set. At the
>>>> moment
>>>> when TSAN is triggered, there is one reader and one writer each with
>>>> their
>>>> own connection to the same db performing a read and a write:
>>>>
>>>> - Reader is running: sqlite3WalFindFrame, specifically the line with
>>>> this
>>>> code (line 59679 in my sqlite.c):
>>>>
>>>> for(iKey=walHash(pgno); aHash[iKey]; iKey=walNextHash(iKey)){
>>>>
>>>>
>>>> - Writer is running walIndexAppend, specifically the line with this code
>>>> (line 57856 in my sqlite.c):
>>>>
>>>> Thanks for the thread-testing. This one is a known condition. Tsan can't
>>> see it, but we think it is safe. See the comment above the block in
>>> sqlite3WalFindFrame() for details:
>>>
>>>    http://www.sqlite.org/src/artifact/aa9cffc7a2bad?ln=2859..2883
>>>
>>> Basically the race condition is only on the append-only hash-table. But
>>> the hash-table is only used to accelerate lookups on the aPgno[] array.
>>> And
>>> since the hash-table is append-only, it can only produce false-positives
>>> (which are filtered out when aPgno[] is inspected).
>>>
>>> Dan.
>>>
>>>
>>>
>>>
>>>
>>> aHash[iKey] = (ht_slot)idx;
>>>>
>>>>
>>>> I apologize if those line numbers aren't helpful, however I hope the
>>>> function names + code are unique enough to locate the lines in question.
>>>> This is sqlite3 version 3.23.1.
>>>>
>>>> Please let me know if there's any other information I can provide.
>>>> Thanks!
>>>>
>>>> Ben
>>>> _______________________________________________
>>>> sqlite-users mailing list
>>>> sqlite-users@mailinglists.sqlite.org
>>>> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>>>>
>>>>
>>> _______________________________________________
>>> sqlite-users mailing list
>>> sqlite-users@mailinglists.sqlite.org
>>> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>>>
>>>
>>
>>
> _______________________________________________
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
Ben
_______________________________________________
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[prev in list] [next in list] [prev in thread] [next in thread]