[prev in list] [next in list] [prev in thread] [next in thread] 

List:       openldap-technical
Subject:    Re: mdb_txn_abort() thread
From:       Howard Chu <hyc () symas ! com>
Date:       2023-06-08 16:04:32
Message-ID: 99d3ecb4-d368-a631-0c2c-65e226481bf0 () symas ! com
[Download RAW message or body]

Sam Dave wrote:
> Very nice, I missed MDB_NOLOCK completely. This might work for me.
> 
> However, one thing got my curious about MDB_NOLOCK docs: "For proper operation [of \
> MDB_NOLOCK] the caller must ... ensure that no readers are using old transactions \
> while a writer is active." 
> Is this also true without MDB_NOLOCK, i.e. does the built-in locking mechanism also \
> "ensure that no readers are using old transactions while a writer is active."? (I \
> guess not because it sounds like it would break "readers don't block writers.")

In the built in mechanism, readers never block writers. When using your own locking, \
readers must block writers otherwise you'll corrupt the views that read txns are \
seeing.

Needless to say, with NOLOCK you must know what you're doing. From the level of this \
conversation so far, I wouldn't recommend you go this route.

Your initial problem seems to be that you don't understand the actual semantics of \
transactions and threads. LMDB expects them to be 1:1. That's it. Anything else is a \
waste, just like M:N threading (multiple user-level threads on top of a different \
number of kernel threads) is a waste and has been abandoned by everyone.
> 
> I was hoping I could do my own concurrency management only for write transactions \
> without worrying about read transaction - I thought that's what "multiversioning" \
> was all about. 
> 
> 
> Jun 7, 2023, 19:17 by hyc@symas.com:
> 
> Sam Dave wrote:
> 
> Read-write txns don't use TLS. TLS is only used for read-only txns.
> 
> 
> Why then is it important that a read-write transaction stays on the same thread \
> (including the final abort or commit)? (I just assumed it was because it needed
> access to TLS, looks like I was wrong)
> 
> 
> Write txns are serialized by a mutex. The mutex is taken when a write txn begins \
> and is released when the txn ends. It is illegal for any thread other than the \
> original locking thread to release the mutex. 
> If you don't want these semantics, just use MDB_NOLOCK and do your own concurrency \
> management. 
> 
> Jun 7, 2023, 17:23 by hyc@symas.com:
> 
> Sam Dave wrote:
> 
> Alternatively, could a more stateful API be provided (as an alternative to API \
> relying on TLS)? 
> Roughly what I have in mind is, mdb_txn_create() returns a state pointer, which \
> should be passed into the next mdb_put(), which returns a state pointer, which
> should be passed into the next mdb_put(), and so on, until the final state is \
> passed into mdb_commit() or mdb_abort(). 
> 
> A txn pointer is a state pointer.
> 
> 
> Jun 7, 2023, 10:26 by samdave@tuta.io:
> 
> Thanks, I was just hoping abort was a (currently undocumented) exception to this :)
> 
> Would it be possible/feasible in the future to bring  MDB_NOTLS support to \
> read-write transactions? (or at least some partial NOTLS support for only e.g. \
> abort) 
> 
> Read-write txns don't use TLS. TLS is only used for read-only txns.
> 
> 
> Motivation: Ability to abort "later" (e.g. at garbage collection in some \
> languages).. as this "later" stuff does not necessarily happen on the original \
> thread. 
> - Samuel
> 
> 
> 
> Jun 6, 2023, 18:44 by hyc@symas.com:
> 
> Sam Dave wrote:
> 
> Hi,
> 
> The documentation for mdb_txn_begin() says "A transaction and its cursors must only \
> be used by a single thread, and a thread may only have a single transaction
> at a time.".
> 
> Does this also hold for mdb_txn_abort() for read-write transactions? I.e. does the \
> abort also have to happen on the same OS thread as all the previous things
> (begin, put, etc.)?
> 
> 
> Calling mdb_txn_abort() requires passing the txn as a parameter, which looks pretty \
> suspiciously to me like "using" it. Is this really a serious question?



-- 
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


[prev in list] [next in list] [prev in thread] [next in thread]