[prev in list] [next in list] [prev in thread] [next in thread] 

List:       openldap-technical
Subject:    Re: LMDB fix sized key and data question
From:       Howard Chu <hyc () symas ! com>
Date:       2024-03-07 14:47:20
Message-ID: 37317393-da8c-45b0-9667-f0c852654ce0 () symas ! com
[Download RAW message or body]

anton@pcloud.com wrote:
> Hello,
> 
> I'm designing an application that will use LMDB. All (or almost all) of my databases will have a fixed \
> size integer key (64bit) and fixed size data. Data is larger than a single integer but generally in \
> 16-32 bytes range (range is for different databases, each database has a fixed data format). 
> I'll not have duplicate key records. I'll mostly insert and retrieve single records. The question is \
> would I benefit and should I use MDB_DUPSORT|MDB_DUPFIXED?  Also the documentation says that \
> MDB_DUPFIXED may only be used with MDB_DUPSORT. However I see code that specifically checks for \
> MDB_DUPFIXED without MDB_DUPSORT and only then sets a flag for DUPFIXED records. However I did not get \
> too deep in the code.

The documentation doesn't lie.

> https://github.com/LMDB/lmdb/blob/mdb.master/libraries/liblmdb/mdb.c#L7743
> 
> My point is that with fixed sized keys and values the btree may chose not to store record sizes, may \
> not use pointers for the binary search but to search values instead and so on. Is that the optimization \
> that is being done for MDB_DUPFIXED or is it something that only concerns duplicate values?

Yes, with DUPFIXED the record sizes don't need to be stored per record, so you'll get some space and \
performance benefits from it.

In other projects, where we don't actually need duplicate keys but we still want to use DUPFIXED, we just
use a dummy key (e.g. 64bit zero) and concatenate the real key & value into the LMDB value.


-- 
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


[prev in list] [next in list] [prev in thread] [next in thread]