[prev in list] [next in list] [prev in thread] [next in thread] 

List:       openldap-technical
Subject:    Re: LMDB fix sized key and data question
From:       Howard Chu <hyc () symas ! com>
Date:       2024-03-07 14:47:20
Message-ID: 37317393-da8c-45b0-9667-f0c852654ce0 () symas ! com
[Download RAW message or body]

anton@pcloud.com wrote:
> Hello,
> 
> I'm designing an application that will use LMDB. All (or almost all) of my \
> databases will have a fixed size integer key (64bit) and fixed size data. Data is \
> larger than a single integer but generally in 16-32 bytes range (range is for \
> different databases, each database has a fixed data format). 
> I'll not have duplicate key records. I'll mostly insert and retrieve single \
> records. The question is would I benefit and should I use MDB_DUPSORT|MDB_DUPFIXED? \
> Also the documentation says that MDB_DUPFIXED may only be used with MDB_DUPSORT. \
> However I see code that specifically checks for MDB_DUPFIXED without MDB_DUPSORT \
> and only then sets a flag for DUPFIXED records. However I did not get too deep in \
> the code.

The documentation doesn't lie.

> https://github.com/LMDB/lmdb/blob/mdb.master/libraries/liblmdb/mdb.c#L7743
> 
> My point is that with fixed sized keys and values the btree may chose not to store \
> record sizes, may not use pointers for the binary search but to search values \
> instead and so on. Is that the optimization that is being done for MDB_DUPFIXED or \
> is it something that only concerns duplicate values?

Yes, with DUPFIXED the record sizes don't need to be stored per record, so you'll get \
some space and performance benefits from it.

In other projects, where we don't actually need duplicate keys but we still want to \
use DUPFIXED, we just use a dummy key (e.g. 64bit zero) and concatenate the real key \
& value into the LMDB value.


-- 
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


[prev in list] [next in list] [prev in thread] [next in thread]