[prev in list] [next in list] [prev in thread] [next in thread] 

List:       monetdb-checkins
Subject:    MonetDB: default - When we have allocated more than GDK_mem_maxs...
From:       Sjoerd Mullender <commits+sjoerd=acm.org () monetdb ! org>
Date:       2016-09-30 13:20:57
Message-ID: hg.8b6c63e57ac2.1475241657.6315528441665844383 () monetdb2 ! cwi-incubator ! nl
[Download RAW message or body]

Changeset: 8b6c63e57ac2 for MonetDB
URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=8b6c63e57ac2
Modified Files:
	gdk/gdk_heap.c
Branch: default
Log Message:

When we have allocated more than GDK_mem_maxsize bytes, start mmapping.
When we get above the limit of how much memory the system has, we
start memory mapping all heaps over 4 times the page size (i.e. a
quarter MB), no matter what the two mmap_minsize parameters say.

Also, always memory map heaps that we are loading when they are at
least 4 times the page size in size, no matter how much memory is in
use, or what the mmap_minsize parameters say.  The rationale here is
that malloc+read is tradded off against mmap.


diffs (34 lines):

diff --git a/gdk/gdk_heap.c b/gdk/gdk_heap.c
--- a/gdk/gdk_heap.c
+++ b/gdk/gdk_heap.c
@@ -99,7 +99,10 @@ HEAPalloc(Heap *h, size_t nitems, size_t
 		GDKerror("HEAPalloc: allocating more than heap can accomodate\n");
 		return GDK_FAIL;
 	}
-	if (h->filename == NULL || h->size < (h->farmid == 0 ? GDK_mmap_minsize_persistent \
: GDK_mmap_minsize_transient)) { +	if (h->filename == NULL ||
+	    h->size < 4 * GDK_mmap_pagesize ||
+	    (GDKmem_cursize() + h->size < GDK_mem_maxsize &&
+	     h->size < (h->farmid == 0 ? GDK_mmap_minsize_persistent : \
GDK_mmap_minsize_transient))) {  h->storage = STORE_MEM;
 		h->base = (char *) GDKmallocmax(h->size, &h->size, 0);
 		HEAPDEBUG fprintf(stderr, "#HEAPalloc " SZFMT " " PTRFMT "\n", h->size, PTRFMTCAST \
h->base); @@ -204,8 +207,7 @@ HEAPextend(Heap *h, size_t size, int may
 		/* extend a malloced heap, possibly switching over to
 		 * file-mapped storage */
 		Heap bak = *h;
-		size_t cur = GDKmem_cursize(), tot = GDK_mem_maxsize;
-		int exceeds_swap = size > (tot + tot - MIN(tot + tot, cur));
+		int exceeds_swap = size >= 4 * GDK_mmap_pagesize && size + GDKmem_cursize() >= \
GDK_mem_maxsize;  int must_mmap = h->filename != NULL && (exceeds_swap || \
h->newstorage != STORE_MEM || size >= (h->farmid == 0 ? GDK_mmap_minsize_persistent : \
GDK_mmap_minsize_transient));  
 		h->size = size;
@@ -621,7 +623,7 @@ HEAPload_intern(Heap *h, const char *nme
 	char *srcpath, *dstpath;
 	int t0;
 
-	h->storage = h->newstorage = h->size < (h->farmid == 0 ? \
GDK_mmap_minsize_persistent : GDK_mmap_minsize_transient) ? STORE_MEM : STORE_MMAP; \
+	h->storage = h->newstorage = h->size < 4 * GDK_mmap_pagesize ? STORE_MEM : \
STORE_MMAP;  if (h->filename == NULL)
 		h->filename = (char *) GDKmalloc(strlen(nme) + strlen(ext) + 2);
 	if (h->filename == NULL)
_______________________________________________
checkin-list mailing list
checkin-list@monetdb.org
https://www.monetdb.org/mailman/listinfo/checkin-list


[prev in list] [next in list] [prev in thread] [next in thread]