[prev in list] [next in list] [prev in thread] [next in thread] 

List:       helix-server-cvs
Subject:    [Server-cvs] engine/core/pub memsys.h,1.1.2.2,1.1.2.3
From:       dcollins () helixcommunity ! org
Date:       2013-08-30 22:58:40
[Download RAW message or body]

Update of /cvsroot/server/engine/core/pub
In directory cvs01.internal.helixcommunity.org:/tmp/cvs-serv25210/pub

Modified Files:
      Tag: SERVER_NUCLEUS
	memsys.h 
Log Message:
Synopsis
========
Fixes RPDS memory leaks, adds out-of-memory defensive restart

Branches:  Jupiter / SERVER_NUCLEUS
Reviewer:  Jamie


Description
===========
This fixes a few RPDS memory leaks I detected while performing cloud
downloads:

* metricspln/EventMetrics was leaking all event objects
* CMSPlugin::HandleFile was leaking mime-type buffers
* LocalFileObjMaker::CreateFileObjectWithPluginhandler() was leaking a 
VerbatimFileName buffer


While investigating, I realized that I hadn't yet implemented the
out-of-memory check for the new memory subsystem in Nucleus/Mercury.

The older Helix-server based SharedMemory subsystem provides this
mechanism, though for RPDS it is only used for debugging.  Because of
the extensive debugging facilities of debug SharedMemory, it is able
to detect out-of-memory immediately and provide the stack of the final
allocation that took us over the limit, which sometimes gives clues
about the leak.

Being light-weight by design, system-memory based SharedMemory cannot
reasonably detect out-of-memory at the exact allocation which exceeds 
the limit without compromising the intended design.  Instead, we now
perform an out-of-memory check once per second.  If memory allocations
exceed the maximum, a message is printed to the debug log and the
service restarts as a defensive measure.

The maximum memory allowed defaults to 512 MB.  It can be controlled
with the -m <memory_in_MB> flag, the same as for debug memory.

Note that memory allocated outside the server's memory subsystem will 
not be counted.  This is by design.

The g_ullSizemmap variable should probably be renamed, but I wanted to
keep the diff as small as possible.



Files Affected
==============
server_rn/nucleus/analytics/metricspln/eventmetrics.cpp
server_rn/nucleus/cms/cmsplin/cmsplin.cpp
server_rn/nucleus/common/util/localfileobjmaker.cpp
server/engine/core/_main.cpp
server/engine/core/memsys.cpp
server/engine/core/pub/memsys.h


Testing Performed
=================
Unit Tests:
- None

Integration Tests:
- Performed cloud downloads using RealPlayer.
- Tested various values of the -m limit, and without it.
- Verified restarting when out-of-memory worked as intended.
- Tested with debug memory and the default system-based memory.
- Tested with other memory-related and debugging settings.

Leak Tests:
- Performed cloud downloads using RealPlayer while leak-checking was 
enabled.
- Tested out-of-memory with the leak-simulation facility enabled
   with --fml (--force-memory-leak).

Performance Tests:
- None

Platforms Tested:  Windows 7 / x86 (Debug)
Builds Verified:  VC10 / x86 / Debug


QA Hints
========
N/A



Index: memsys.h
===================================================================
RCS file: /cvsroot/server/engine/core/pub/memsys.h,v
retrieving revision 1.1.2.2
retrieving revision 1.1.2.3
diff -u -d -r1.1.2.2 -r1.1.2.3
--- memsys.h	11 Jul 2013 22:44:05 -0000	1.1.2.2
+++ memsys.h	30 Aug 2013 22:58:23 -0000	1.1.2.3
@@ -20,6 +20,8 @@
 #endif
 #endif // _PAGESIZE
 
+#define DEFAULT_SYSMEM_MAX 512 // default SharedMemory max usage, in MB
+
 class SharedMemory {
 public:
 			    	    ~SharedMemory();
@@ -44,6 +46,7 @@
     static HX_SIZE_T                UpdateBytesInUse();
     static HX_SIZE_T                BytesAllocated();
     static HX_SIZE_T                BytesInPool();
+    static void                     CheckMemoryUsage();
 
     static void                     SetSingleMaxAllocation(INT32 nVal);
     static void                     EnableDebugMemory();


_______________________________________________
Server-cvs mailing list
Server-cvs@helixcommunity.org
http://lists.helixcommunity.org/mailman/listinfo/server-cvs
[prev in list] [next in list] [prev in thread] [next in thread]