[prev in list] [next in list] [prev in thread] [next in thread] 

List:       pypy-svn
Subject:    [pypy-commit] pypy py3.5: Issue #2579
From:       arigo <pypy.commits () gmail ! com>
Date:       2017-06-25 9:43:47
Message-ID: 594f85d3.438b1c0a.f04db.3ab6 () mx ! google ! com
[Download RAW message or body]

Author: Armin Rigo <arigo@tunes.org>
Branch: py3.5
Changeset: r91646:fd92a8bc06b6
Date: 2017-06-25 11:43 +0200
http://bitbucket.org/pypy/pypy/changeset/fd92a8bc06b6/

Log:	Issue #2579

	First approach to fix a "leak" in _lzma.py

diff --git a/lib_pypy/_lzma.py b/lib_pypy/_lzma.py
--- a/lib_pypy/_lzma.py
+++ b/lib_pypy/_lzma.py
@@ -10,6 +10,7 @@
 import weakref
 import sys
 import io
+import __pypy__
 
 from _lzma_cffi import ffi, lib as m
 
@@ -688,6 +689,16 @@
         self.lock = threading.Lock()
         self.flushed = 0
         self.lzs = _new_lzma_stream()
+        # Issue #2579:
+        # Setting up the stream for encoding takes around 17MB of RAM
+        # on my Linux 64 system.  That's potentially too much to sanely
+        # leave it to the GC: in case we're compressing a large number
+        # of small files, the following line puts a big pressure on the
+        # major collections.  Still better than without it, where it
+        # would allocate huge amount of RAMs before doing any collection.
+        # Ideally we should do something more clever like reusing old
+        # streams after we're finished with them.
+        __pypy__.add_memory_pressure(1024*1024*17)
         if format == FORMAT_XZ:
             if filters is None:
                 if check == -1:
_______________________________________________
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit
[prev in list] [next in list] [prev in thread] [next in thread]