[prev in list] [next in list] [prev in thread] [next in thread]
List: subversion-dev
Subject: RE: svn commit: r1523465 - /subversion/trunk/subversion/svnserve/svnserve.c
From: "Bert Huijben" <bert () qqmail ! nl>
Date: 2013-09-15 18:43:43
Message-ID: 00f901ceb243$81d7a770$8586f650$ () qqmail ! nl
[Download RAW message or body]
This is a multipart message in MIME format.
Ah, I read the patch too fast.
In this snippet
[[
+/* Clear and release the given connection POOL.
+ */
+static void
+release_connection_pool(apr_pool_t *pool)
+{
+ svn_error_t *err;
+ svn_pool_clear(pool);
+
+ err = svn_mutex__lock(connection_pools_mutex);
+ if (err)
+ {
+ svn_error_clear(err);
+ svn_pool_destroy(pool);
+ }
+ else
+ {
+ APR_ARRAY_PUSH(connection_pools, apr_pool_t *) = pool;
+ svn_error_clear(svn_mutex__unlock(connection_pools_mutex,
+ SVN_NO_ERROR));
+ }
+}
+
]]
I automatically assumed the 'svn_pool_clear' was just a variable assignment,
because usually we have a white line before the first real code. instead of
just after the first line.
Maybe we should fix the whitespace here to follow our usual layout?
Bert
From: Stefan Fuhrmann [mailto:stefan.fuhrmann@wandisco.com]
Sent: zondag 15 september 2013 20:25
To: Bert Huijben
Cc: Subversion Development; commits@subversion.apache.org
Subject: Re: svn commit: r1523465 -
/subversion/trunk/subversion/svnserve/svnserve.c
On Sun, Sep 15, 2013 at 8:07 PM, Bert Huijben <bert@qqmail.nl
<mailto:bert@qqmail.nl> > wrote:
> -----Original Message-----
> From: stefan2@apache.org <mailto:stefan2@apache.org>
[mailto:stefan2@apache.org <mailto:stefan2@apache.org> ]
> Sent: zondag 15 september 2013 19:47
> To: commits@subversion.apache.org <mailto:commits@subversion.apache.org>
> Subject: svn commit: r1523465 -
> /subversion/trunk/subversion/svnserve/svnserve.c
>
> Author: stefan2
> Date: Sun Sep 15 17:46:36 2013
> New Revision: 1523465
>
> URL: http://svn.apache.org/r1523465
> Log:
> As it turns out, allocating memory from the OS in a multi-threaded
> environment is relatively costly. With APR pools, this happens
> every time we use a newly created root pool.
>
> Therefore, teach svnserve to recycle the connection pools, keeping
> those precious memory blocks allocated instead of disposing and
> re-allocating them.
Is this really the best way to do this?
Not sure. I'm open for suggestions.
Can't we create a subpool here? (Or do we also need multiple allocators,
etc.)
No. Those pools will be used concurrently by their
respective worker threads.
In the implementation I see that the existing pools are re-used, but they
are not *cleared* before re-use?
release_connection_pool() clears them.
Shouldn't we at least release the used memory (and thate) when handing back
the memory to the pool allocator?
-- Stefan^2.
[Attachment #3 (text/html)]
<html xmlns:v="urn:schemas-microsoft-com:vml" \
xmlns:o="urn:schemas-microsoft-com:office:office" \
xmlns:w="urn:schemas-microsoft-com:office:word" \
xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" \
xmlns="http://www.w3.org/TR/REC-html40"><head><META HTTP-EQUIV="Content-Type" \
CONTENT="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 15 \
(filtered medium)"><style><!-- /* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:"Courier New";}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple><div \
class=WordSection1><p class=MsoNormal><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Ah, I read \
the patch too fast.<o:p></o:p></span></p><p class=MsoNormal><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p \
class=MsoNormal><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>In this \
snippet<o:p></o:p></span></p><p class=MsoNormal><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>[[<o:p></o:p></span></p><p \
class=MsoNormal style='background:#CCCCCC'><span lang=EN \
style='font-size:10.0pt;font-family:"Courier New";color:black'>+/* Clear and release \
the given connection POOL.<o:p></o:p></span></p><p class=MsoNormal \
style='background:#CCCCCC'><span lang=EN style='font-size:10.0pt;font-family:"Courier \
New";color:black'>+ */<o:p></o:p></span></p><p class=MsoNormal \
style='background:#CCCCCC'><span lang=EN style='font-size:10.0pt;font-family:"Courier \
New";color:black'>+static void<o:p></o:p></span></p><p class=MsoNormal \
style='background:#CCCCCC'><span lang=EN style='font-size:10.0pt;font-family:"Courier \
New";color:black'>+release_connection_pool(apr_pool_t *pool)<o:p></o:p></span></p><p \
class=MsoNormal style='background:#CCCCCC'><span lang=EN \
style='font-size:10.0pt;font-family:"Courier \
New";color:black'>+{<o:p></o:p></span></p><p class=MsoNormal \
style='background:#CCCCCC'><span lang=EN style='font-size:10.0pt;font-family:"Courier \
New";color:black'>+ svn_error_t *err;<o:p></o:p></span></p><p class=MsoNormal \
style='background:#CCCCCC'><span lang=EN style='font-size:10.0pt;font-family:"Courier \
New";color:black'>+ svn_pool_clear(pool);<o:p></o:p></span></p><p \
class=MsoNormal style='background:#CCCCCC'><span lang=EN \
style='font-size:10.0pt;font-family:"Courier \
New";color:black'>+<o:p></o:p></span></p><p class=MsoNormal \
style='background:#CCCCCC'><span lang=EN style='font-size:10.0pt;font-family:"Courier \
New";color:black'>+ err = \
svn_mutex__lock(connection_pools_mutex);<o:p></o:p></span></p><p class=MsoNormal \
style='background:#CCCCCC'><span lang=EN style='font-size:10.0pt;font-family:"Courier \
New";color:black'>+ if (err)<o:p></o:p></span></p><p class=MsoNormal \
style='background:#CCCCCC'><span lang=EN style='font-size:10.0pt;font-family:"Courier \
New";color:black'>+ {<o:p></o:p></span></p><p class=MsoNormal \
style='background:#CCCCCC'><span lang=EN style='font-size:10.0pt;font-family:"Courier \
New";color:black'>+ \
svn_error_clear(err);<o:p></o:p></span></p><p class=MsoNormal \
style='background:#CCCCCC'><span lang=EN style='font-size:10.0pt;font-family:"Courier \
New";color:black'>+ \
svn_pool_destroy(pool);<o:p></o:p></span></p><p class=MsoNormal \
style='background:#CCCCCC'><span lang=EN style='font-size:10.0pt;font-family:"Courier \
New";color:black'>+ }<o:p></o:p></span></p><p class=MsoNormal \
style='background:#CCCCCC'><span lang=EN style='font-size:10.0pt;font-family:"Courier \
New";color:black'>+ else<o:p></o:p></span></p><p class=MsoNormal \
style='background:#CCCCCC'><span lang=EN style='font-size:10.0pt;font-family:"Courier \
New";color:black'>+ {<o:p></o:p></span></p><p class=MsoNormal \
style='background:#CCCCCC'><span lang=EN style='font-size:10.0pt;font-family:"Courier \
New";color:black'>+ APR_ARRAY_PUSH(connection_pools, \
apr_pool_t *) = pool;<o:p></o:p></span></p><p class=MsoNormal \
style='background:#CCCCCC'><span lang=EN style='font-size:10.0pt;font-family:"Courier \
New";color:black'>+ \
svn_error_clear(svn_mutex__unlock(connection_pools_mutex,<o:p></o:p></span></p><p \
class=MsoNormal style='background:#CCCCCC'><span lang=EN \
style='font-size:10.0pt;font-family:"Courier \
New";color:black'>+ & \
nbsp; &nb \
sp; \
SVN_NO_ERROR));<o:p></o:p></span></p><p class=MsoNormal \
style='background:#CCCCCC'><span lang=EN style='font-size:10.0pt;font-family:"Courier \
New";color:black'>+ }<o:p></o:p></span></p><p class=MsoNormal \
style='background:#CCCCCC'><span lang=EN style='font-size:10.0pt;font-family:"Courier \
New";color:black'>+}<o:p></o:p></span></p><p class=MsoNormal \
style='background:#CCCCCC'><span lang=EN style='font-size:10.0pt;font-family:"Courier \
New";color:black'>+<o:p></o:p></span></p><p class=MsoNormal><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>]]<o:p></o:p></span></p><p \
class=MsoNormal><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>I \
automatically assumed the ‘svn_pool_clear’ was just a variable \
assignment, because usually we have a white line before the first real code… \
instead of just after the first line. <o:p></o:p></span></p><p class=MsoNormal><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p \
class=MsoNormal><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Maybe we \
should fix the whitespace here to follow our usual layout?<o:p></o:p></span></p><p \
class=MsoNormal><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p \
class=MsoNormal><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> \
Bert<o:p></o:p></span></p><p class=MsoNormal><a name="_MailEndCompose"><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></a></p><div \
style='border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt'><div><div \
style='border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p \
class=MsoNormal><b><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>From:</span></b><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif"'> Stefan Fuhrmann \
[mailto:stefan.fuhrmann@wandisco.com] <br><b>Sent:</b> zondag 15 september 2013 \
20:25<br><b>To:</b> Bert Huijben<br><b>Cc:</b> Subversion Development; \
commits@subversion.apache.org<br><b>Subject:</b> Re: svn commit: r1523465 - \
/subversion/trunk/subversion/svnserve/svnserve.c<o:p></o:p></span></p></div></div><p \
class=MsoNormal><o:p> </o:p></p><div><div><div><p class=MsoNormal>On Sun, Sep \
15, 2013 at 8:07 PM, Bert Huijben <<a href="mailto:bert@qqmail.nl" \
target="_blank">bert@qqmail.nl</a>> wrote:<o:p></o:p></p><blockquote \
style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm \
6.0pt;margin-left:4.8pt;margin-right:0cm'><p class=MsoNormal><br><br>> \
-----Original Message-----<br>> From: <a \
href="mailto:stefan2@apache.org">stefan2@apache.org</a> [mailto:<a \
href="mailto:stefan2@apache.org">stefan2@apache.org</a>]<br>> Sent: zondag 15 \
september 2013 19:47<br>> To: <a \
href="mailto:commits@subversion.apache.org">commits@subversion.apache.org</a><br>> \
Subject: svn commit: r1523465 -<br>> \
/subversion/trunk/subversion/svnserve/svnserve.c<br>><br>> Author: \
stefan2<br>> Date: Sun Sep 15 17:46:36 2013<br>> New Revision: \
1523465<br>><br>> URL: <a href="http://svn.apache.org/r1523465" \
target="_blank">http://svn.apache.org/r1523465</a><br>> Log:<br>> As it turns \
out, allocating memory from the OS in a multi-threaded<br>> environment is \
relatively costly. With APR pools, this happens<br>> every time we use a \
newly created root pool.<br>><br>> Therefore, teach svnserve to recycle the \
connection pools, keeping<br>> those precious memory blocks allocated instead of \
disposing and<br>> re-allocating them.<br><br>Is this really the best way to do \
this?<o:p></o:p></p></blockquote><div><p \
class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Not sure. I'm open \
for suggestions.<br> <o:p></o:p></p></div><blockquote \
style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm \
6.0pt;margin-left:4.8pt;margin-right:0cm'><p class=MsoNormal>Can't we create a \
subpool here? (Or do we also need multiple allocators, \
etc.)<o:p></o:p></p></blockquote><div><p \
class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>No. Those pools \
will be used concurrently by their<br>respective worker \
threads.<o:p></o:p></p></div><div><p \
class=MsoNormal> <o:p></o:p></p></div><blockquote \
style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm \
6.0pt;margin-left:4.8pt;margin-right:0cm'><p class=MsoNormal>In the implementation I \
see that the existing pools are re-used, but they are not *cleared* before \
re-use?<o:p></o:p></p></blockquote><div><p \
class=MsoNormal><o:p> </o:p></p></div><div><p \
class=MsoNormal>release_connection_pool() clears them.<o:p></o:p></p></div><div><p \
class=MsoNormal> <o:p></o:p></p></div><blockquote \
style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm \
6.0pt;margin-left:4.8pt;margin-right:0cm'><p class=MsoNormal>Shouldn't we at least \
release the used memory (and thate) when handing back the memory to the pool \
allocator?<o:p></o:p></p></blockquote><div><p \
class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>-- \
Stefan^2.<o:p></o:p></p></div></div><p \
class=MsoNormal><o:p> </o:p></p></div></div></div></div></body></html>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic