'vmodule data, that would be private per thread'

[prev in list] [next in list] [prev in thread] [next in thread] 

List:       varnish-misc
Subject:    vmodule data, that would be private per thread
From:       "Mikhail T." <mi+thun () aldan ! algebra ! com>
Date:       2012-10-24 13:26:19
Message-ID: 5087EC7B.1010602 () aldan ! algebra ! com
[Download RAW message or body]

[Attachment #2 (multipart/alternative)]

I need to cache and reuse some data in each thread -- to avoid 
lock-contention. It does not seem like PRIV_VCL and PRIV_CALL offer 
that, so, what I am doing, is this:

 1. init_function creates static version of the data once and
    initializes thread key with pthread_key_create()
 2. each worker functions attempts to extract its thread's copy of the
    data with pthread_getspecific()
 3. if the pthread_getspecific() returned NULL (first call in this
    thread), new memory is allocated and recorded with pthread_setspecific()

Once loaded, the memory is not -- and needs not -- be freed until 
varnishd is restarted. This seems to work, however:

  * I see twice as many new-memory allocations as the top limit on the
    number of worker-threads: I run varnishd with -w 4,7 argument, but,
    when I hit it with ab, 14 lines indicating creation of a new copy of
    data are logged (with 14 different thread-IDs).
  * I'm wondering, if there is already Varnish API, that would hide
    these pthread_* manipulations...

Any ideas? Thank you!

    -mi

[Attachment #5 (text/html)]

<html>
  <head>

    <meta http-equiv="content-type" content="text/html; charset=ISO-8859-1">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    I need to cache and reuse some data in each thread -- to avoid
    lock-contention. It does not seem like <tt>PRIV_VCL</tt> and <tt>PRIV_CALL</tt>
    offer that, so, what I am doing, is this:<br>
    <ol>
      <li>init_function creates static version of the data once and
        initializes thread key with <tt>pthread_key_create()</tt></li>
      <li>each worker functions attempts to extract its thread's copy of
        the data with <tt>pthread_getspecific()</tt></li>
      <li>if the <tt>pthread_getspecific()</tt> returned NULL (first
        call in this thread), new memory is allocated and recorded with
        <tt>pthread_setspecific()</tt></li>
    </ol>
    Once loaded, the memory is not -- and needs not -- be freed until <tt>varnishd</tt>
    is restarted. This seems to work, however:<br>
    <ul>
      <li>I see twice as many new-memory allocations as the top limit on
        the number of worker-threads: I run <tt>varnishd</tt> with <tt>-w
          4,7</tt> argument, but, when I hit it with ab, 14 lines
        indicating creation of a new copy of data are logged (with 14
        different thread-IDs).<br>
      </li>
      <li>I'm wondering, if there is already Varnish API, that would
        hide these pthread_* manipulations...</li>
    </ul>
    Any ideas? Thank you!<br>
    <blockquote>-mi<br>
    </blockquote>
    <blockquote><br>
    </blockquote>
  </body>
</html>

_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

[prev in list] [next in list] [prev in thread] [next in thread] 