[prev in list] [next in list] [prev in thread] [next in thread]
List: geos-devel
Subject: Re: [geos-devel] Thread safety issues
From: Tamas Szekeres <szekerest () gmail ! com>
Date: 2017-06-29 22:00:01
Message-ID: CACALY+SHk=kbSf4Pb7DbVyHNK6yaR8Vnf0HZk+yJuWj0U98Kcw () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
Thank you, it is sometimes suggested to call the per-thread alternative
uselocale instead of setlocale, for example:
locale_t loc = newlocale(LC_ALL_MASK, "C", NULL);
uselocale(loc);
freelocale(loc)// Do your thing
Or setting the locale on a per stream basis, like:
std::istringstream text( s );
text.imbue(std::locale::classic());
Not sure if those are applicable in this specific case, however.
Best regards,
Tamas
2017-06-29 23:35 GMT+02:00 Even Rouault <even.rouault@spatialys.com>:
> On jeudi 29 juin 2017 23:19:30 CEST Tamas Szekeres wrote:
>
> > Dear developers,
>
> >
>
> > We have encountered thread safety issues when invoking the geos C api
> from
>
> > multiple threads on a site with strong traffic.
>
> >
>
> > Specifically CLocalizer is using std::setlocale which is not thread
> safe. (
>
> > http://en.cppreference.com/w/cpp/locale/setlocale). Do we have a
> workaround
>
> > to this?
>
>
>
> This strongly reminds me of similar issues we have hit in GDAL and proj.4
>
>
>
> In GDAL, we've now a CPLThreadLocaleC class that use Linux & Windows
> thread-locale specific API to force the C locale when available, and
> otherwise default to a setlocale() call protected by a mutex (which only
> protects 2 concurrent calls to setlocale(), but not a setlocale() with a
> printf())
>
>
>
> See
>
> https://github.com/OSGeo/gdal/blob/trunk/gdal/port/cpl_conv.cpp#L2785
>
> https://github.com/OSGeo/gdal/blob/trunk/gdal/port/cpl_conv.cpp#L2843
>
>
>
> But ultimately I ended up rewriting my CPLvsnprintf() to avoid messing up
> with the C locale, and just using vsnprintf() with a later replacing of the
> comma by decimal point (I guess this must be the use case in GEOS too ?) :
>
> https://github.com/OSGeo/gdal/blob/trunk/gdal/port/cpl_string.cpp#L1119
>
>
>
> And similarly for CPLsscanf() :
>
> https://github.com/OSGeo/gdal/blob/trunk/gdal/port/cpl_string.cpp#L1430
>
>
>
>
>
> Not sure if those are the best way of addressing the issue (I'd wish there
> would be API using always the C-locale), but they are now rather well
> proven.
>
>
>
> Even
>
>
>
> --
>
> Spatialys - Geospatial professional services
>
> http://www.spatialys.com
>
[Attachment #5 (text/html)]
<div dir="ltr">Thank you, it is sometimes suggested to call the per-thread \
alternative uselocale instead of setlocale, for example:<div><br></div><div><pre \
class="gmail-lang-cpp gmail-prettyprint gmail-prettyprinted" \
style="margin-top:0px;margin-bottom:1em;padding:5px;border:0px;font-variant-numeric:in \
herit;font-stretch:inherit;font-size:13px;line-height:inherit;font-family:Consolas,Menlo,Monaco,"Lucida \
Console","Liberation Mono","DejaVu Sans \
Mono","Bitstream Vera Sans Mono","Courier \
New",monospace,sans-serif;vertical-align:baseline;width:auto;max-height:600px;ove \
rflow:auto;background-color:rgb(239,240,241);color:rgb(57,51,24);word-wrap:normal"><code \
style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font- \
weight:inherit;font-stretch:inherit;line-height:inherit;font-family:Consolas,Menlo,Monaco,"Lucida \
Console","Liberation Mono","DejaVu Sans \
Mono","Bitstream Vera Sans Mono","Courier \
New",monospace,sans-serif;vertical-align:baseline;white-space:inherit"><span \
class="gmail-typ" style="margin:0px;padding:0px;border:0px;font-style:inherit;font-var \
iant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:in \
herit;font-family:inherit;vertical-align:baseline;color:rgb(43,145,175)">locale_t</span><span \
class="gmail-pln" style="margin:0px;padding:0px;border:0px;font-style:inherit;font-var \
iant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:inherit;vertical-align:baseline;color:rgb(48,51,54)"> \
loc </span><span class="gmail-pun" \
style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font- \
weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:inherit;vertical-align:baseline;color:rgb(48,51,54)">=</span><span \
class="gmail-pln" style="margin:0px;padding:0px;border:0px;font-style:inherit;font-var \
iant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:inherit;vertical-align:baseline;color:rgb(48,51,54)"> \
newlocale</span><span class="gmail-pun" \
style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font- \
weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:inherit;vertical-align:baseline;color:rgb(48,51,54)">(</span><span \
class="gmail-pln" style="margin:0px;padding:0px;border:0px;font-style:inherit;font-var \
iant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:in \
herit;font-family:inherit;vertical-align:baseline;color:rgb(48,51,54)">LC_ALL_MASK</span><span \
class="gmail-pun" style="margin:0px;padding:0px;border:0px;font-style:inherit;font-var \
iant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:in \
herit;font-family:inherit;vertical-align:baseline;color:rgb(48,51,54)">,</span><span \
class="gmail-pln" style="margin:0px;padding:0px;border:0px;font-style:inherit;font-var \
iant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:inherit;vertical-align:baseline;color:rgb(48,51,54)"> \
</span><span class="gmail-str" \
style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font- \
weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:inherit;vertical-align:baseline;color:rgb(125,39,39)">"C"</span><span \
class="gmail-pun" style="margin:0px;padding:0px;border:0px;font-style:inherit;font-var \
iant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:in \
herit;font-family:inherit;vertical-align:baseline;color:rgb(48,51,54)">,</span><span \
class="gmail-pln" style="margin:0px;padding:0px;border:0px;font-style:inherit;font-var \
iant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:inherit;vertical-align:baseline;color:rgb(48,51,54)"> \
NULL</span><span class="gmail-pun" \
style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font- \
weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:inherit;vertical-align:baseline;color:rgb(48,51,54)">);</span><span \
class="gmail-pln" style="margin:0px;padding:0px;border:0px;font-style:inherit;font-var \
iant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:inherit;vertical-align:baseline;color:rgb(48,51,54)">
uselocale</span><span class="gmail-pun" \
style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font- \
weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:inherit;vertical-align:baseline;color:rgb(48,51,54)">(</span><span \
class="gmail-pln" style="margin:0px;padding:0px;border:0px;font-style:inherit;font-var \
iant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:in \
herit;font-family:inherit;vertical-align:baseline;color:rgb(48,51,54)">loc</span><span \
class="gmail-pun" style="margin:0px;padding:0px;border:0px;font-style:inherit;font-var \
iant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:in \
herit;font-family:inherit;vertical-align:baseline;color:rgb(48,51,54)">);</span><span \
class="gmail-pln" style="margin:0px;padding:0px;border:0px;font-style:inherit;font-var \
iant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:inherit;vertical-align:baseline;color:rgb(48,51,54)">
freelocale</span><span class="gmail-pun" \
style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font- \
weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:inherit;vertical-align:baseline;color:rgb(48,51,54)">(</span><span \
class="gmail-pln" style="margin:0px;padding:0px;border:0px;font-style:inherit;font-var \
iant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:in \
herit;font-family:inherit;vertical-align:baseline;color:rgb(48,51,54)">loc</span><span \
class="gmail-pun" style="margin:0px;padding:0px;border:0px;font-style:inherit;font-var \
iant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:in \
herit;font-family:inherit;vertical-align:baseline;color:rgb(48,51,54)">)</span><span \
class="gmail-pln" style="margin:0px;padding:0px;border:0px;font-style:inherit;font-var \
iant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:inherit;vertical-align:baseline;color:rgb(48,51,54)">
</span><span class="gmail-com" \
style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font- \
weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:inherit;vertical-align:baseline;color:rgb(133,140,147)">// \
Do your thing</span></code></pre>Or setting the locale on a per stream basis, \
like:</div><div><br></div><div><pre class="gmail-lang-c gmail-prettyprint \
gmail-prettyprinted" \
style="margin-top:0px;margin-bottom:1em;padding:5px;border:0px;font-variant-numeric:in \
herit;font-stretch:inherit;font-size:13px;line-height:inherit;font-family:Consolas,Menlo,Monaco,"Lucida \
Console","Liberation Mono","DejaVu Sans \
Mono","Bitstream Vera Sans Mono","Courier \
New",monospace,sans-serif;vertical-align:baseline;width:auto;max-height:600px;ove \
rflow:auto;background-color:rgb(239,240,241);color:rgb(57,51,24);word-wrap:normal"><code \
style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font- \
weight:inherit;font-stretch:inherit;line-height:inherit;font-family:Consolas,Menlo,Monaco,"Lucida \
Console","Liberation Mono","DejaVu Sans \
Mono","Bitstream Vera Sans Mono","Courier \
New",monospace,sans-serif;vertical-align:baseline;white-space:inherit"><span \
class="gmail-pln" style="margin:0px;padding:0px;border:0px;font-style:inherit;font-var \
iant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:in \
herit;font-family:inherit;vertical-align:baseline;color:rgb(48,51,54)">std</span><span \
class="gmail-pun" style="margin:0px;padding:0px;border:0px;font-style:inherit;font-var \
iant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:in \
herit;font-family:inherit;vertical-align:baseline;color:rgb(48,51,54)">::</span><span \
class="gmail-pln" style="margin:0px;padding:0px;border:0px;font-style:inherit;font-var \
iant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:in \
herit;font-family:inherit;vertical-align:baseline;color:rgb(48,51,54)">istringstream \
text</span><span class="gmail-pun" \
style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font- \
weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:inherit;vertical-align:baseline;color:rgb(48,51,54)">(</span><span \
class="gmail-pln" style="margin:0px;padding:0px;border:0px;font-style:inherit;font-var \
iant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:inherit;vertical-align:baseline;color:rgb(48,51,54)"> \
s </span><span class="gmail-pun" \
style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font- \
weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:inherit;vertical-align:baseline;color:rgb(48,51,54)">);</span><span \
class="gmail-pln" style="margin:0px;padding:0px;border:0px;font-style:inherit;font-var \
iant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:inherit;vertical-align:baseline;color:rgb(48,51,54)">
text</span><span class="gmail-pun" \
style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font- \
weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:inherit;vertical-align:baseline;color:rgb(48,51,54)">.</span><span \
class="gmail-pln" style="margin:0px;padding:0px;border:0px;font-style:inherit;font-var \
iant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:in \
herit;font-family:inherit;vertical-align:baseline;color:rgb(48,51,54)">imbue</span><span \
class="gmail-pun" style="margin:0px;padding:0px;border:0px;font-style:inherit;font-var \
iant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:in \
herit;font-family:inherit;vertical-align:baseline;color:rgb(48,51,54)">(</span><span \
class="gmail-pln" style="margin:0px;padding:0px;border:0px;font-style:inherit;font-var \
iant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:in \
herit;font-family:inherit;vertical-align:baseline;color:rgb(48,51,54)">std</span><span \
class="gmail-pun" style="margin:0px;padding:0px;border:0px;font-style:inherit;font-var \
iant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:in \
herit;font-family:inherit;vertical-align:baseline;color:rgb(48,51,54)">::</span><span \
class="gmail-pln" style="margin:0px;padding:0px;border:0px;font-style:inherit;font-var \
iant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:in \
herit;font-family:inherit;vertical-align:baseline;color:rgb(48,51,54)">locale</span><span \
class="gmail-pun" style="margin:0px;padding:0px;border:0px;font-style:inherit;font-var \
iant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:in \
herit;font-family:inherit;vertical-align:baseline;color:rgb(48,51,54)">::</span><span \
class="gmail-pln" style="margin:0px;padding:0px;border:0px;font-style:inherit;font-var \
iant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:in \
herit;font-family:inherit;vertical-align:baseline;color:rgb(48,51,54)">classic</span><span \
class="gmail-pun" style="margin:0px;padding:0px;border:0px;font-style:inherit;font-var \
iant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:in \
herit;font-family:inherit;vertical-align:baseline;color:rgb(48,51,54)">());</span></code></pre></div><div>Not \
sure if those are applicable in this specific case, \
however.</div><div><br></div><div>Best \
regards,</div><div><br></div><div>Tamas</div><div><br></div><div><br></div><div><br></div><div><br></div></div><div \
class="gmail_extra"><br><div class="gmail_quote">2017-06-29 23:35 GMT+02:00 Even \
Rouault <span dir="ltr"><<a href="mailto:even.rouault@spatialys.com" \
target="_blank">even.rouault@spatialys.com</a>></span>:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><u></u> <div style="font-family:'Sans \
Serif';font-size:9pt;font-weight:400;font-style:normal"><span class=""> <p \
style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">On \
jeudi 29 juin 2017 23:19:30 CEST Tamas Szekeres wrote:</p> <p \
style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> \
Dear developers,</p> <p \
style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> \
</p> <p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> \
We have encountered thread safety issues when invoking the geos C api from</p> <p \
style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> \
multiple threads on a site with strong traffic.</p> <p \
style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> \
</p> <p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> \
Specifically CLocalizer is using std::setlocale which is not thread safe. (</p> <p \
style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> \
<a href="http://en.cppreference.com/w/cpp/locale/setlocale" \
target="_blank">http://en.cppreference.com/w/<wbr>cpp/locale/setlocale</a>). Do we \
have a workaround</p> <p \
style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">> \
to this?</p> <p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px"> \
</p> </span><p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">This \
strongly reminds me of similar issues we have hit in GDAL and proj.4</p> <p \
style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px"> \
</p> <p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">In \
GDAL, we've now a CPLThreadLocaleC class that use Linux & Windows \
thread-locale specific API to force the C locale when available, and otherwise \
default to a setlocale() call protected by a mutex (which only protects 2 concurrent \
calls to setlocale(), but not a setlocale() with a printf())</p> <p \
style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px"> \
</p> <p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">See</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px"><a \
href="https://github.com/OSGeo/gdal/blob/trunk/gdal/port/cpl_conv.cpp#L2785" \
target="_blank">https://github.com/OSGeo/gdal/<wbr>blob/trunk/gdal/port/cpl_conv.<wbr>cpp#L2785</a></p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px"><a \
href="https://github.com/OSGeo/gdal/blob/trunk/gdal/port/cpl_conv.cpp#L2843" \
target="_blank">https://github.com/OSGeo/gdal/<wbr>blob/trunk/gdal/port/cpl_conv.<wbr>cpp#L2843</a></p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px"> \
</p> <p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">But \
ultimately I ended up rewriting my CPLvsnprintf() to avoid messing up with the C \
locale, and just using vsnprintf() with a later replacing of the comma by decimal \
point (I guess this must be the use case in GEOS too ?) :</p> <p \
style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px"><a \
href="https://github.com/OSGeo/gdal/blob/trunk/gdal/port/cpl_string.cpp#L1119" \
target="_blank">https://github.com/OSGeo/gdal/<wbr>blob/trunk/gdal/port/cpl_<wbr>string.cpp#L1119</a></p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px"> \
</p> <p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">And \
similarly for CPLsscanf() :</p> <p \
style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px"><a \
href="https://github.com/OSGeo/gdal/blob/trunk/gdal/port/cpl_string.cpp#L1430" \
target="_blank">https://github.com/OSGeo/gdal/<wbr>blob/trunk/gdal/port/cpl_<wbr>string.cpp#L1430</a></p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px"> \
</p> <p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px"> \
</p> <p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">Not \
sure if those are the best way of addressing the issue (I'd wish there would be \
API using always the C-locale), but they are now rather well proven.</p> <p \
style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px"> \
</p> <p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">Even</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px"> \
</p> <p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">-- \
</p> <p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">Spatialys \
- Geospatial professional services</p> <p \
style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px"><a \
[Attachment #6 (text/plain)]
_______________________________________________
geos-devel mailing list
geos-devel@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/geos-devel
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic