[prev in list] [next in list] [prev in thread] [next in thread] 

List:       perl5-changes
Subject:    [Perl/perl5] 8a656e: POSIX::mblen() Make thread-safe; allow shift state...
From:       Karl Williamson <noreply () github ! com>
Date:       2019-12-30 16:19:24
Message-ID: Perl/perl5/push/refs/heads/locale/000000-fc9527 () github ! com
[Download RAW message or body]

  Branch: refs/heads/locale
  Home:   https://github.com/Perl/perl5
  Commit: 8a656e3d84b0e1c031e641aafa8559889a42ce53
      https://github.com/Perl/perl5/commit/8a656e3d84b0e1c031e641aafa8559889a42ce53
  Author: Karl Williamson <khw@cpan.org>
  Date:   2019-12-30 (Mon, 30 Dec 2019)

  Changed paths:
    M embedvar.h
    M ext/POSIX/POSIX.xs
    M ext/POSIX/lib/POSIX.pod
    M ext/POSIX/t/mb.t
    M intrpvar.h
    M locale.c
    M perl.h
    M pod/perldelta.pod
    M t/porting/known_pod_issues.dat

  Log Message:
  -----------
  POSIX::mblen() Make thread-safe; allow shift state control

This commit changes to use mbrlen() (mostly) transparently, when
available, to achieve thread-safe operation.  It is not completely
transparent because mbrlen may set errno differently upon errors, and
no effort is made to hide that difference.

This commit also changes the behavior so that it takes a scalar
parameter instead of a char *, and this might not be forceable into a
valid PV.  If not a PV, mblen() is always used (with a semaphore) to
initialize the shift state.  Previously the shift state was always
reinitialized with every call, which meant this could not work on
locales with shift states.

[perl #133928] showed that someone was having trouble with shift states.


  Commit: 653ef2ca336b45ca58c393a116bb0a04e281edc5
      https://github.com/Perl/perl5/commit/653ef2ca336b45ca58c393a116bb0a04e281edc5
  Author: Karl Williamson <khw@cpan.org>
  Date:   2019-12-30 (Mon, 30 Dec 2019)

  Changed paths:
    M ext/POSIX/t/mb.t

  Log Message:
  -----------
  ext/POSIX/t/mb.t: Add another mblen test

Also fix a skip count


  Commit: 6f71fd2bb7991c49a763d398ab0dc4078238faae
      https://github.com/Perl/perl5/commit/6f71fd2bb7991c49a763d398ab0dc4078238faae
  Author: Karl Williamson <khw@cpan.org>
  Date:   2019-12-30 (Mon, 30 Dec 2019)

  Changed paths:
    M ext/POSIX/POSIX.xs
    M ext/POSIX/lib/POSIX.pod
    M ext/POSIX/t/mb.t
    M pod/perldelta.pod

  Log Message:
  -----------
  POSIX::mblen: Make 2nd parameter optional

Now that an SV is passed instead of a PV, its length is available
without having to have a second parameter.  The second parameter is now
only used if it restricts the desired length to scan to less than that
available.


  Commit: fc9527e64174b9653a276790a19145773ee58b7d
      https://github.com/Perl/perl5/commit/fc9527e64174b9653a276790a19145773ee58b7d
  Author: Karl Williamson <khw@cpan.org>
  Date:   2019-12-30 (Mon, 30 Dec 2019)

  Changed paths:
    M embedvar.h
    M ext/POSIX/POSIX.xs
    M ext/POSIX/lib/POSIX.pod
    M ext/POSIX/t/mb.t
    M intrpvar.h
    M locale.c
    M pod/perldelta.pod
    M t/porting/known_pod_issues.dat

  Log Message:
  -----------
  Fixup POSIX::mbtowc, wctomb

This commit enhances these functions so that they use mbrtowc and
wcrtomb when available, making them thread safe.  The substitution isn't
completely transparent, as no effort is made to hide any differences in
errno setting upon error.

This commit also changes the behaviors so that they take a scalar
parameter instead of a char *, and this might be 'undef' or not be
forceable into a valid PV.  If not a PV, the non-r version is always
used (with a semaphore) to initialize the shift state.  Previously the
shift state was always reinitialized with every call, which meant these
could not work on locales with shift states.

In addition, there were several issues in mbtowc and wctomb that this
commit fixes.

mbtowc is now run with a semaphore (when mbrtowc isn't available).  This
avoids races if called at the same time in another thread.

The final parameter to mbtowc is now optional, as passing an SV allows
us to determine the length without the need for an extra parameter.  It
is now used only to restrict the parsing of the string to shorter than
the actual length.

wctomb would segfault if the string parameter was shared or hadn't
been pre-allocated with a string of sufficient length to hold the
result.

wctomb is now run with a semaphore (when wcrtomb isn't available).  This
avoids races if called at the same time in another thread.


Compare: https://github.com/Perl/perl5/compare/8a656e3d84b0%5E...fc9527e64174
[prev in list] [next in list] [prev in thread] [next in thread]