[prev in list] [next in list] [prev in thread] [next in thread]
List: cfe-dev
Subject: Re: [cfe-dev] Issue where Lambda capture of reference to globals doesn't actually capture anything
From: Richard Smith via cfe-dev <cfe-dev () lists ! llvm ! org>
Date: 2020-10-27 20:56:58
Message-ID: CAOfiQqkWWRrkbR=iSth0Z27Pp3waxyKb2N4w_eZKO+ahvHK+Fg () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
On Tue, 27 Oct 2020 at 13:50, Hollman, Daisy Sophia <dshollm@sandia.gov>
wrote:
> Yeah, but if you're explicit about the constexpr it works fine in both
> compilers: https://godbolt.org/z/xdad8v. Here's a summary of frontend
> divergence on this issue: https://godbolt.org/z/KfdKcT. I think basically
> this is one of those areas where implementation divergence is so
> significant that no one can write portable code dependent on it, so we
> should consider simplifying. I understand the desire to make []{ return
> x; }, [=]{ return x; }, and [x]{ return x; } all do the same thing
> (though that doesn't save us from the fact that [x=x]{ return x; } does
> something different), but the reality is that anyone who is writing code
> that's dependent on this behavior is either only using Clang
> or—worse—depending on the wrong behavior and avoiding Clang. And that
> doesn't even always work; consider this example:
> https://godbolt.org/z/ETMGEq where adding a default capture causes Clang
> to treat the reference to a global as a reference to const.
>
That's a really interesting example. Yes, we're certainly mishandling that
per the language rules.
> I'm not sure where the references-are-implicitly-constexpr part came from
>
> I think this is part of the problem. I think most people would find this
> at least slightly less confusing:
>
> int foo = 1;
> int main() {
> constexpr auto& myfoo = foo;
> auto x = [=]{ return myfoo; };
> std::cout << x() << std::endl;
> foo = 2;
> std::cout << x() << std::endl;
> }
>
> But I think probably the best solution would be to consider deprecating
> the use of constexpr references in capture-by-value lambdas. The argument
> in many cases is pretty similar to the problems with implicit this
> capture. Very few people would be surprised by the behavior of this
> program, for instance:
>
> int foo = 1;
> int main() {
> constexpr auto& myfoo = foo;
> auto x = [=, &myfoo]{ return myfoo; };
> std::cout << x() << std::endl;
> foo = 2;
> std::cout << x() << std::endl;
> }
>
> The fact that this works the same way currently on all of the major
> implementations is another bonus.
>
> But this is really the wrong place to have such discussions :)
>
> Yeah, I agree that whatever happens, I think this should probably be a
> Core issue that we should switch this discussion over to the ISO mailing
> list.
>
Sounds reasonable to me.
> Thanks,
>
> Daisy 🌼
>
> On October 27, 2020 at 1:00:31 PM, Richard Smith (richard@metafoo.co.uk)
> wrote:
>
> On Tue, 27 Oct 2020 at 12:11, Arthur O'Dwyer <arthur.j.odwyer@gmail.com>
> wrote:
>
>> On Mon, Oct 26, 2020 at 8:14 PM Richard Smith via cfe-dev <
>> cfe-dev@lists.llvm.org> wrote:
>> > On Mon, 26 Oct 2020 at 16:57, Lewis, Cannada via cfe-dev <
>> cfe-dev@lists.llvm.org> wrote:
>> >>
>> >> I'm not a standards reading expert but does
>> >>
>> >> Note 7:
>> >> An id-expression that is not an odr-use refers to the original entity,
>> never to a member of the closure type.
>> >> However, such an id-expression can still cause the implicit capture of
>> the entity.
>> >> — end note
>> >>
>> >> From the text imply that the program https://godbolt.org/z/feKxdK is
>> actually implementation defined? Or does gcc have a bug here?
>> >
>> > GCC has a bug, according to the standard wording. The mention of myfoo
>> does not constitute an odr-use, so is not rewritten to use the capture.
>> Clang's behavior is correct per the standard wording.
>> >
>> > The standard rule is certainly surprising in this particular case. I
>> think the rule in question is driven by the desire for adding a
>> capture-default to a valid lambda to not change its meaning. For example:
>> https://godbolt.org/z/nrWsvj
>>
>> Hmm. It's insane that you can use local variable `x` inside a lambda that
>> doesn't capture anything; I wasn't aware of that.
>>
>
> Aside: "insane" (here) and "crazy" (below) are somewhat more charged
> language than we'd prefer here.
>
>
>> If the formal wording really says you can do that, then that seems like a
>> formal-wording bug to me.
>>
>
> It's entirely intentional, though the case that was being thought about at
> the time was primarily local constant non-references, not local references
> (though the same rules apply to both).
>
> void f() {
> constexpr int k = 5;
> [] { int arr[k]; };
> }
>
> ... should obviously be valid. And more broadly, the rule is that anything
> that you can name without an odr-use is valid. And this isn't at all
> special to lambdas; the same thing happens with all nested constructs:
>
> int n;
> void g() {
> const int q = 6;
> int &r = n;
> constexpr int *p = &n;
> struct A {
> void f() {
> int arr[q]; // ok
> r = *p + 1; // ok
> }
> };
> }
>
> https://godbolt.org/z/avG7qx
>
> GCC and Clang disagree in a different way about whether it's okay to
>> dereference a *pointer* inside a lambda without capturing it; here it's
>> GCC that is doing the crazy thing, and Clang that is reasonably refusing to
>> compile a reference to `x` when `x` hasn't been captured.
>> https://godbolt.org/z/bjbr3f
>>
>
> Yeah, GCC's behavior is not in line with the language rules in the pointer
> case. I'd guess they're using a more-liberal evaluation mode when
> evaluating the initializer of the reference, rather than using the strict
> constant initializer rules, and that more-liberal mode permits them to read
> the initializer values of all const-qualified variables, not only the ones
> that are usable in constant expressions. (Clang has a more-liberal mode
> too, and I'd expect we also have bugs where a conforming program can tell
> the difference.)
>
> The rules in question are approximately equivalent to: if a variable of
> const integral or enumeration type, or of reference type, can be constexpr,
> then it is implicitly constexpr. And constexpr variables can be used from
> anywhere, without triggering an odr-use, if only their value, not their
> address, is used. (We don't actually say such things are implicitly
> constexpr, and there are corner cases where the outcome is different -- in
> particular, for static data members, where constexpr implies inline but
> const does not. But "implicitly constexpr" is how they behave.)
>
> If you change your example to declare the pointer to be constexpr then the
> two compilers agree again.
>
> The rules about when compilers are able to "constant-fold" away variables
>> that otherwise would need to be captured, strikes me as similar in spirit
>> to the rules about when compilers are able to "constant-fold" away
>> complicated expressions in constant expressions. Some compilers seem to be
>> trying to be "helpful" by letting the optimizer's knowledge leak into the
>> front-end, and some are following the same rules as one's "head compiler"
>> would.
>>
>
> I think this is at least not the proximal cause in the standard-conforming
> case -- correctly implementing the odr-use rules in the Clang frontend was
> a non-trivial effort and wasn't related to constant folding / optimizer
> knowledge leaking through. But I do think the standard rules here probably
> originally came from looking at what implementations at the time happened
> to do and writing that down (eg, use of a const static data member with no
> definition is OK, because implementations happen to not emit a reference to
> the symbol), and complexity has built up from that point. I'm not sure
> where the references-are-implicitly-constexpr part came from, but the
> const-ints-are-implicitly-constexpr part is a backwards-compatibility
> concern inherited from C++98.
>
> From a language design perspective, I think it would have made a lot more
> sense if we'd said that constexpr variables are implicitly static (so the
> question of capture or of use from other scopes can be answered trivially),
> and deprecated the "implicit constexpr" behavior for const
> integral/enumeration variables and references. Where we've ended up isn't a
> great compromise. But this is really the wrong place to have such
> discussions :)
>
>
[Attachment #5 (text/html)]
<div dir="ltr"><div dir="ltr">On Tue, 27 Oct 2020 at 13:50, Hollman, Daisy Sophia \
<<a href="mailto:dshollm@sandia.gov">dshollm@sandia.gov</a>> \
wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" \
style="margin:0px 0px 0px 0.8ex;border-left:1px solid \
rgb(204,204,204);padding-left:1ex">
<div>
<div class="gmail-m_-8426244070307930285bloop_markdown">
<p>Yeah, but if you're explicit about the <code>constexpr</code> it works fine in \
both compilers: <a href="https://godbolt.org/z/xdad8v" \
target="_blank">https://godbolt.org/z/xdad8v</a>. Here's a summary of frontend \
divergence on this issue: <a href="https://godbolt.org/z/KfdKcT" \
target="_blank">https://godbolt.org/z/KfdKcT</a>. I think basically this is one of \
those areas where implementation divergence is so significant that no one can write \
portable code dependent on it, so we should consider simplifying. I understand the \
desire to make <code>[]{ return x; }</code>, <code>[=]{ return x; }</code>, and \
<code>[x]{ return x; }</code> all do the same thing (though that doesn't save us from \
the fact that <code>[x=x]{ return x; }</code> does something different), but the \
reality is that anyone who is writing code that's dependent on this behavior is \
either only using Clang or—worse—depending on the wrong behavior and avoiding \
Clang. And that doesn't even always work; consider this example: <a \
href="https://godbolt.org/z/ETMGEq" target="_blank">https://godbolt.org/z/ETMGEq</a> \
where adding a default capture causes Clang to treat the reference to a global as a \
reference to <code>const</code>.</p></div></div></blockquote><div>That's a really \
interesting example. Yes, we're certainly mishandling that per the language \
rules.</div><blockquote class="gmail_quote" style="margin:0px 0px 0px \
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div \
class="gmail-m_-8426244070307930285bloop_markdown"> <blockquote>
<p>I'm not sure where the references-are-implicitly-constexpr part came from</p>
</blockquote>
<p>I think this is part of the problem. I think most people would find this at least \
slightly less confusing:</p> <pre><code>int foo = 1;
int main() {
constexpr auto& myfoo = foo;
auto x = [=]{ return myfoo; };
std::cout << x() << std::endl;
foo = 2;
std::cout << x() << std::endl;
}
</code></pre>
<p>But I think probably the best solution would be to consider deprecating the use of \
constexpr references in capture-by-value lambdas. The argument in many cases is \
pretty similar to the problems with implicit <code>this</code> capture. Very few \
people would be surprised by the behavior of this program, for instance:</p> \
<pre><code>int foo = 1; int main() {
constexpr auto& myfoo = foo;
auto x = [=, &myfoo]{ return myfoo; };
std::cout << x() << std::endl;
foo = 2;
std::cout << x() << std::endl;
}
</code></pre>
<p>The fact that this works the same way currently on all of the major \
implementations is another bonus.</p> <blockquote>
<p>But this is really the wrong place to have such discussions :)</p>
</blockquote>
<p>Yeah, I agree that whatever happens, I think this should probably be a Core issue \
that we should switch this discussion over to the ISO mailing \
list.</p></div></div></blockquote><div>Sounds reasonable to me.</div><div> \
</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px \
solid rgb(204,204,204);padding-left:1ex"><div><div> <div>Thanks,
<div><br>
</div>
<div>Daisy 🌼</div>
</div>
<br>
<p>On October 27, 2020 at 1:00:31 PM, Richard Smith (<a \
href="mailto:richard@metafoo.co.uk" target="_blank">richard@metafoo.co.uk</a>) \
wrote:</p> <blockquote type="cite"><span>
<div>
<div></div>
<div>
<div dir="ltr">
<div dir="ltr">On Tue, 27 Oct 2020 at 12:11, Arthur O'Dwyer <<a \
href="mailto:arthur.j.odwyer@gmail.com" \
target="_blank">arthur.j.odwyer@gmail.com</a>> wrote:<br> </div>
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid \
rgb(204,204,204);padding-left:1ex"> <div dir="ltr">On Mon, Oct 26, 2020 at 8:14 PM \
Richard Smith via cfe-dev <<a href="mailto:cfe-dev@lists.llvm.org" \
target="_blank">cfe-dev@lists.llvm.org</a>> wrote:<br> > On Mon, 26 Oct 2020 at \
16:57, Lewis, Cannada via cfe-dev <<a href="mailto:cfe-dev@lists.llvm.org" \
target="_blank">cfe-dev@lists.llvm.org</a>> wrote:<br> >><br>
>> I'm not a standards reading expert but does<br>
>><br>
>> Note 7:<br>
>> An id-expression that is not an odr-use refers to the original entity, never \
to a member of the closure type.<br> >> However, such an id-expression can \
still cause the implicit capture of the entity.<br> >> — end note<br>
>><br>
>> From the text imply that the program <a href="https://godbolt.org/z/feKxdK" \
target="_blank"> https://godbolt.org/z/feKxdK</a> is actually implementation defined? \
Or does gcc have a bug here?<br> ><br>
> GCC has a bug, according to the standard wording. The mention of myfoo does not \
constitute an odr-use, so is not rewritten to use the capture. Clang's behavior \
is correct per the standard wording.<br> ><br>
> The standard rule is certainly surprising in this particular case. I think the \
rule in question is driven by the desire for adding a capture-default to a valid \
lambda to not change its meaning. For example: <a href="https://godbolt.org/z/nrWsvj" \
target="_blank">https://godbolt.org/z/nrWsvj</a><br> <br>
Hmm. It's insane that you can use local variable `x` inside a lambda that \
doesn't capture anything; I wasn't aware of that.</div> </blockquote>
<div><br>
</div>
<div>Aside: "insane" (here) and "crazy" (below) are somewhat more \
charged language than we'd prefer here.</div> <div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid \
rgb(204,204,204);padding-left:1ex"> <div dir="ltr">If the formal wording really says \
you can do that, then that seems like a formal-wording bug to me.</div> </blockquote>
<div><br>
</div>
<div>It's entirely intentional, though the case that was being thought about at \
the time was primarily local constant non-references, not local references (though \
the same rules apply to both).</div> <div><br>
</div>
<div>void f() {</div>
<div> constexpr int k = 5;</div>
<div> [] { int arr[k]; };</div>
<div>}</div>
<div><br>
</div>
<div>... should obviously be valid. And more broadly, the rule is that anything that \
you can name without an odr-use is valid. And this isn't at all special to \
lambdas; the same thing happens with all nested constructs:</div> <div><br>
</div>
<div>int n;</div>
<div>void g() {</div>
<div> const int q = 6;</div>
<div> int &r = n;</div>
<div> constexpr int *p = &n;</div>
<div> struct A {</div>
<div> void f() {</div>
<div> int arr[q]; // ok</div>
<div> r = *p + 1; // ok</div>
<div> }</div>
<div> };</div>
<div>}</div>
<div><br>
</div>
<div><a href="https://godbolt.org/z/avG7qx" \
target="_blank">https://godbolt.org/z/avG7qx</a><br> </div>
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid \
rgb(204,204,204);padding-left:1ex"> <div dir="ltr">
<div>GCC and Clang disagree in a different way about whether it's okay to \
dereference a <i>pointer</i> inside a lambda without capturing it; here it's GCC \
that is doing the crazy thing, and Clang that is reasonably refusing to compile a \
reference to `x` when `x` hasn't been captured.</div> <div><a \
href="https://godbolt.org/z/bjbr3f" \
target="_blank">https://godbolt.org/z/bjbr3f</a></div> </div>
</blockquote>
<div><br>
</div>
<div>Yeah, GCC's behavior is not in line with the language rules in the pointer \
case. I'd guess they're using a more-liberal evaluation mode when evaluating \
the initializer of the reference, rather than using the strict constant initializer \
rules, and that more-liberal mode permits them to read the initializer values of all \
const-qualified variables, not only the ones that are usable in constant expressions. \
(Clang has a more-liberal mode too, and I'd expect we also have bugs where a \
conforming program can tell the difference.)</div>
<div><br>
</div>
<div>The rules in question are approximately equivalent to: if a variable of const \
integral or enumeration type, or of reference type, can be constexpr, then it is \
implicitly constexpr. And constexpr variables can be used from anywhere, without \
triggering an odr-use, if only their value, not their address, is used. (We \
don't actually say such things are implicitly constexpr, and there are corner \
cases where the outcome is different -- in particular, for static data members, where \
constexpr implies inline but const does not. But "implicitly constexpr" is \
how they behave.)</div> <div><br>
</div>
<div>If you change your example to declare the pointer to be constexpr then the two \
compilers agree again.</div> <div><br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid \
rgb(204,204,204);padding-left:1ex"> <div dir="ltr">The rules about when compilers are \
able to "constant-fold" away variables that otherwise would need to be \
captured, strikes me as similar in spirit to the rules about when compilers are able \
to "constant-fold" away complicated expressions in constant expressions. \
Some compilers seem to be trying to be "helpful" by letting the \
optimizer's knowledge leak into the front-end, and some are following the same \
rules as one's "head compiler" would.<br> </div>
</blockquote>
<div><br>
</div>
<div>I think this is at least not the proximal cause in the standard-conforming case \
-- correctly implementing the odr-use rules in the Clang frontend was a non-trivial \
effort and wasn't related to constant folding / optimizer knowledge leaking \
through. But I do think the standard rules here probably originally came from \
looking at what implementations at the time happened to do and writing that down (eg, \
use of a const static data member with no definition is OK, because implementations \
happen to not emit a reference to the symbol), and complexity has built up from that \
point. I'm not sure where the references-are-implicitly-constexpr part came from, \
but the const-ints-are-implicitly-constexpr part is a backwards-compatibility concern \
inherited from C++98.</div> <div><br>
</div>
<div>From a language design perspective, I think it would have made a lot more sense \
if we'd said that constexpr variables are implicitly static (so the question of \
capture or of use from other scopes can be answered trivially), and deprecated the \
"implicit constexpr" behavior for const integral/enumeration variables and \
references. Where we've ended up isn't a great compromise. But this is really \
the wrong place to have such discussions :)</div> </div>
</div>
</div>
</div>
</span></blockquote>
</div>
<div class="gmail-m_-8426244070307930285bloop_markdown">
<p></p>
</div>
</div>
</blockquote></div></div>
[Attachment #6 (text/plain)]
_______________________________________________
cfe-dev mailing list
cfe-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic