[prev in list] [next in list] [prev in thread] [next in thread] 

List:       fedora-devel-list
Subject:    Re: Fedora 32 System-Wide Change proposal: Build Python 3 to statically link with libpython3.8.a for
From:       Charalampos Stratakis <cstratak () redhat ! com>
Date:       2019-11-15 12:42:38
Message-ID: 1822893449.12247392.1573821758894.JavaMail.zimbra () redhat ! com
[Download RAW message or body]



----- Original Message -----
> From: "Victor Stinner" <vstinner@redhat.com>
> To: devel@lists.fedoraproject.org
> Sent: Friday, November 15, 2019 10:21:44 AM
> Subject: Re: Fedora 32 System-Wide Change proposal: Build Python 3 to statically \
> link with libpython3.8.a for better performance
> 
> Hi Jan,
> 
> With the helper of Florian Weimer and Charalampos Stratakis, we also agreed
> to test this flag in priority. I understood that it disables the LD_PRELOAD
> feature: it's no longer possible to override symbols in libpython with
> LD_PRELOAD. Thanks to that, the compiler can avoid PLT indirection for
> function calls and can inline more function functions in libpython. I'm
> talking about a function call from libpython to libpython: something which
> is very common in python. Basically, almost all function calls are calls
> from libpython to libpython.
> 
> I'm impressed. Thanks to -fno-semantic-interposition, I get the same speedup
> on a dynamically linked Python (libpython) compared to statically linked
> Python!
> 
> Yesterday, I tried on a vanilla Python compiled manually:
> 
> ./configure --enable-optimizations --with-lto --enable-shared
> CFLAGS="-fno-semantic-interposition" LDFLAGS="-fno-semantic-interposition"
> 
> I saw the same speed up than avoiding --enable-shared. Today I validated this
> result using the RPM generated by Charalampos's PR:
> https://src.fedoraproject.org/rpms/python38/pull-request/53
> 
> In short, https://fedoraproject.org/wiki/Changes/PythonStaticSpeedup is
> useless: there is no need to modify Python to statically link it to
> libpython. We can keep the dynamically library libpython and keep Python
> dynamically linked to it. We only need to pass -fno-semantic-interposition
> to compiler and linker flags when building Python!
> 
> I'm not sure if we need a Fedora change just for a compiler flag. Again, the
> only drawback is that we will no longer be able to override a symbol using
> LD_PRELOAD. Honestly, I never did that. I don't see any use case for that.
> But I used LD_PRELOAD on the libc multiple times to mock the system clock
> for example.
> 
> If someone really needs LD_PRELOAD, it's quite easy to build a custom Python
> without -fno-semantic-interposition.
> 
> Victor
> _______________________________________________
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-leave@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
> 

Thanks Victor for running the benchmarks.

The change will be withdrawn and another self-contained one will be created. I think \
not being able to override symbols on the system python is a better tradeoff than the \
size/speed and possible incompatibilities.

Side note: the list of packages that still link to libpython without embedding the \
interpreter will be used to mass file bugs in order to unlink them.

-- 
Regards,

Charalampos Stratakis
Software Engineer
Python Maintenance Team, Red Hat
_______________________________________________
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org



[prev in list] [next in list] [prev in thread] [next in thread]