[prev in list] [next in list] [prev in thread] [next in thread] 

List:       fedora-devel-list
Subject:    Re: Fedora 32 System-Wide Change proposal: Build Python 3 to statically link with libpython3.8.a for
From:       John Reiser <jreiser () bitwagon ! com>
Date:       2019-11-10 14:45:16
Message-ID: 11ec33f1-f122-a012-07bc-08f5e8703b85 () bitwagon ! com
[Download RAW message or body]

On 11/5/19, Ben Cotton wrote:
> https://fedoraproject.org/wiki/Changes/PythonStaticSpeedup
> 
> == Summary ==
> Python 3 traditionally in Fedora was built with a shared library
> libpython3.?.so and the final binary was dynamically linked against
> that shared library. This change is about creating the static library
> and linking the final python3 binary against it, as it provides
> significant performance improvement, up to 27% depending on the
> workload. The static library will not be shipped. The shared library
> will continue to exist in a separate subpackage. In essence, python3
> will no longer depend on libpython.
   <<snip>>

There are alternatives that provide gradations in the tradeoffs.

0) Include _Py_UnixMain in libpython3.POINTVER.so, also set ElfXX_Ehdr.e_entry
and include enough -startfiles so that execve(libpython3.POINTVER.so, ...)
will act as if execve(python, ...).  Compare execve("/lib64/libc.so.6", ...)
which prints the credits for glibc.  Then python3 and libpython3.POINTVER.so
can be hardlinked or symlinked.  This removes one DT_NEEDED from the
startup of python3.

1) Do not flag python3 and libpython3.POINTVER.so with DT_BIND_NOW or DF_BIND_NOW.
This removes the need to perform relocation processing for every slot in the PLT
at process startup.

2) Use -Wl,-Bsymbolic during the build (static bind) of libpython3.POINTVER.so.
This removes all intra-library symbolic relocations (hence PLT slots) at the cost
of also removing the ability to override (interpose) them.

3) Compile and build libpython3.POINTVER.so as ET_EXEC (without -fPIC, without
-shared, without -fPIE), static bind with
      -Wl,-Ttext-segment=$(< /proc/sys/vm/mmap-min-addr)
to put the library below the pages of any default ET_EXEC, static bind with
--export-dynamic (or --dynamic-list=) to make visible all Python primitives,
and enhance the dynamic linker ld-linux to dlopen(ET_EXEC, ...) as if ET_DYN
but OR-in MAP_FIXED when mmap() of PT_LOAD.  The dynamic linker can be
tricked today by changing ElfXX_Ehdr.e_type from ET_EXEC to ET_DYN,
as long as the linux kernel honors the hint of mmap(non_zero, ...)
without MAP_FIXED.

Today's /lib64/libpython3.7m.so.1.0 occupies about 3.4 MB of pages, which fits
between default mmap-min-addr of 64K and default -Ttext-segment of 4M.
_______________________________________________
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

[prev in list] [next in list] [prev in thread] [next in thread]