[prev in list] [next in list] [prev in thread] [next in thread]
List: r-sig-mac
Subject: Re: [R-SIG-Mac] R 4.0 toolchain
From: Patrick Schratz <patrick.schratz () gmail ! com>
Date: 2020-04-02 7:34:14
Message-ID: 8ab7658b-67de-4fd5-8e91-d5b2126a970d () Spark
[Download RAW message or body]
Thanks Simon.
Branching off here to discuss the toolchain in general.
I see the danger posting not 100% correct scripts and in blog posts in general \
because they might get outdated and not always reflect the official documentation. On \
the other side: What is the middle way? Arguably installing a robust toolchain (on \
Catalina especially but for macOS in general) is not just done with the installation \
of the signed package. Most normal users are not aware about all the little \
specialties on macOS (missing openMP enabled compiler, SDK issues) and why \
adjustments might be needed.
Even though the idea is to require as less manual configuration as possible, this is \
not always practical in the end (unless all maintainers agree on configuring their \
package without the need of an openMP enabled C compiler for example). And I also \
think that "installing from source" is not a dev-only thing.
1. By using the Apple internal clang as the default, will maintainers be indirectly \
forced to remove the dependency on an openMP enabled compiler with respect to getting \
source installations on macOS working? 2. Can there be an official section easily \
accessible for users which recommends which compiler to use (and how to set the \
Makevars) if users really need/want to do so? a. Regarding openMP enabled compilers: \
llvm is probably the one with the least differences to Apples clang. It can be \
installed via brew (brew install llvm) and then linked in Makevars. Would that be an \
alternative? 3. SDK: I did not wanted to induce that setting the SDK vars is needed \
for the CRAN build chain (maybe we got each other wrong here). However, as of now, \
there are real issues with SDK 10.15 that result in errors that are really hard to \
track down or even fall under the radar when testing (if someone does not run \
Catalina). Therefore I am linking locally against SDK 10.13 since some days and that \
resolved many issues (see again the Rcpp issue and the igraph issue). Since Catalina \
is the latest release and many people run it there should be test/instructions \
somewhere on mac.rproject that deal with this/suggest not to use SDK 10.15 for R IMO. \
Maybe this issue deserves a special thread.
Cheers, Patrick
On 1. Apr 2020, 22:29 +0200, Simon Urbanek <simon.urbanek@r-project.org>, wrote:
> Patrick,
>
> firstly, please don't post such things - they are often wrong (as are parts of what \
> you included it this e-mail) and it's impossible for us to track all blogs that \
> have incorrect advice. Unfortunately, a lot of the issues we see out there are due \
> to people finding bad advice and using it. Ideally, it should be sufficient to \
> point to the official R documentation since that is the canonical source. If you \
> post a blog, it should point to the official documentation. If that documentation \
> is missing or not covering a need, please post here so we can fix it. It is more \
> important that the official source is complete and much better for the community \
> than to have random tidbits that may or may not be correct floating around.
> As for the script, please do not use R tar balls - first, you're picking the wrong \
> one (there is no R-devel for el-capitan. R 4.0.0 pre-releases are in R-4-0-branch \
> and they are built for high sierra). Second, the tar ball doesn't have the \
> entitlements enabled so it doesn't match the release. Please use the signed R \
> package. This may change, but with the complications of the Apple notarization \
> process that's what it is today.
> Then there is the question of the SDK. What are you trying to setup here? What you \
> posted here has nothing to do with the CRAN setup. You don't need 10.13 SDK to \
> compile igraph, it compiles just fine using stock command line tools on High Sierra \
> which is what we use on CRAN. You can, of course, pick any SDK you like, but please \
> don't tell people that's what they should do. One of the most annoying issues is \
> that people are recommending messing with Makevars which is incredibly fragile and \
> will cause things to break left and right with any update. It's ok to do that for a \
> very special purpose, but no regular user should have Makevars in their home.
> So if you really want to serve the community, please suggest improvements to the \
> official documentation and post them here. If you have special needs, ask about \
> them here.
> Thanks,
> Simon
>
>
>
> > On 1/04/2020, at 9:47 PM, Patrick Schratz <patrick.schratz@gmail.com> wrote:
> >
> > Thanks Simon,
> >
> > This simplifies things a lot and clears up the process. While the instructions on \
> > https://mac.r-project.org/ are more clear now I think there is still \
> > simplification needed for the install process and custom modifications that are \
> > needed. This not only applies to the dev page but finally also to production end \
> > users if the SDK 10.15 issues (see below) persists.
> > I've scripted all installation steps (R-devel, gfortran, SDK 10.13) and will \
> > probably write a blog post about it to make the R community more aware about it. \
> > Let me know if anything is wrong about it - otherwise I'll use the code below in \
> > the blog post.
> > # install R-devel
> > wget https://mac.r-project.org/el-capitan/R-devel/R-devel-el-capitan-sa-x86_64.tar.gz
> > sudo tar fvxz R*.tar.gz -C /
> > rm R-devel-el-capitan-sa-x86_64.tar.gz
> >
> > # install gfortran
> > wget https://github.com/fxcoudert/gfortran-for-macOS/releases/download/8.2/gfortran-8.2-Mojave.dmg
> > sudo hdiutil attach gfortran*.dmg
> > sudo installer -package /Volumes/gfortran*/gfortran*/gfortran*.pkg -target /
> > sudo hdiutil detach /Volumes/gfortran-8.2-Mojave
> >
> > # install SDK10.13
> > wget https://github.com/phracker/MacOSX-SDKs/releases/download/10.15/MacOSX10.13.sdk.tar.xz
> > tar fvxz MacOSX10.13.sdk.tar.xz
> > sudo mv MacOSX10.13.sdk /Library/Developer/CommandLineTools/SDKs/
> > rm -rf MacOSX10.13*
> >
> > In addition people need to set custom CFLAGS in ~/.R/Makevars to actually use it:
> >
> > CFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.13.sdk
> > CCFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.13.sdk
> > CXXFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.13.sdk
> > CPPFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.13.sdk
> >
> > In addition there really seem to be major problems regarding the 10.15 SDK. For \
> > example, {igraph} which is a package on which lots of other packages depend, \
> > cannot be installed from source with SDK 10.15 but works with SDK 10.13.
> > Patrick
> > On 1. Apr 2020, 09:38 +0200, Simon Urbanek <simon.urbanek@r-project.org>, wrote:
> > > Hervé,
> > >
> > >
> > > > On 1/04/2020, at 6:19 PM, Hervé Pagès <hpages@fredhutch.org> wrote:
> > > >
> > > > Thanks Simon. A couple of days ago we've started to use the R 4.0.0 alpha \
> > > > build from https://mac.r-project.org/ for the Bioconductor build system. \
> > > > Bioconductor packages depend on thousands of CRAN packages and one thing that \
> > > > makes it hard for us and for our users to build/check Bioconductor packages \
> > > > at the moment is the absence of Mac binary packages for R 4.0.0 on CRAN. Do \
> > > > you have an estimate of when they will become available?
> > >
> > > Both the packages and R are now available. I always recommend using the \
> > > Mac-master https://mac.R-project.org as "mirror" since that one is most \
> > > up-to-date.
> > >
> > > > Another question is where they are going to be hosted on CRAN. We need to \
> > > > follow CRAN's layout for Bioconductor package repositories.
> > >
> > >
> > > We have removed the extra layer, the package type is back to "mac.binary", so \
> > > bin/macosx/contrib is the location.
> > > Cheers,
> > > Simon
> > >
> > >
> > >
> > > > Thanks!
> > > > H.
> > > >
> > > >
> > > > On 3/31/20 21:27, Simon Urbanek wrote:
> > > > > Dear Mac users,
> > > > > R 4.0.0 will be using an entirely new toolchain, entirely new build system \
> > > > > on entirely new macOS version and hardware. Therefore I would like to ask \
> > > > > you kindly to test the binaries from \
> > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__mac.R-2Dproject.org&d=D \
> > > > > wIFAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA \
> > > > > &m=-PNBcsZhHb1MLy6fZakmLP6WDLMwL-VigJVzeFtPxOI&s=FJYxweW9WiMXbNMPpMM0bHXy020Gx9P-dHMrWmt7YzE&e=
> > > > > before the release as much as you can. Raising any issues after the \
> > > > > release is too late! So please, please, test the pre-releases. Report any \
> > > > > issues either directly to me or this mailing list. The nightly builds are \
> > > > > signed, but not necessarily notarized. However, the build fulfils Apple's \
> > > > > conditions and is known to pass notarization (in fact the the package \
> > > > > available for download today is actually notarized) so it should be a good \
> > > > > test for the release which will be notarized and should work on Catalina. \
> > > > > For those that want to replicate our setup - technical details: we are now \
> > > > > building with macOS 10.13 (High Sierra) as target (i.e. the oldest \
> > > > > supported system), regular Apple Xcode/command line tools and GNU Fortran \
> > > > > 8.2. R builds are running on macOS 10.15 (Catalina) with Xcode 11.4 using \
> > > > > macOS 10.13 target. Packages are built on macOS 10.13 VMs with just Apple \
> > > > > command line tools (this should make it easy to replicate the setup using \
> > > > > Travis, for example). All 3rd party libraries that CRAN uses are available \
> > > > > in https://urldefense.proofpoint.com/v2/url?u=http-3A__mac.r-2Dproject.org_l \
> > > > > ibs-2D4_&d=DwIFAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvS \
> > > > > XAJJKaaPhzWA&m=-PNBcsZhHb1MLy6fZakmLP6WDLMwL-VigJVzeFtPxOI&s=3ZbcXBticg_8MfXltGKdQ9SFNqqeLqqwK_pUhEet7QY&e=
> > > > > The new R build system is in
> > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__svn.r-2Dproject.org_R-2 \
> > > > > Ddev-2Dweb_trunk_QA_Simon_R4&d=DwIFAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvim \
> > > > > eWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=-PNBcsZhHb1MLy6fZakmLP6WDLMwL-VigJVzeFtPxOI&s=Mmt7_pVM5aUDR3hz93-e-PtadG7wSYpaZe8Ayi5Pv4I&e=
> > > > > Packages build system has not changed and is in
> > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__svn.r-2Dproject.org_R-2 \
> > > > > Ddev-2Dweb_trunk_QA_Simon_packages&d=DwIFAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3 \
> > > > > XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=-PNBcsZhHb1MLy6fZakmLP6WDLMwL-VigJVzeFtPxOI&s=eId6n08LVeIAMhMXe3jvLFKn9QGi4_by8owFreDDc84&e=
> > > > > We also plan to have a mac-builder available with similar function as the \
> > > > > win-builder where pre-submission tests can be performed and potentially a \
> > > > > Travis template. Please test R pre-releases and provide feedback!
> > > > > Thanks,
> > > > > Simon
> > > > > _______________________________________________
> > > > > R-SIG-Mac mailing list
> > > > > R-SIG-Mac@r-project.org
> > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_li \
> > > > > stinfo_r-2Dsig-2Dmac&d=DwIFAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_ \
> > > > > wJYbW0WYiZvSXAJJKaaPhzWA&m=-PNBcsZhHb1MLy6fZakmLP6WDLMwL-VigJVzeFtPxOI&s=ubqivtpb1OZ2I0MEbVvGlGDK-tfti3DPtIv4LQ-Ks6A&e=
> > > > >
> > > >
> > > > --
> > > > Hervé Pagès
> > > >
> > > > Program in Computational Biology
> > > > Division of Public Health Sciences
> > > > Fred Hutchinson Cancer Research Center
> > > > 1100 Fairview Ave. N, M1-B514
> > > > P.O. Box 19024
> > > > Seattle, WA 98109-1024
> > > >
> > > > E-mail: hpages@fredhutch.org
> > > > Phone: (206) 667-5791
> > > > Fax: (206) 667-1319
> > > >
> > > > _______________________________________________
> > > > R-SIG-Mac mailing list
> > > > R-SIG-Mac@r-project.org
> > > > https://stat.ethz.ch/mailman/listinfo/r-sig-mac
> > >
> > > _______________________________________________
> > > R-SIG-Mac mailing list
> > > R-SIG-Mac@r-project.org
> > > https://stat.ethz.ch/mailman/listinfo/r-sig-mac
> >
> > [[alternative HTML version deleted]]
> >
> > _______________________________________________
> > R-SIG-Mac mailing list
> > R-SIG-Mac@r-project.org
> > https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>
[[alternative HTML version deleted]]
_______________________________________________
R-SIG-Mac mailing list
R-SIG-Mac@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-mac
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic