'Re: [Bioc-devel] workflow building issues on mac and windows'

[prev in list] [next in list] [prev in thread] [next in thread] 

List:       bioc-devel
Subject:    Re: [Bioc-devel] workflow building issues on mac and windows
From:       Andrzej_Oleś <andrzej.oles () gmail ! com>
Date:       2017-08-31 21:25:08
Message-ID: CAH7mKkjMxkY4iB9pyOgOTsCYQOggTAVWG2Cgeq87gNHjQdFk=Q () mail ! gmail ! com
[Download RAW message or body]

Hi Leo,

first of all, many thanks for your efforts in troubleshooting workflow
build issues and the detailed description of your findings. Also,
congratulations on the successful publication of your new workflow on the
Bioconductor website!

We are now beta testing a new workflow build engine which enables some
features not possible to achieve in the previous approach, such as
cross-references. The goal is to properly render the output for the BioC
website directly from the Rmd file authored for
BiocWorkflowTools::f1000_article without any special tweaks or hacks. This
new engine is enabled by adding the special '.html_output' token file in
the vignettes/ dir. Ideally, you should be able to get decent output from
the original Rmd file which you used for the F1000 Research submission
without any manual Bioc-specific modifications. Note that in principle you
could even leave your primary document output format as
`BiocWorkflowTools::f1000_article`. This is because the workflows for the
website are build through a call to `rmarkdown::render` by specifying a
custom render format, so the output format set in the document header is
discarded anyway; it is important only for the vignette included in the
package tarball. But it's also fine to use either of the formats provided
by BiocStyle.

It shouldn't be necessary to use the captioner package anymore, as figure
numbering is now handled by bookdown. Gosia used it in cytofWorkflow before
I enabled the new html engine for her workflow, which I did only yesterday.

Re the specific builder issues: I've deleted the /tmp/udcCache dir on Mac,
but it didn't help. I'm not sure yet what could be the problem there. The
citeproc conversion error on Windows might be because of the older pandoc
version (1.17.2), will look into this. The convert warnings appear because
the new BiocStyle formats set the `knitr::opts_chunk(crop=TRUE)` to crop
out excessive white space around plots, will try to sanitize this a little
bit too.

Cheers,
Andrzej

On Thu, Aug 31, 2017 at 8:15 PM, Leonardo Collado Torres <lcollado@jhu.edu>
wrote:

> Hi,
>
> I recently got a workflow accepted and I've been trying to get the
> workflow builder to successfully complete on Mac and Windows which is
> based on bioc-release. Note that the workflow did build properly on
> those operating systems using the SPB (bioc-devel).
>
>
> ## Linux: working
>
> After looking at
> https://hedgehog.fhcrc.org/bioconductor/trunk/madman/
> workflows/cytofWorkflow
> and https://hedgehog.fhcrc.org/bioconductor/trunk/madman/
> workflows/rnaseqGene
> I was able to:
>
> (1) get the figure references and captions working using captioner
> (2) actually get the figures to show
> (3) use the new layout (thanks to vignettes/.html_output)
>
> Since the build completes on Linux,
> http://bioconductor.org/help/workflows/recountWorkflow/ is now live.
> Only the links to the Mac and Windows binaries fail.
>
>
> ## Mac: could be a permissions issue
>
> The issue with Mac is (details at
> http://docbuilder.bioconductor.org:8080/job/recountWorkflow/label=
> vagrantmac/9/console)
> is that I eventually run into this error:
>
> Quitting from lines 735-747 (recount-workflow.Rmd)
> Error: processing vignette 'recount-workflow.Rmd' failed with diagnostics:
> UCSC library operation failed
> Execution halted
>
>
> The referenced lines break because of:
>
> regions <- expressed_regions("SRP045638", "chr21", cutoff = 5L,
>     maxClusterGap = 3000L)
>
> This code uses rtracklayer::import.bw() with a URL that gets
> forwarded. I made sure that I am requiring the latest bioc-release
> rtracklayer in the description file, so this leads me to think that
> this issue is a repeat of
> https://stat.ethz.ch/pipermail/bioc-devel/2016-August/009599.html
> where the solution involved changing some permissions. Dan stated in
> that thread (3rd email): "Actually it looks like it was a permissions
> issue with the directory /tmp/udcCache. I removed this directory (as
> superuser) and that error no longer happens."
>
> Does this sound like something that could be happening in the Mac builder?
>
>
>
> ## Windows: figure and pandoc-citeproc issues
>
>
> In Windows, I see warnings like this:
>
> Invalid Parameter - -trim
> Warning: running command 'C:\Windows\system32\cmd.exe /c convert
> "exondeanalysis1-1.png" -trim "exondeanalysis1-1.png"' had status 4
> Warning in shell(paste(c(cmd, args), collapse = " ")) :
>   'convert "exondeanalysis1-1.png" -trim "exondeanalysis1-1.png"'
> execution failed with error code 4
>
> Details at http://docbuilder.bioconductor.org:8080/job/
> recountWorkflow/label=winbuilder1/9/console.
> I originally thought that it was related to the figure paths, which is
> why I added this knitr code:
>
> knitr::opts_chunk$set(fig.path = "")
>
> But that didn't resolve the issue. I get these warnings only with
> BiocStyle::html_document2 and not with BiocStyle::html_document
> (http://docbuilder.bioconductor.org:8080/job/recountWorkflow/label=
> winbuilder1/7/console
> using v0.99.28).
>
> Ultimately, the warnings might not matter. Though I don't see them on
> the build reports for other workflows (cytofWorkflow, rnaseqGene).
>
> The Windows builds fail (with BiocStyle::html_document2 or
> BiocStyle::html_document) with error messages like this:
>
> pandoc.exe: Error running filter pandoc-citeproc Filter returned error
> status 1073807366 Warning: running command
> '"C:/Progra~2/Pandoc/pandoc" +RTS -K512m -RTS recount-workflow.utf8.md
> --to html --from
> markdown+autolink_bare_uris+ascii_identifiers+tex_math_single_backslash
> --output recount-workflow.html --smart --email-obfuscation none
> --self-contained --standalone --section-divs --table-of-contents
> --toc-depth 3 --template
> "C:\Windows\TEMP\RtmpchKz3E/BiocStyle/template.html" --no-highlight
> --variable highlightjs=1 --number-sections --css
> "C:\PROGRA~1\R\R-34~1.0\library\BIOCST~1\RESOUR~1\html\BIOCON~2.CSS"
> --variable "theme:bootstrap" --include-in-header
> "C:\Windows\TEMP\RtmpchKz3E\rmarkdown-str3ec7c88cea.html" --mathjax
> --variable "mathjax-url:https://mathjax.rstudio.com/latest/MathJax.js?
> config=TeX-AMS-MML_HTMLorMML"
> --bibliography recount-workflow.bib --filter pandoc-citeproc' had
> status 83 Error: processing vignette 'recount-workflow.Rmd' failed
> with diagnostics: pandoc document conversion failed with error 83
> Execution halted
>
> (from http://docbuilder.bioconductor.org:8080/job/recountWorkflow/label=
> winbuilder1/9/console)
>
>
> Googling the error pointed me towards having weird symbols on the .bib
> file. So I removed some accents and the like, but I still get this
> error. I am using a csl file for the bibliography and like I said,
> this works with the SPB whose last build report was
> http://bioconductor.org/spb_reports/recountWorkflow_
> buildreport_20170807114418.html
> (v0.99.18). There's not much different between that version and
> 0.99.30 as you can see at
> https://github.com/LieberInstitute/recountWorkflow/commits/master **.
>
> The Linux machine has pandoc 1.19.2.1 installed
> (http://bioconductor.org/help/workflows/recountWorkflow/#
> session-information)
> and I guess that the Windows one has the same version. But maybe that
> could be the issue. BiocStyle 2.5.15 is installed in the Linux machine
> too (Bioconductor/BiocStyle@2a1ba75) which is newer than the latest
> bioc-release (2.4.1), but again, I assume that the Windows machine has
> the same version.
>
>
>
> Anyhow, I haven't been able to fix these issues and was wondering if
> anyone else had ideas that could resolve them. Regardless, I'm happy
> that I got http://bioconductor.org/help/workflows/recountWorkflow/ up
> and looking good ^^. It's just the links to the Mac/Windows
> binaries...
>
>
> Thank you,
> Leo
>
>
>
> ** I've been pushing changes via svn, I know that workflows are not
> part of the git transition. I just keep everything in sync manually
> with the GitHub repo.
>
>
> Leonardo Collado Torres, Ph. D., Data Scientist
> Lieber Institute for Brain Development
> 855 N Wolfe St, Suite 300
> Baltimore, MD 21205
> Website: http://lcolladotor.github.io
>
> _______________________________________________
> Bioc-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>

	[[alternative HTML version deleted]]

_______________________________________________
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel
[prev in list] [next in list] [prev in thread] [next in thread] 