[prev in list] [next in list] [prev in thread] [next in thread] 

List:       r-devel
Subject:    Re: [Rd] issue with parallel package
From:       Simon Urbanek <simon.urbanek () r-project ! org>
Date:       2014-05-22 4:18:36
Message-ID: 41943519-EBEC-4F02-8FE1-C968D35922E3 () r-project ! org
[Download RAW message or body]


On May 21, 2014, at 1:17 PM, Josef Leydold <josef.leydold@wu.ac.at> wrote:

> Dear maintainers of the parallel package,
> 
> I ran into an issue with the parallel package in R-3.1.0.
> 
> The following code prints the message "NULL!" quite a lot.
> 
> library(parallel)
> for (n in 1:1000) {
> p <- mcparallel(sqrt(n))
> res <- mccollect(p, wait=FALSE, timeout=1000)
> mccollect(p)
> if (is.null(res)) cat(n,"  NULL!\n")
> }
> 
> 
> It does not happen in R-3.0.3.
> (To be exact, I have checked out R-devel revision r65130
> for my current tests.)
> 
> I found that this behavior is introduced in revision r65143.
> 
> When I insert
> Sys.sleep(0.1)
> in the loop then this behavior disappears in R-3.1.0.
> 
> There is also another difference between R-3.1.0 and R-3.0.3:
> When the line
> mccollect(p)
> is replaced by
> mccollect(p, wait=FALSE, timeout=1000)
> then in R-3.1.0 some child processes are not terminated and some
> pipes remain open. This does not happen in R-3.0.3.
> 
> Remark:
> If the line
> mccollect(p)
> is omitted at all then "NULL!" is never printed but
> R aborts after 510 iterations of the loop with the message
> "Error in mcfork(detached) : unable to create a pipe".
> Then none of the child processes is terminated and
> number of possible open pipes is exhausted.
> 
> Is this behavior intended or a bug?
> 

As noted in the follow-up this is absolutely expected. wait=FALSE merely checks if \
the result is available - if it's not at that point then NULL is returned - that's \
what you see and is exactly what one would expect. Analogously, if you don't collect \
the results then the child processes will be still hanging around waiting for you to \
collect the results and therefore you'll run out of descriptors. The fact that it \
worked in 3.0.3 is just a coincidence that it used more time managing the children \
and checking their status before you got to your attempted collect so the result was \
more likely to be already available. Because 3.1.0 is more efficient (it no longer \
manages children manually but rather uses signal-based cleanup), it comes faster to \
the collect point and the results may not be available yet.

Cheers,
Simon



> 
> > version
> _                           
> platform       x86_64-pc-linux-gnu         
> arch           x86_64                      
> os             linux-gnu                   
> system         x86_64, linux-gnu           
> status                                     
> major          3                           
> minor          1.0                         
> year           2014                        
> month          04                          
> day            10                          
> svn rev        65387                       
> language       R                           
> version.string R version 3.1.0 (2014-04-10)
> nickname       Spring Dance                
> 
> 
> best whishes
> Josef
> 
> -- 
> 
> 
> -----------------------------------------------------------------------------
> 
> Josef Leydold
> 
> Institute for Statistics and Mathematics
> WU (Vienna University of Economics and Business)
> 
> Welthandelsplatz 1, Building D4, 1020 Vienna, Austria
> 
> Phone +43 1 31336 4695  FAX +43 1 31336 904695  email josef.leydold@wu.ac.at
> 
> -----------------------------------------------------------------------------
> Alles Unglueck kam daher, dass die Denkenden nicht mehr handeln konnten,
> und die Handelnden keine Zeit mehr fanden zu denken.       (Marlen Haushofer)
> 
> ______________________________________________
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
> 

______________________________________________
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel


[prev in list] [next in list] [prev in thread] [next in thread]