'Re: [R-SIG-Mac] R run time question'

[prev in list] [next in list] [prev in thread] [next in thread] 

List:       r-sig-mac
Subject:    Re: [R-SIG-Mac] R run time question
From:       Simon Urbanek <simon.urbanek () r-project ! org>
Date:       2011-07-13 20:35:26
Message-ID: 97167CB0-AB4D-42E3-8D86-57C8405E1350 () r-project ! org
[Download RAW message or body]

On Jul 13, 2011, at 3:50 PM, Steven McKinney wrote:

> 
> 
> 
> > -----Original Message-----
> > From: r-sig-mac-bounces@r-project.org [mailto:r-sig-mac-bounces@r-project.org] On \
> >                 Behalf Of Wayne Gray
> > Sent: July-13-11 12:25 PM
> > To: r-sig-mac@r-project.org
> > Subject: [R-SIG-Mac] R run time question
> > 
> > All,
> > 
> > We are doing ANOVAs that take a long time (> 3 hrs) to run in 64-bit mode on an \
> > Intel MacPro with 8 gbytes and 2.66 GHZ Intel Core i7 machine.
> > 
> > We have just tried running these on our server where we don't care as much how \
> > long it takes. The server is a Mac "2 x 3 GHz Dual-Core Intel Xeon" with 13 GB \
> > 667 MHz DDR2 FB-DIMM. 
> > We have installed R "R 2.10.1 GUI 1.31-np Tiger build 32-bit (5538)" on the \
> > server.
> 
> Why are you installing a 32 bit R?  And why an older version of R?
> 
> Can you not install a 64 bit version of the latest R?
> 
> > 
> > The data.frame is composed of the following observations, factors, and numerals:
> > 
> > > str(e1jit41blkAll)
> > 'data.frame':	21648 obs. of  6 variables:
> > $ subjectid: Factor w/ 22 levels "1379","1744",..: 1 1 1 1 1 1 1 1 1 1 ...
> > $ cond     : Factor w/ 2 levels "Visual","Auditory": 1 1 1 1 1 1 1 1 1 1 ...
> > $ block    : Factor w/ 8 levels "2","3","4","5",..: 1 1 1 1 1 1 1 1 1 1 ...
> > $ cbtime   : Factor w/ 41 levels "1","2","3","4",..: 1 10 11 12 13 14 15 16 17 18 \
> > ... $ dirtime  : Ord.factor w/ 3 levels "Early"<"Middle"<..: 1 1 1 1 1 1 1 1 1 1 \
> > ... $ jitter   : num  7.78 7.33 5.56 5 6.11 ...
> > 
> > When we run the following ANOVA on this dataframe:
> > 
> > > anova.ALL.e1jit41.blkAll  <- with(e1jit41blkAll, aov(jitter ~ \
> > > cond*block*dirtime*cbtime +
> > error(subjectid/(block*dirtime*cbtime)), data = e1jit41blkAll))
> > 
> > We get the following feedback:
> > 
> > Error: cannot allocate vector of size 3.5 Gb
> 
> Running the ANOVA requires more than 4GB of RAM, which can not be done
> with 32 bit versions of software.  So you will need 64 bit R to do this analysis.
> 
> > 
> > > 
> > R(29720,0xa000d000) malloc: *** vm_allocate(size=3749089280) failed (error \
> > code=3) R(29720,0xa000d000) malloc: *** error: can't allocate region
> > R(29720,0xa000d000) malloc: *** set a breakpoint in szone_error to debug
> > R(29720,0xa000d000) malloc: *** vm_allocate(size=3749089280) failed (error \
> > code=3) R(29720,0xa000d000) malloc: *** error: can't allocate region
> > R(29720,0xa000d000) malloc: *** set a breakpoint in szone_error to debug
> > > 
> > 
> > So two categories of questions for all of you Mac-R wizards out there.
> > 
> > First category of question: can this analysis be run faster (and if so, how) on \
> > our laptop Intel Macs? 
> > Second category of question: is there anything that can be done to the Apple \
> > Server or to our analysis so that the analysis runs on our server?
> 
> What OS version is on your server?  If it's an old 32 bit version, upgrading the \
> server to a recent 64 bit OS will help.  Then install a recent 64 bit version of R.
> 

FWIW: you can run R in 64-bit on Tiger as well, you just have to disable Quartz and \
the GUI since Tiger doesn't have 64-bit Cocoa.

Cheers,
Simon

> If you can do this, you can better assess whether this analysis can be done 
> faster on the 8GB laptops.
> 
> R runs entirely in RAM, so if the analysis needs more than 8GB of RAM, you may
> be experiencing "swapping" on the laptops.  The operating system allocates
> as much memory as the job needs - if that amount of memory exceeds the amount
> of RAM you have, then the rest is allocated to "virtual memory" which is
> on the disk drive.  Swapping occurs when part of the virtual memory of your job
> that is on disk is needed - the OS has to copy a chunk of what is in RAM out to
> disk, so it can copy the other chunk on disk back into RAM.
> 
> Monitor the laptop with a shell command line process such as "top" 
> and you will be able to determine how much virtual memory your R job
> is using, and how many "swaps" are involved.  Swapping really slows things
> down as reading and writing from the disk is slow.  If your job is swapping
> on the laptop, but uses less than 13GB of memory in total, it will run faster
> on the server.  If the job is not swapping at all on the laptop,  it will not
> run appreciably faster on the server.
> 
> 
> HTH
> 
> Steve McKinney
> 
> 
> > 
> > Many thanks,
> > 
> > Wayne Gray
> > 
> > 	[[alternative HTML version deleted]]
> > 
> > _______________________________________________
> > R-SIG-Mac mailing list
> > R-SIG-Mac@r-project.org
> > https://stat.ethz.ch/mailman/listinfo/r-sig-mac
> 
> _______________________________________________
> R-SIG-Mac mailing list
> R-SIG-Mac@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
> 
> 

_______________________________________________
R-SIG-Mac mailing list
R-SIG-Mac@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-mac

[prev in list] [next in list] [prev in thread] [next in thread] 