[prev in list] [next in list] [prev in thread] [next in thread] 

List:       r-sig-mixed-models
Subject:    Re: [R-sig-ME] Question about what is "shrinkage"...
From:       Daniel Wright <Daniel.Wright () act ! org>
Date:       2013-09-20 14:37:47
Message-ID: 464c2e6d78774cd89afc4c87531907b1 () BLUPR04MB022 ! namprd04 ! prod ! outlook ! com
[Download RAW message or body]

Hi Emmanuel and others,

I would think of shrinkage as a characteristic result of what a lot of estimation \
methods do, rather than as a method in itself. The Efron and Morris paper focused on \
showing that this characteristic can be good, and they discuss this in light of some \
popular methods used then (they wrote several influential and more technical papers \
on empirical Bayes during this period). A lot of mixed/multilevel folks discuss \
different methods, but the value of this characteristic is well illustrated in Bates' \
last email and the "borrowing strength" phrase. So, yes, the values in caterpillar \
plots and the like (conditional modes, though often called level 2 residuals) are \
estimates of the individual unit which borrow information from the other units, so \
are "shrunken". 

One way to show the effect of "shrinking" is show a plot with all the individual \
regression lines (just two variables) and then show the lines with with the slope and \
intercept estimated with shrinkage. An example is comparing Figures 3.7 and 3.8 of \
Kreft and de Leeuw's Introducing Multilevel Modeling. Here is some code to make an \
example. The plot on the left shows the OLS estimates, and there are two level 2 \
units which are very different from the others. The shrunken estimates on the right \
borrow information from the other 8 and mean the slopes of these two, while still \
different from the others, are a little less different.

set.seed(818)
library(lme4)
lev2 <- rep(1:10,10)
x <- rnorm(100)
y <- rep(rnorm(10),10) + (rep(rnorm(10),10)+2)*x + rnorm(100)
par(mfrow=c(1,2))
plot(x,y,cex=.5)
for (i in 1:10)
  abline(lm(y[lev2==i]~x[lev2==i]))
plot(x,y,cex=.5)
m1 <- coef(lmer(y~x+(x|lev2)))$lev2
for (i in 1:10)
abline(m1[i,1],m1[i,2])

Dan


-----Original Message-----
From: Emmanuel Curis [mailto:emmanuel.curis@parisdescartes.fr] 
Sent: Thursday, September 19, 2013 10:41 AM
To: Daniel Wright; bates@stat.wisc.edu
Cc: r-sig-mixed-models@r-project.org
Subject: Re: [R-sig-ME] Question about what is "shrinkage"...

Hello,

Thanks Daniel for the link on the clear article (despite I indeed do not know \
anything about baseball) and Douglas for the detailed answer. Quite interestingly, \
the article is more on the side of the estimator and Douglas' answer on the side of \
the reduced variance, at least as I understand it, but I think I begin to understand \
the link between the two.

But there are still a few questions I have, some of them philosophical...

When reading the paper, the two examples correspond to setups that could be handled \
by random-effect models (the baseball player or the town). In fact, in the end of the \
paper, individual mean values coming from a random variable is mentionned.

Does it mean that individual means obtained by random effect models as used in lmer, \
for instance, are themselves a kind of shrinkage estimator --- that is, already \
corrected by a shrinkage factor, but not given by a formula similar to the one cited \
in the paper? I know that random effects themselves are not (conditionnal) means, but \
modes, but when added to the fixed effects parts, corresponding to the mean (at least \
in linear models), aren't they comparable to (shrinked) means?

Would it be, in this case, an argument for prefering random effects over fixed \
effects when the number of modalities is < high > (>= 3 if I read correctly the \
paper, but may be another limit for such models and for cases of unkwnown, estimated \
variance?), beside convergence problems, and instead prefer fixed effects below even \
if philosophically a random effect would be needed (experiments on two patients only) \
--- and that there is a link between the efficiency of the shrinkage effect and the \
ability to estimate correctly the variance?

This would also explain how it is possible to associate a shrinkage to each random \
effect...

As far as I could see, however, the shrinkage estimator can also improve regression \
coefficients, when they are more than 3. Does it still holds when dealing with \
multidimensionnal vectors of which each composent represent very different things ? \
And for regression coefficients, if shrinked version gives better values, wouldn't it \
be logical to build tests on these coefficients on the shrinked values? Is it \
possible? (but these questions are on the frontier to be off-topic I guess).

My other concern is about the usage of shrinkage as a diagnostic. If I understood \
correctly Douglas answer, size of shrinkage measures how informative is the data of a \
single patient to estimate its own value. Hence, if shrinkage is important, does it \
mean that the model is not suitable for looking into individual predictions, but only \
average ones --- hence, useless in PK-pop for adaptating doses for instance? Is there \
any guides to define what is an acceptable shrinkage? And does it have other values \
for model's diagnostic and interpretation?

Last point: I understand well in the paper how to calculate the shrinkage factor \
(there seems to be several different but close formulas according to the reference, \
but I guess these are only variants?), using obtained values for each individual. But \
for several linear models, as mentionned by Douglas, it is not possible to obtain \
individual parameters. In such case, how is shrinkage computed/estimated ?

Thanks again in advance for any answer,

-- 
                                Emmanuel CURIS
                                emmanuel.curis@parisdescartes.fr

Page WWW: http://emmanuel.curis.online.fr/index.html

_______________________________________________
R-sig-mixed-models@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models


[prev in list] [next in list] [prev in thread] [next in thread]