[prev in list] [next in list] [prev in thread] [next in thread]
List: r-sig-mixed-models
Subject: Re: [R-sig-ME] Confidence interval around random effect variances in
From: Andrew Robinson <apro () unimelb ! edu ! au>
Date: 2021-04-05 0:18:00
Message-ID: d30fb535-94ff-4908-bb04-a24154ec8613 () Spark
[Download RAW message or body]
Surely any correspondence would be attenuated - IIRC there's some tuned shrinkage \
towards zero in the estimates of the variance components?
Best wishes,
Andrew
--
Andrew Robinson
Director, CEBRA, and Professor of Biosecurity Risk and Applied Statistics
Schools of BioSciences and Mathematics & Statistics
University of Melbourne, VIC 3010 Australia
Tel: (+61) 0403 138 955
Email: apro@unimelb.edu.au
Website: http://cebra.unimelb.edu.au/
On 5 Apr 2021, 8:58 AM +1000, Ben Bolker <bbolker@gmail.com>, wrote:
This would make an interesting simulation and/or theoretical exercise
(I'm going to resist the urge to do it), i.e. identifying the
correspondence between p-values constructed from parametric bootstrap
full-vs-reduced model comparisons and p-values estimated as fraction of
PB fits of full model that give variance=0 for the tested variance
component(s).
On 4/2/21 8:22 PM, Jack Solomon wrote:
Well, how about concluding so:
If a (say 2-level) model gives a singular fit (even though perhaps there
is a "tol" that is small but not exactly "0" for that warning to show
up), that would mean we have a "practically" non-significant
random-effect variance component.
On Fri, Apr 2, 2021 at 7:15 PM Ben Bolker <bbolker@gmail.com
<mailto:bbolker@gmail.com>> wrote:
I'm not sure that the bootstrapped CIs *wouldn't* work; they might
return the correct proportion of singular fits ...
On 4/2/21 8:12 PM, Jack Solomon wrote:
Thank you all very much. So, I can conclude that a likelihood
ratio test
and/or a parametric bootstrapping can be used for random effect
variance
component hypothesis testing.
But I also concluded that the idea of simply using a bootstrapped
CI for
a random-effect variance component [e.g., in lme4;
confint(model,method="boot",oldNames=FALSE) ] by definition
can't be
used for significance testing, because it requires the
possibility of
seeing sd = 0 which can't be "strictly" captured by such a CI from a
multilevel model (at least not easily so).
I hope my conclusions are correct,
Thank you all, Jack
On Fri, Apr 2, 2021 at 6:51 PM Ben Bolker <bbolker@gmail.com
<mailto:bbolker@gmail.com>
<mailto:bbolker@gmail.com <mailto:bbolker@gmail.com>>> wrote:
Sure. If all you want is p-values, I'd recommend parametric
bootstrapping (implemented in the pbkrtest package) ... that
will avoid
these difficulties. (I would also make sure that you know
*why* you
want p-values on the random effects ... they have all of the
issues of
regular p-values plus some extras:
http://bbolker.github.io/mixedmodels-misc/glmmFAQ.html#testing-significance-of-random-effects
<http://bbolker.github.io/mixedmodels-misc/glmmFAQ.html#testing-significance-of-random-effects>
<http://bbolker.github.io/mixedmodels-misc/glmmFAQ.html#testing-significance-of-random-effects \
<http://bbolker.github.io/mixedmodels-misc/glmmFAQ.html#testing-significance-of-random-effects>>
)
On 4/2/21 7:37 PM, Jack Solomon wrote:
> Thanks. Just to make sure, to declare a statistically
NON-significant
> random effect variance component, the lower bound of the
CI must be
> EXACTLY "0", right?
>
> Tha is, for example, a CI like: [.0002, .14] is a
> statistically significant random-effect variance component but
one that
> perhaps borders a p-value of relatively close to but
smaller than
.05,
> right?
>
> On Fri, Apr 2, 2021 at 6:19 PM Ben Bolker
<bbolker@gmail.com <mailto:bbolker@gmail.com>
<mailto:bbolker@gmail.com <mailto:bbolker@gmail.com>>
> <mailto:bbolker@gmail.com <mailto:bbolker@gmail.com>
<mailto:bbolker@gmail.com <mailto:bbolker@gmail.com>>>> wrote:
>
> This seems like a potential can of worms (as
indeed are all
> hypothesis tests of null values on a boundary ...)
However,
in this
> case
> bootstrapping (provided you have resampled appropriately -
you may need
> to do hierarchical bootstrapping ...) seems reasonable,
because a null
> model would give you singular fits (i.e. estimated
sd=0) half
of the
> time ...
>
> Happy to hear more informed opinions.
>
> On 4/2/21 6:55 PM, Jack Solomon wrote:
> > Dear All,
> >
> > A colleague of mine suggested that I use the
bootstrapped CIs
> around my
> > model's random effect variances in place of
p-values for them.
> >
> > But random effect variances (or sds) start from
"0". So,
to declare a
> > statistically NON-significant random effect variance
component, the
> > lower bound of the CI must be EXACTLY "0", right?
> >
> > Thank you very much,
> > Jack
> >
> > [[alternative HTML version deleted]]
> >
> > _______________________________________________
> > R-sig-mixed-models@r-project.org
<mailto:R-sig-mixed-models@r-project.org>
<mailto:R-sig-mixed-models@r-project.org
<mailto:R-sig-mixed-models@r-project.org>>
> <mailto:R-sig-mixed-models@r-project.org
<mailto:R-sig-mixed-models@r-project.org>
<mailto:R-sig-mixed-models@r-project.org
<mailto:R-sig-mixed-models@r-project.org>>> mailing list
> >
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
<https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models>
<https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
<https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models>>
>
<https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
<https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models>
<https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
<https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models>>>
> >
>
> _______________________________________________
> R-sig-mixed-models@r-project.org
<mailto:R-sig-mixed-models@r-project.org>
<mailto:R-sig-mixed-models@r-project.org
<mailto:R-sig-mixed-models@r-project.org>>
> <mailto:R-sig-mixed-models@r-project.org
<mailto:R-sig-mixed-models@r-project.org>
<mailto:R-sig-mixed-models@r-project.org
<mailto:R-sig-mixed-models@r-project.org>>> mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
<https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models>
<https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
<https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models>>
>
<https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
<https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models>
<https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
<https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models>>>
>
_______________________________________________
R-sig-mixed-models@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
[[alternative HTML version deleted]]
_______________________________________________
R-sig-mixed-models@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic