[prev in list] [next in list] [prev in thread] [next in thread]
List: r-sig-mixed-models
Subject: Re: [R-sig-ME] Interpreting GLMM output and is this the right model?
From: Boku mail <luca.corlatti () boku ! ac ! at>
Date: 2020-12-08 13:31:21
Message-ID: 48b7837b-c4e5-4c98-a71d-1e26aa19afdf () Spark
[Download RAW message or body]
Hi Gabriella,
looks like your residuals are not behaving well, so I would not trust your model \
output. I have no idea about your data, and there are many people more competent than \
me in this list, who can give you useful advice. At least you know that your starting \
point needs to be redefined =) It might be that you actually need to change \
conditional distribution and pick one that can handle skewed responses in linear \
mixed models(?).
Try have look here: https://cran.r-project.org/web/packages/skewlmm/skewlmm.pdf
(You can fit the same model and look again at the residuals)
Alternatively, a similar model could be fitted with the package ‘brms', in a \
Bayesian framework:
library(brms)
mod.1.brm <- brm(sentiment ~ primetype + age + (1|id), data=DF, family=skew_normal()) \
# get a coffee, it might take a while... predict.mod.1.brm <- \
t(posterior_predict(mod.1.brm)) # extract posterior predictions to feed to DHARMa \
library(DHARMa) sim.resid.mod.1.brm <- createDHARMa(simulatedResponse = \
predict.mod.1.brm, observedResponse = DF$sentiment) plot(sim.resid.mod.1.brm) # other \
stuff should be inspected, but this should allow you to preliminary check if the \
distributional assumption is ok library(parameters)
parameters(mod.1.brm, ci=0.95, digits=3, ci_digits=3)
L.
Il 8 dic 2020, 13:47 +0100, Gabriella Kountourides \
<gabriella.kountourides@sjc.ox.ac.uk>, ha scritto:
> Dear Luca,
>
> Many thanks!
>
> That code worked well, output is attached, I'm not sure how to interpret it, but \
> the KS test p=0 (devation is sig), the dispersion test p=0.992(devation is not \
> sig) and but the outlier test p=0 (devation is sig)
> Thank you for your well explained answer, it is all much clearer to me, I really \
> appreciate it!
> From: Boku mail <luca.corlatti@boku.ac.at>
> Sent: 07 December 2020 18:07
> To: r-sig-mixed-models <r-sig-mixed-models@r-project.org>; Gabriella Kountourides \
> <gabriella.kountourides@sjc.ox.ac.uk>
> Subject: Re: [R-sig-ME] Interpreting GLMM output and is this the right model?
>
> Dear Gabriella,
> A few thoughts:
>
> 1) generally speaking, it doesn't make much sense to look at the distribution of \
> the raw response. The choice of the ‘family' argument is rather based on the \
> conditional distribution, i.e., the distribution of the response across the fitted \
> line (or, in other words, the distribution of the response after accounting for the \
> linear predictor). In this respect, I guess it's hard to say whether the skewness \
> you mention may be a problem or not (same for the zeros).
> 2) you fitted a mixed model with Gaussian conditional distribution. Whether this is \
> a ‘good model' or not, is hard to say (e.g., does your linear predictor include \
> the *supposedly* important explanatory variables, in the correct form? etc.), but \
> at least I would inspect the residuals' behavior. This would allow you at least to \
> check if the model is not grossly wrong. Suppose your model name is "mod.1", then \
> you could do:
> library(DHARMa)
> sim.mod.1 <- simulateResiduals(mod.1)
> plot(sim.mod.1)
>
> Or, alternatively:
> library(performance)
> check_model(mod.1) # but DHARMa would be handier if you decided to change \
> conditional distribution, and fit a GLMM
> If the model is not grossly wrong, residuals should be distributed in an \
> unsystematic way. If you have weird patterns, then you're probably off-track. If \
> that happens, you should inspect what's wrong in your model (conditional \
> distribution? Missing variables? etc.)
> 3) once you're confident your model is ‘well behaved', then you can inspect the \
> results. What your current summary says, basically, is that after accounting for \
> primetype, if you increase age by 1 unit, you'll have a decrease of -0.0020644 \
> in sentiment, and this decrease is statistically significant (basically, Estimate/ \
> Std. Error = z value; the z-score measures how far [in standard errors] your \
> estimate is from zero; z-score follows a standard normal distribution, hence a \
> value of z < -1.96 or z > 1.96 will be statistically significant at an alpha = \
> 0.5). Whether this is also biologically significant, up to you to say! The same \
> reasoning applies to the primetype levels, except that this is a categorical \
> variable, so primetype2 will be 0.0907564 'sentiment scores' higher than the \
> baseline primetype1 (the intercept). Should you be interested in comparing \
> different primetype levels in a pairwise manner:
> library(emmeans)
> emmeans(mod.1, ~primetype) # haven't used this in a while, so cross-check in the \
> package :-)
> Result interpretation is straightforward in the case of a LMM, which uses an \
> identity link function; should you use a different link, then things would be \
> tricker to interpret, and plotting the marginal effects would be wise:
> library(visreg)
> visreg(mod.1, "primetype", scale="response")
>
> However, I guess the crucial step for you will be to inspect the behavior of the \
> model in the first place.
> Hope this helps!
> Luca
>
>
>
>
> Il 7 dic 2020, 18:10 +0100, Gabriella Kountourides \
> <gabriella.kountourides@sjc.ox.ac.uk>, ha scritto:
> > Hi everyone,
> >
> > I emailed a few weeks ago, but am still struggling with this data.
> > The description of the question below, and model/code/output at the bottom. Many \
> > thanks for reading.
> >
> > I want to look at whether there is a relationship between the way a question is \
> > asked (positive, negative, neutral wording) and the sentiment of the response. I \
> > have 2638 people asked a question about symptoms. 1/3 of the people were asked it \
> > with a negative wording, 1/3 with a neutral one, 1/3 with a positive one. From \
> > this, I did sentiment analysis (using Trincker's package) to see whether their \
> > responses were more positive or negative, depending on the wording of the \
> > question. Sentiment analysis breaks down responses into sentences, so I have 2638 \
> > people, but 7924 sentences, so I would assume to fit ID as a random effect.
> > The big question is: does the way the question is asked (primetype) affect the \
> > polarity/sentiment of the response? My data is negatively skewed, and has a lot \
> > of 0s (this is because some people felt 'neutral' and so they scored '0'.
> > Model using the dataframe DF, to see how primetype (this is the way the question \
> > is asked) predicts sentiment (the polarity score, which is negatively skewed with \
> > lots of 0s), fixed effect is age, and random effect is ID
> > ```
> > glmmTMB(sentiment ~ primetype + age + (1|id), data=DF)
> > ```
> >
> >
> > Output:
> >
> > ```
> > Family: gaussian ( identity )
> > Formula: sentiment ~ primetype + age + (1 | id)
> > Data: DF
> >
> > AIC BIC logLik deviance df.resid
> > 7254.9 7296.5 -3621.4 7242.9 7556
> >
> > Random effects:
> >
> > Conditional model:
> > Groups Name Variance Std.Dev.
> > id (Intercept) 8.732e-11 9.344e-06
> > Residual 1.526e-01 3.906e-01
> > Number of obs: 7562, groups: id, 2520
> >
> > Dispersion estimate for gaussian family (sigma^2): 0.153
> >
> > Conditional model:
> > Estimate Std. Error z value Pr(>|z|)
> > (Intercept). -0.1655972 0.0204310 -8.105 5.27e-16 ***
> > primetype2 0.0907564 0.0114045 7.958 1.75e-15 ***
> > primetype3 0.0977533 0.0115802 8.441 < 2e-16 ***
> > age -0.0020644 0.0006483 -3.184 0.00145 **
> > ---
> > Signif. codes: 0 ‘***' 0.001 ‘**' 0.01 ‘*' 0.05 ‘.' 0.1 ‘ ' 1
> > >
> > ```
> > How can I interpret whether the model is a good one for my data, is there \
> > something else I should be doing? I'm not sure how to interpret the output at \
> > all. Would be immensely grateful for any insight
> >
> > Thanks all
> >
> >
> > Gabriella Kountourides
> >
> > DPhil Student | Department of Anthropology
> >
> > Evolutionary Medicine and Public Health Group
> >
> > St. John's College, University of Oxford
> >
> > gabriella.kountourides@sjc.ox.ac.uk
> >
> > Tweet me: https://twitter.com/GKountourides
> >
> > ________________________________
> >
> >
> >
> > [[alternative HTML version deleted]]
> >
> >
> > _______________________________________________
> > R-sig-mixed-models@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
[[alternative HTML version deleted]]
_______________________________________________
R-sig-mixed-models@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic