Step-by-Step Guide 11 (Jamovi):
Independent T-Tests
Overview
Here you will learn the following:
How to run a Students T-Test (Student's/Welch/ Mann Whitney U test);
how to generate a plot to display the output visually; and
how to run a Robust T-Test using Walrus.
T-Tests is an easy test that will allow you to explore whether there is a difference between the two groups. Here you will learn how to use them and select the correct T-Test.
Note: You can also run a T-Test for paired data, but this is not covered here. For more details, you can watch this clip here and read about them here.
Dataset used for Independent Sample Tests
Skoczylis, Joshua, 2021, "Extremism, Life Experiences and the Internet", https://doi.org/10.7910/DVN/ICTI8T, Harvard Dataverse, Version 3.
Independent Sample T-Tests (Student's/Welch/Mann Whitney U)
Independent Sample T-Tests Hypothesis
Ho: There is no difference in the extremism score of males and females.
Ha: There is a difference in the extremism score of males and females.
Independent Sample T-Tests variables required
Dependent Variable(s):
Extremism_Score_Scaled: This variable measures levels of extremism
Independent Variable(s):
Gender: Participants Gender
Independent T-Test Assumptions
Checking your Assumptions:
Checking your assumptions in Jamovi is very easy. Under the Assumptions tab, you will be able to select a test to check for Unequal Variance and Normality. Once selected a table will appear in the results panel.
These tests are based on the Hypothesis that there is no difference between the distributions. If these tests return a p-value of below 0.05 then you know that your assumptions have been violated.
Note: The larger your sample, the more sensitive your assumption checks will be, so it is always wise to check a descriptive plot too.
If Assumptions are not met:
If any of the below assumptions are not met, you should use a Robust T-Test. You will be introduced to this further down.
Welch's Test Assumptions (use as default)
Use this test as your default unless its assumptions are violated.
Independence: Your observations must be independent of each other.
Random Sampling: Your data should be a random sample of the target population
Normality: Your Dependent variable should be approximately normally distributed.
Note: This test is designed to cope with unequal variance.
Student's T-Test Assumptions
Independence: Your observations in each sample should be independent
Random Sampling: Your data should be a random sample of the target population
Normality: Your Dependent variable should be approximately normally distributed.
Equal Variance (Homogeneity): Both groups should have approximately the same variance
Mann Whitney U Test Assumptions (default for non-parametric data)
Ordinal or Continuous: Your dependent variable must be either ordinal or continuous.
Independence: Your observations must be independent of each other.
Distribution Shape: The shape of the distribution for your two groups should be roughly the same.
Independent Sample T-Tests: Step-by-Step Guide
1.
Make sure Assumptions are not violated
Navigate to Analyses > T-Test > Independent Samples T-Test
Now select your dependent and independent variable and drag and drop them in the relevant fields.
Results will appear to your right, ignore them for now as we will need to check that the assumptions are not violated first.
In the Assumptions Checks section select Homogeneity test and Normality Test boxes.
You can also select a Q-Q plot which will give you a visual representation. On a Q-Q plot, your data points should be as close as possible to the line. If the data points flair out at either end your assumptions are violated. By default stick with the Assumption test.
Below are the outputs of the Assumption Test. As you can see in our case both the Normality Test (p <.001) and the Homogeneity Test (p <.001) have been violated.
This means the assumptions for all three test have been violated.
The assumptions for a Mann Whitney U Test are also violated, as it assumes equality of variance. Looking at a descriptive plot (below), it does look like the data is 'roughly' equally distributed. So, we will report the Mann Whitney U test outcome.
However, we will carry out a robust t-test further down to check our results are reliable.
Note: What test you select will depend on the outcome of your Assumptions tests.
2.
Select the relevant Statistics, Descriptive tables and plots
Once you have selected the correct test, select your Additional Statistics.
If you have selected a Student's T-test or the Welch's test select the Mean Difference & Confidence intervals (the Mann Whitney U test ranks the data and is based on the Sum of ranks - so the mean is not a useful statistic in this case).
For all your test you should include the Effect size & Confidence Intervals.
You can also select a table with all the Descriptives in it, as well as a Descriptive Plot
Finally, you can specify if your Hypothesis was directional or not.
Results: Reject the Null Hypothesis
1.
Mann Whitney U test results
Based on the outcome of our Mann Whitney U Test, we reject the Null Hypothesis (p <.001).
The effect size suggests that the effect of gender on extremism is small to medium (0.299).
The T-Test descriptive table gives you some additional information, which you can use to contextualise your results.
Finally, you can also easily generate a plot to visually display your results. Just select Descriptive Plot.
I prefer the plot generated by JJStatsPlot (JJStatsPlot > Graphs & Plots), which I have displayed below.
The graph below compares the extremist score for men and women and again highlights the differences between genders. This is a Box Plot with scatter and a violin Plot.
Robust T-Test
Robust T-Test Hypothesis
Ho: There is no difference in the extremism score of males and females.
Ha: There is a difference in the extremism score of males and females.
This is the same Hypothesis as above.
Modules Required
To run a robust T-Test you will need to install the following Modules:
Walrus: This module provides you with a series of Robust tests that can be used when the normality assumptions are not meet.
Robust T-Test variables required
Here we use the same variables as in the test above.
Gender
Extremism_score_Scaled
Robust T-Test Assumptions
Independence: Your observations in each sample should be independent
Random Sampling: Your data should be a random sample of the target population
Robust T-Test: Step-by-Step Guide
1.
Select your Variables
Navigate to Analyses > Walrus > Robust Independent Sample T-Test
Now drag & drop your two variables into the correct place.
2.
Select your Statistics & Plot
Leave the default setting of Yuen's T-Test in place. In addition, also select the following:
Mean Difference (and its confidence intervals); and the
Effect Size (and Confidence intervals).
Depending on your Data, you can also select the M-estimator. Use this if your data has extreme outliers unless your data is a reflection of the true population.
Using the M-estimator will return a p-value that will allow you to reject/accept the null hypothesis. If used in conjunction with Yuen's T-Test you have got two powerful indicators that allow you to assess your hypothesis.
Results: Reject the Null Hypothesis
1.
Independent Samples T-Test Results
As you can see from the output below, we can reject the Null Hypothesis (p <.001). Again the effect of Gender on extremism remains is small to medium (0.408) The confidence intervals suggest that the effect is somewhere between 0.363 and 0.452.
Given that the Extremism variable has some extreme outliers, we have also used the M-estimator which also returns a significant result (p <.001).
So looking at the outputs of the Mann Whitney U and Robust T-Test we can say that there is a significant relationship between gender and extremism and that the effect of gender on extremism is small to medium.
Again, you can also add a Plot using JJStatsPlot. It is the same plot I have used above, so it is not included below.
