If you want to learn more about how the line is fitted, watch this Stats Quest video. 

It is worth quickly re-capping what RSquare and adjusted RSquare are. In brief, R Squared is the proportion of the variation in the dependent variable that is predicable from the independent variable(s). RSquared and adjusted RSquare is a number from 0-1. So 0 means 0% of the variation is explained while 1 means 100% is explained. 

When using more than one independent variable always use adjusted R Square, rather than RSquare. The adjusted RSquared gives you a more precise view as it takes into account how many independent variables you have added to the model. In a model with more than one independent variable the RSquare will continue to go up giving you skewed results. Hence we use the Adjusted RSquare. A very easy to follow along video explaining RSquare is  available here. 

Interpreting RSquared:

• 0 to 0.33 - is a very weak model. 

• 0.34 to 0.50 is a weak to moderate model

• 0.51 to 0.67 is moderate model

• 0.68 to 0.75 is a moderate to strong model

• 0.76 and above is a  very strong model.

Estimate Coefficient:

The other value that you get when running a regression is the Estimate Coefficient. This represents the relationship between the dependent variable and the independent variable. 

• positive values mean that as the independent variable increases so does the dependent variable

• negative values mean that as the independent variable increases the dependent variable decreases. 

The Estimate Coefficient represents the mean change in the response given a one unit change in the independent variable. This means that if your Estimate is +0.5 your dependent variable will increase by 0.5 for each unit increase of the independent variable. 

As noted above, nominal/ordinal variables are represented using dummy variables. A dummy variable transforms nominal/ordinal data into a series of variables that take the value 0 and 1, where 0 indicates the absence or presence of something. (e.g. a 0 may indicate 'over 65' and 1 'under 65'). For more details how how dummy variables function follow this link. There are, surprise surprise, other ways of dealing with nominal variables. This is not covered here, but you can read more about it here. 

Note, you can treat ordinal data as continuous, however, this implies that you are removing its constrains and are treating the variable as if the distance between each step is equal - which it is not. Poor quality data - means poor quality results. 

The coefficients of dummy variables measure the average difference between the group coded with the value 1 and the group coded with the value 0 (the default or reference group). Most stats programmes should allow you to set your reference group easily. 

Standard Error of the Regression

Regression output will also display the Standard Error of the Regression or Standard Error. The SE represents that mean distances of the observed values from the regression line. It give us an indication of how wrong our regression model is on average. The smaller the SE, the better. Remember the SE is measures in Standard Deviations. So you would expect 95% of all observations to fall within -/+ 2 SD away from the mean. In some ways if you are trying to predict outcomes, looking at the SE can give a more accurate picture of how good the predictions are. 

Let's take a look at an example.

Let's quickly try and look at the results of the regression above. Remember, we have not checked our assumptions yet, so the results are only preliminary.  

As we can see the p-value shows that the model is significant (<.001), but the the Adjusted RSquared is not great at 0.162. 

As you can see each of the predictor variables (independent variables) has a significant p-value. If you have variables that are not significant, consider removing them from your model, but be aware that this will change all of the results. It is worth pointing out, that sometimes it is also worth reporting values when they are not significant, especially if you expected them to be so. 

Let's look at two of the variables Gender and Confidence in Self. 

Gender: 

The reference level here is Male. The difference between men and women is significant with a p-value of <.001. Looking at the Estimate, we can see that the mean difference in the extremism score for men and women is -0.495. This means women have a lower extremism score. Looking at the Confidence Intervals we can be 95% sure that the difference lies somewhere between -0.642 and -0.347. The SE is small, which is good. 

Confidence in Self:

Again the p-value indicates (0.006) that this relationship is significant. The Estimate tells us that for each unit increase in the Extremism Score confidence in self drops by -0.100. Looking at the Confidence intervals we can be 95% sure that the drop would be somewhere -0.171 and - 0.029. Again the SE is pretty small. 

The plot below visualises the relationship between Gender and Confidence in Self. As you can see the Extremism score for men is higher than that for women. We can also see a downward trend in self-confidence as the Extremism score increases for both men and women. 

Overall though, given the Adjusted RSquare value this is not a great model. 

Before we accept the results of any linear regression we need to make sure that all of the a series of assumptions are met. 

The assumptions are:

• Linearity: the relationship between x and y is linear

A Scatterplot should give you a good idea if your data has a linear or curvy-linear relationship

• Homoscedacity: The variance of residuals is the same as for any value of x.

• Normality: For any fixed value of X, Y is normally distributed

You can check the two assumptions above by looking at a Q-Q plot and residual plots. There are also test that test for normality - if you get a p-value of <.05 then the assumption is violated.

• Multicollinearity:  an independent variable can be predicted by linearly predicted by another independent variable with some accuracy. 

To check that this assumption is met you use the VIF (Variance Inflation Factor) and the Tolerance. The VIF should be as low as possible. It should be no more than 4 or 5. The Tolerance should be above .20. 

• No auto-correlation (use this only if you are comparing variables with similar variables from earlier studies - this checks for carry over effect).

A quick note on interpreting Residual Plots and Q-Q plots.

Residual Plots:

When you check your residual plots, you need to make sure that there are no patterns. If there are, then you will need to use alternative methods to analyse your data. Sometimes, it just indicates that the relationship is curvy-linear. 

Q-Q Plots:

You can also check the Q/Q plot to check for normality assumptions. Essentially, you need to ensure that the datapoints follow the line. In the below example you can see that the normality assumptions are not met as the datapoints are nowhere near the line. For a detailed explanation watch this video on quantiles and this video for interpreting Q/Q plots. 

In many ways interpreting Logistic Regressions is similar to interpreting the outcomes of a Linear Regression. 

Enter the world of Pseudo RSquared. Is it more difficult to calculate the RSquared for Logistic Regressions. We therefore use something called the Pseudo RSquared instead. You report this in the same way you would report RSquared for a Linear Regression. Note, don't expect it to be as big as RSquared. 

The most commonly used Pseudo RSquared is McFadden's here anything between 0.2 and 0.4 is an excellent fit for your model. 

The Estimate is the Log(odds) and Z is the Estimate standardised - so the value tells you how many standard deviations the estimate is away from the mean. The higher the Z value, the less likely it is to be significant. 

Let's explore how to interpret the results of a Multinominal Regression (you interpret a Binomial Regression in the same way).

In the Example below we explore the impact of Age and Gender on having Qualification or not. 

We have set no Qualification and Male as the reference levels 

As you can see from the big table above, there is a significant correlation between the selected variables (p-value <.001). The Pseudo RSquared is not great at 0.026. 

Let's have a look at the third row - the likelihood of having a postgraduate degree versus not having Qualification. Based on the p-value we see there there is a significant difference due to gender (p 0.022) and age (p 0.008). The odds ration for this row indicates that woman are 3.766 time more likely to have a postgraduate degree than men. Looking at the Confidence intervals, we can be 95% sure that this outcome lies somewhere between 1.211 and 11.715. 

In terms of age, we can see that as age goes up the likelihood of getting a postgraduate degree goes down. For each year you get older, the odds of having a postgraduate degree go down by 0.943. We can be 95% sure that the true value is somewhere between 0.910 and 0.981.

Let's visually display the results.