Editor’s note: this post originally appeared on my personal blog.

As I mentioned in a previous post, I’ve decided to try “flipping the classroom” this semester, meaning I’m posting the lectures online and using class time mostly for activities that reinforce core concepts and create incentives for students to keep up with the lectures from week to week. Look below the fold for a description of the second activity, which concerns the interpretation of regression results.

First, I described the data I analyzed, which the students themselves provided to me last week (anonymously).

Height is measured in inches (because ‘Merica), age in years, female = 1 if the student self-identified as female and 0 otherwise; lastcell is the numeric value of the last digit of the student’s cell phone number; laststud is the numeric value of the last digit of the student’s identification number provided by the university; cellev is equal to 1 if lastcell is an even number and 0 otherwise; and studev is equal to 1 if laststud is an even number and 0 otherwise.

Then I showed them a table summarizing the results of two models fit to these data. (Note that to make things easier on my students, I don’t show them actual coefficient estimates, standard errors, or p-values, just signs and stars. Yes, I know there’s more to statistical analysis than signs and stars. But this is an intro course, and I’m probably pushing the limits by asking this much of them.)

If this looks unreasonably challenging to you, keep in mind that before coming to class today, students were to have listened to lectures on probability and statistical analysis. In the latter of those, I showed students a table that was very similar to the one below and told them how such tables are interpreted. I wasn’t testing their preexisting knowledge here, but penalizing those who haven’t yet listened to the lectures despite repeated warnings that future activities would be a lot more challenging than the first one.

While a fair number of students were able to correctly summarize all of the results, many were not. Common mistakes included: drawing the table without making any attempt to explain what it meant; interpreting blank cells as indicating no association between the variables (as if we’d estimated a coefficient that we’re confident is close to zero when in fact no information was provided at all); and failing to distinguish between statistically significant relationships and those that are not statistically significant. A few students apparently don’t even know what it means to ask about relationships between variables in a data set, since they summarized the relationship between “cellev” and height as “even” or between age and height as “19”. Hopefully, these students learned today that they really do need to listen to the lectures (which about half of them have yet to do, according to Blackboard’s statistics tracking feature.)

Once they had all handed in their answers, I then talked through the results. Though one of goals for today was simply to convince them that it’s really not a good idea to try to get through the course without listening to the lectures, another goal was to help them see the limitations of focusing on statistical significance. Though I don’t think anyone who understood what the table was telling them was surprised to discover that the female students in this class are, on average, shorter than their male counterparts, the other statistically significant relationships are almost certainly coincidences. I can think of no reason why there should be an association between age and height among college students (obviously it would different if I performed the same activity with grade schoolers), let alone why that relationship should emerge in model 2 but not model 1. Moreover, neither I nor my students could think of any reason to believe that the negative association between height and having a cell phone number whose last digit is divisible by two would generalize to the broader population. Hopefully, this helped students see that “unlikely to be observed by random chance if in fact there was no relationship” means exactly that—no more and no less.