I: General Discussion example
The survey design and data for the example section below is adapted from a classroom project by Fettig, López Fuentes, and Villarreal (2019).
Introduction…
Method… [see Appendix C]
Study 1…
Results… [see Appendix E]
Discussion… [see Appendix G]
Study 2
Results… [see Appendix E]
Discussion… [see Appendix G]
General Discussion
At the outset of this study, we predicted that if students are given a chance to indicate their levels of daily student stress on a continuum rather than as categories, there would be a greater chance of connecting these stressors to sleep quality.
This was true at one level. That is, we did not find any effects in Study 1, where both independent variables were categorical in nature. But we did find an effect in Study 2, where both independent variables were on continuous scales. Thus, at least at one level, it is true that perhaps we need scales with finer resolution in order to detect the relations between daily student stressors and sleep quality.
However, Study 2 also showed us that it was not merely the difference between categorical and continuous scales of measurement that led to differences in detecting the relation between daily student stressors and overall sleep quality. Rather, it was the difference between the part and the whole. That is, although both questions in Study 2 were given on continuous scales, the first (academic course load) was yet another sub-component of overall daily stressors, whereas the second (overall work load) asked students to indicate their combined stress level across all domains of their daily student life (academic, work, and extracurricular). Only the latter was significant, which compromises the notion that it was categorical versus continuous scales that mattered most in this study.
Instead, what seems to be true is that it was not, in fact, this difference across scales of measurement that mattered, but rather the whole versus the components of the construct. Critically, the second question in Study 2 combined the three previous questions (i.e., both from Study 1, and the first from Study 2) into one. None of the first three turned out to be statistically significant, whereas the last question did. In the end, this suggest that what matters most is what you are measuring, not how you are doing it.
As it should be easy to infer, a major weakness of this study was its initial premise: namely, that what matters most in studies of psychological phenomena is quantitative precision. Quantitative resolution is indeed important at some level. We do not dispute this. But it is subordinate to the importance of finding measures that truly address the phenomenon under scrutiny. If we had restricted ourselves to the subcomponents of overall daily workload without ever asking our last, single question regarding overall workload, we might never have found how they must be combined in order to arrive at a measure that can predict sleep quality.
All that said, it is also possible (indeed likely) that a composite variable of three continuous variables, each of which addressed the workload subcomponents, would have given us a similar, or even better connection with overall sleep quality. Naturally, this was not possible in the current study since two of the three variables here were categorical.
In the future, we hope that our study might serve as a lesson to future researchers who get too drawn in by issues of statistical measurement. That is, make sure that your statistical measures follow your construct development, and not the other way around. In other words, do not put the cart before the horse.