This week’s class was about the criteria that might help us to judge the quality of qualitative research. Although I knew that qualitative researchers would have distinguished criteria from qualitative ones (i.e., acknowledging the potential biases rather than believing in completing objectivity), I have not thoroughly tried to conceptualise it. Hence, the paper by Guba and Lincoln (1982) was intriguing to read. The authors argue for the fundamental difference between the rationalistic and naturalistic paradigm. For instance, the use of random sampling, often used in scientific research, is impractical in a natural setting that consists of the complex social values of stakeholders. Hence, the authors conclude that the judgements and methods used in the rationalistic world are not adequate tools for the naturalistic world. This argument reminded me of the epistemological positions that we discussed in the previous week, and it was clear that the authors were more fond of interpretivism than positivism.
The paper later suggests the various standards that we could use to assess the quality of qualitative research. Although there were other many worthwhile criteria, I found neutrality and transferability the most interesting and relevant to my research.
Neutrality is concerned with the influence that the values of the researchers might have on the research findings. In a rationalistic paradigm, one can attain this through the intersubjective agreement of quantitative notions. However, in the naturalistic paradigm, the impossibility of complete objectivity is recognised (Guba and Lincoln, 1982). That is, one cannot be value-free but bound by the social environment that inevitably influences their decisions regarding the approaches to their research. For instance, the personal belief of the researcher might influence their choice of the relevant theories and interpretation of the data. In this case, we are more interested in the confirmability of the data rather than the researcher’s level of skills to collect the data.
I feel like this is a crucial part of my research. Firstly, I am interested in the use of games in learning because I like playing games. It was my interest that motivated the research question. Admittedly, this could lead to potential bias in favour of the use of games. Simundić (2013) suggests that the researchers might extrapolate the collected data to support the original hypothesis. Hence, in our class, someone suggested using the opposite perspectives to justify the methods and interpretations could be valuable. I might base my argument on a paper that disagrees with the usefulness of game-based learning and try to explain why I believe its methods or interpretation are inappropriate. From there, I could start developing my line of reasoning for my data collection and its analysis.
The second criterion that interested me was transferability. This criterion is concerned whether the findings from particular research can be applied to the other context. In a rationalistic paradigm, the researcher can attain this by generalisation through random sampling and argues that the sample represents the whole population. However, as individual minds are different, so as the context, the flawless generalisation would be impractical in qualitative research (Guba and Lincoln, 1982). Therefore, we are more interested in the degree of transferability of findings and their justification.
The assessment of transferability is important due to the nature of the design-based research. As the method often involves the creation of educational ecology that might result in the discontinuity and fundamental difference from the standard curriculum. In most of the DBR, it overcomes this issue through an iterative process where the researchers can discover critical variables and limitations (Amiel and Reeves, 2008).
The data collection methods of DBR could vary and could be a mixture of more than one method. For my research, I think it would be better to use at least two methods, observation and focus group, to collect sufficient data. Observations will allow the researcher to elicit the actions from the participants in a naturalistic setting. The purpose would be to observe how students would interact with the games and the instructions provided.
Yet, the issue with the observation is that we are not sure about the cognitive process of the participants. Although their physical movements, such as the number of clicks that they made, we cannot confirm that their emotional state during the activity
So to confirm the congruence, the use of triangulation is often considered. Triangulation is when you use different methods and theories to cross-check the data collected. For instance, in a focus group, participants can elaborate on their experiences while playing games. Of course, their opinions might be skewed due to various factors such as classroom dynamics or the presence of the researcher. Thus, comparing the results from different methods would be crucial for the credibility of the research.
References
Guba, E. G., & Lincoln, Y. S. (1982). Epistemological and methodological bases of naturalistic inquiry. Educational Communication & Technology, 30(4), 233–252.
Simundić A. M. (2013). Bias in research. Biochemia Medica, 23(1), 12–15. https://doi.org/10.11613/bm.2013.003