Arthur Langer’s article Reflecting on Practice: Using Learning Journals in Higher and Continuing Education, attempted to evaluate the effectiveness of learning journals when utilized in the nontraditional classroom (2002). He used a two part methodology involving random sampling of a non-traditional college technology class and self-selection based volunteers to examine the usefulness of learning journals in this forum. This evaluator analyzed Langer’s research question, review of literature, methodology, empirical evidence and discussion of these factors attempting to determine the validity of this study. It is the conclusion of this evaluator that while Langer is a well organized writer whose research seems to fill a void in the education dialogue on learning journals, he failed to create a credible, reproducible study for a variety of reasons, including the fact that he did not explain the significance of his focus on non-traditional students, seemingly ignored the potential limits of his sample, failed to explain in depth his instrument of choice for evaluating journals and regrettably did not mention any of his possible limitations in his explanation of findings.
One of Langer’s strong points as a researcher is his ability to write in an organized fashion. He began with a strong overview which made it easy for this researcher to follow his progression through the documented study. His purpose was clearly presented in the beginning of his study when he remarked, “ how the use of journals impacted the learning process of adult students of the latter category [non-traditional students] and how the impact compared to that of students of the former category [traditional students]” (Langer, 2002, p. 337). He then provided an overview on the method chosen for this study: a three semester sampling of a required technology course at Columbia University. Finally, Langer hinted at a conflict between his findings and previous research, namely the scholarly presumption that “ journals provide a tool for learning” (2002, p. 338). While Langer’s introduction hints at the solid organization and research he used throughout his research analysis, it failed to indicate one of the first major problems with his study.
Langer produced a plethora of evidence on learning journals and traditional student critical thinking throughout his literature review; unfortunately, he neglected to examine the need for research on non-traditional student learning, a seemingly key component to his research. His research purpose was clearly presented in the beginning of his study when he noted, “ how the use of journals impacted the learning process of adult students of the latter category [non-traditional students] and how the impact compared to that of students of the former category [traditional students]” (Langer, 2002, p. 337). It is obvious to this researcher that there is significance to using journals for reflective learning and then some discernable difference in the mind of Langer to warrant research on non-traditional students. To begin he scrutinized a variety of current and credible sources of his time to explain the significance of reflective learning and learning journals. When discussing reflective learning, Langer examined works from researchers such as Mezirow, who spent over a decade researching reflective and critical thinking practices in academia, specifically in the field learning journals and with adult learners (Kemper et al., 1999, p. 20). While reflective learning can be covered in a variety of ways, Langer chose to narrow his research to learning journals, which his research illustrated was proven as a source of valid evidence gathering when attempting to understand reflective learning (Kemper et al, 1999). As Langer explained that there was little research specifically on technological field, his review of research examined similar fields requiring, “ knowledge based axioms and demonstratable proof” (Langer, 2002, p. 339), such as the science and engineering fields. He also focused on the student population for this study by reviewing research on how journal reflection applies to both academic thinking and real life experiences. Finally, Langer summarized the three categories of journals that are most often examined in this genre of research, namely unstructured, structured, and dialogue journals. This review of previous research led Langer to justify his research direction by noting, “ Research on the use of learning journals in technology fields in higher education among non-traditional students, however, is lacking” (2002, p. 341). Unfortunately, there is one oversight. A key component to his research is the fact that the non-traditional student has been all but ignored from research in this field (2002). Since his review of research examined a variety of academic endeavors, it is logical to assume that traditional learners have been more thoroughly researched. While Langer did discuss how other researchers perceived learning journals impact students experienced outside the classroom, there was no evidence presented that illustrated this is significantly different for non-traditional students. On the contrary, Langer himself seemed to illustrate that the two student types were similar. At one point, Langer cited Grumbacher (1987) when explaining how students using learning journals in the science classroom can become better problem-solvers. Yet this is a quality that Langer pointed out was also important to non-traditional students when he discusses Moon’s research on learning journals (2000). This leads this researcher to wonder what Langer was seeking to find different between these two demographics.
Langer’s second potential flaw, from this evaluator’s perspective, was in the sample chosen for this study. Consistent with his introduction, Langer explained the sample make up, size and significance. Specifically, he chose random students taking a required technology course in a certification program designed for adults changing their career paths (2002). Over three different semesters a total of thirty students had their sets of fifteen journals reviewed for the first segment of the study. This is a relatively small sample considering the claim he makes in the beginning of his paper; that students of the two different demographics have different findings in relation to learning journals and reflectiveness.
The second sample flaw was the interviewing process of these students. Rather than interviewing all students who participated in the first segment of the study, Langer and his assistants requested volunteers of this group to meet and be interviewed on their journals. While self-selected samples are often used in research they are potentially biased in one significant way relating to this study, namely those with high positive or negative interest will self-select. One researcher noted in regards to self-selecting samples involving questionnaires, “ people who have a particular interest in the subject matter or the research itself are more likely to return mail questionnaires than those who are less interested” (Fowler, 1984, p. 49). It is possible then that the students who participated in this second segment had strong feelings toward the use of learning journals, which would eliminate the responses of students who were potentially less impacted by the act of journaling.
One final problem with Langer’s methodology was his lack of explanation on the instrument utilized for his study. Earlier he mentioned that his study is based, in part, on the Kemper et al. (1999) modifications to Mezirow’s (1991) categories as they related to journal writing and reflective thinking (Langer, 2002, p. 339). However, there was no developed explanation as to these categories. This is significant because without further research into this model, it is not possible for this researcher to evaluate whether Langer’s research followed the categories thoroughly enough to compare results between the studies, something done later in the article by Langer.
While it is unclear as to the validity of the claim that Langer matched his current study to previous researchers, he did clearly report on the results of both the evaluations and interviews of the student participants. In relation to the journals, Langer’s first observation was how closely students stayed aligned to the sample journals provided by the instructor, even when the instructor provided directions to be creative (2002, p. 343). He also noted that over half of the participants seemed to become more reflective as the course progressed. Langer was quick to support this with sample student comments that illustrate high self-reflectiveness on the part of the student (2002, p. 343).
Langer also provided a detailed summary of the ten interviews of the volunteer participants. Because he clearly defined each question asked by the interviewers, this researcher was able to understand the context of the answers as they related to Langer’s study. For example, one important question asked the participants how they used these journals over the fifteen week course. While all students were assigned the journals for a course assignment, answers varied on their usefulness from study aids and understanding the materials to using the journals as a way of compensating for a tension between the students preferred learning style and the professor’s method of teaching (2002, p. 345). Langer then used the results from both segments of his qualitative study to illustrate common themes when using learning journals in a non-traditional technology based course.
Disappointingly, however, while Langer provided these detailed results and explained their significance to future studies, he failed to examine his own study’s flaws or limitations. One limitation that Langer should have acknowledged was not only the small sample size, but the small scope of the geographic location of this study. All participants were part of a sample of Columbia University in New York City. While it is possible and probable that courses such as the one Langer used for his focus exist in other schools in other states and even in other countries, every population has factors that must be taken into consideration. For example, New York City is urban nature. Would these results be the same if the course was in a college in West Virginia? Are students in the northern United States statistically different from their peers in other areas of the country? While this might not have any bearing on the conclusions, the demographics do pose a potential limitation that Langer should have addressed.
Finally, while Langer did reproduce some results from previous findings, such as, “ learning journals can improve knowledge for students” (2002, p. 349), he focused instead on possible discrepancies, notably the lower rate of critical analysis in this sample of non-traditional student journals. Langer explained some possible reasons for this discrepancy such as the professor’s potential lack of instruction or lack of understanding by the student. However, nowhere did Langer consider that the possibility is that the sample size provided a limited scope to allow for this comparison.
Although Arthur Langer’s research should have voice in the dialogue on reflective learning through journals, his lack of justification for the scope of his study left this researcher to question the significance this study has in relation to others of its kind. Langer is a researcher whose organization allowed for easy comprehension of his thought process, however, he failed to explain the significance of his focus on non-traditional students, seemingly ignored the potential limits of his sample, did not explain in depth his instrument of choice for evaluating journals and regrettably did not mention any of this studies possible limitations in his explanation of findings. This researcher has learned not only the value of the topic of learning journals through Langer’s in depth analysis and review, but also the importance of explaining and justifying the scope of the study being presented. If this researcher hope to create studies that stand up to future scrutiny and allow for others to test the validity of the arguments presented, it is also important to clearly explain the instrument used in the research endeavor. Finally, it is humbling to note that no matter the precision of research and application of methodology, all research inherently comes with researcher biases and limitations. It is wise to remember this, lest the study be dismissed as unreliable or unrealistic for lack of acknowledging these limitations.