Published Online: April 15, 2015
Abstract: The use of asynchronous discussion forums (ADFs) is thought to assist in enhancing students’ collaborative learning and critical thinking throughout higher education. However, previous research has mainly focused on individual critical thinking while the investigation of critical thinking during group work has been generally overlooked. Furthermore, few studies have investigated critical thinking processes of the individual and of the group in a single study to present a comprehensive picture of collaborative critical thinking (CCT). To address these gaps, I examined the demonstration of CCT on ADFs in a graduate subject at an Australian university over two academic semesters as students discussed topics online. In this paper, I discuss the ontological and methodological challenges in conducting the above research and present possible solutions to these challenges. At the ontological level, I discuss challenges in conceptualising and defining CCT. At the methodological level, I present challenges in constructing a coding scheme to measure the demonstration of CCT on ADFs. I then discuss ways to tackle the above challenges, propose an operational definition of CCT and present a synthetic coding scheme for measuring CCT in computer-supported collaborative learning contexts such as on ADFs.
Keywords: asynchronous discussion forum, collaborative critical thinking, computer-supported collaborative learning (CSCL), higher education, methodology, online discussion, ontology
Background and Review of Current Research
From the late 1970s and early 1980s developing critical thinking skills in students gained noticeable prominence in higher education (Moore, 2011). In higher education, knowledge construction is viewed as both a process and a product of argumentation and scientific reasoning (Derry, Seymour, Steinkuehler, Lee, & Siegel, 2004). In recent years, in many Western countries such as Britain and Australia, critical thinking has become a major graduate attribute that universities strive for students to develop throughout their tertiary studies and to master by the time they graduate (Moore, 2011).
Critical thinking definitions
As to the term itself, critical thinking and its definition have been long debated, partly due to variations in the terms used to define and describe critical thinking. For instance, critical thinking is seen as equivalent to higher-level thinking (Paul, 1992; Sternberg, 1987) or reflective thinking (Dewey, 1998; Norris & Ennis, 1989), or as a subcategory of higher-level thinking (Geertsen, 2003).
While there are numerous definitions of critical thinking, they fall under two categories; kernel and taxonomical definitions (Moore, 2011). Kernel definitions of critical thinking try to state the nature of critical thinking in a sentence or two (Moore, 2011). Some kernel definitions adhere to a positivist generalist approach to critical thinking, in which critical thinking is defined as a generic skill, and the critical thinker as independent of the context in which critical thinking skills are applied (e.g. Ennis, 1987; Siegel, 1988). For instance, Ennis (1987, p. 10) has defined critical thinking as “reasonable, reflective thinking that is focused on deciding what to believe or do”. Similarly, Siegel’s (1988, p. 127) definition of the critical thinker is “the individual who is appropriately moved by reasons”. Other kernel definitions adhere to a relativist approach to critical thinking, in which critical thinking is defined as a situated, contextual and domain-specific skill (e.g. McPeck, 1981; Paul, 1989). For instance, MacPeck (1981, p. 7) has defined critical thinking as the “appropriate use of reflective scepticism within the problem area under consideration”. Similarly, Paul (1989, p. 214) has described critical thinking as “disciplined, self-directed thinking which exemplifies the perfection of thinking appropriate to a particular domain of thinking”. Nonetheless, Kernel definitions of critical thinking often overlap, as these definitions address the key aspect of critical thinking which is making judgement of some sort (Davidson, 1998; Moore, 2011).
On the other hand, taxonomical definitions of critical thinking outline a range of skills and sub-skills which constitute the activity of critical thinking (Moore, 2011). Some taxonomical definitions are framed in terms of hierarchical levels, also called phases such as Bloom’s taxonomy of cognitive domain (Bloom, Engelhart, Furst, Hill, & Krathwohl, 1956). In these definitions, different critical thinking skills are considered as being at the higher or lower levels on a linear scale based on the degree of abstraction required in each level. The critical thinking skills at the lower levels are less cognitively complex, while the critical thinking skills at the higher levels demand deeper and more complex critical engagement. Unlike kernel definitions, taxonomical definitions are formed not only to clarify the concept of critical thinking, but also to create a framework for teaching and assessing critical thinking. For instance, Bloom’s taxonomy of cognitive domain has been widely used to describe and evaluate critical thinking in educational settings such as in higher education.
Computer-supported collaborative learning (CSCL)
With higher education increasingly delivered in blended learning modes that are both offline and online (Amhag & Jakobsson, 2009; Lee, 2009), a line of research has emerged that investigates whether demonstrations of critical thinking are present in computer-supported and online learning contexts (e.g. McLoughlin & Mynard, 2009; Richardson & Ice, 2010; Schellens, Van Keer, De Wever, & Valcke, 2009). Such an investigation is called a search for “transversal relationships” (Kern, 2006, p. 202), which is an investigation of the transferability of a learning skill from one communicative modality and context to another.
Furthermore, parallel to the shifts in education towards social theories of learning, technology-enhanced learning research has also shifted its focus to collaborative learning, and to how online learning tools correspond with the broader ecological context that influences learning (Warschauer, 1998). Computer-Supported Collaborative Learning (CSCL) research was born out of this shift in focus on human cognition and learning. Embedded in the macro level of society, the meso level of educational institutions, and the micro level of classroom and task design, CSCL research investigates collaborative learning processes delivered via computers and the Internet.
One main tenet of CSCL is that learning takes place through group interaction and computer mediation (Chapelle, 2001; Stacey, 2005; Stahl, 2006). Koschmann, Hall, and Miyake (2002) have stated that “CSCL is a field of study centrally concerned with meaning and the practices of meaning making in the context of joint activity, and the ways in which these practices are mediated through designed artifacts” (p. 18). One mediating artifact is “the computer software with which a learner interacts in addition to other learners who collaborate in the same room or from remote locations through networked computers” (Chapelle, 2001, p. 32). It is argued that CSCL tools not only provide a platform for group members’ active co-construction of knowledge (i.e. group-mediated cognition), but can also serve as significant mediating tools for such knowledge construction (i.e. computer-mediated cognition) (Garrison, Anderson, & Archer, 2001; Gunawardena, Lowe, & Anderson, 1997; J. Smith, 1994; Stahl, 2006).
Asynchronous discussion forums (ADFs)
Email, asynchronous discussion forums, blogs and wikis are among the CSCL tools used to complement face-to-face classroom interactions. Asynchronous discussion forums (ADFs) in particular are used frequently in higher education (Amhag & Jakobsson, 2009; Dringus & Ellis, 2010) because they are text-based and deemed suitable for serious academic discussion (Motteram, 2001). Learners’ engagement in asynchronous online discussions is a form of computer-supported group-mediated collaborative activity in which an electronic medium is used (Deloach & Greenlaw, 2005). One tenet of CSCL is that individuals visibly demonstrate what they have learnt and what they are learning in the process of collaboration because individuals display to each other their understanding of the meaning that is being constructed and negotiated in the online discussions (Stahl, Koschmann, & Suthers, 2006). Therefore, it is proposed that utterances produced during interactions such as online postings produced on ADFs can be considered valuable data for measuring learning (Gunawardena, et al., 1997; Stahl, et al., 2006).
ADFs are proposed to have the potential to take discussions to a more critical level, since through online discussions students can create a discourse community where they negotiate with one another in an extended period of time (Land, Choi, & Ge, 2007). Such negotiations are claimed to have the potential to lead to cognitive conflict, which in turn, can trigger exploratory talk (Song, 2008). Similarly, Guiller (2008) argues that “the increase in the time available to think and consult sources of information before responding in an asynchronous discussion may give rise to an increase in the use of formal, research-based evidence and the quality of critical thinking” (p. 188). Moreover, ADFs are proposed to provide a platform for expressing multiple perspectives, negotiating meaning, understanding knowledge gaps and resolving issues (Haavind, 2006; Land, Choi, & Ge, 2007). Therefore, due to the specific features of ADFs, it is contended that students can benefit from extended learner-learner interactions on ADFs in ways not feasible in face-to-face classrooms (Ling, 2007).
When incorporating CSCL tools such as ADFs into higher education curriculums, efforts have been made to promote critical thinking, although having students respond to these efforts by engaging in critical thinking has proven to be difficult (e.g. Derry, Gance, Gance, & Schlager, 2000; Derry, et al., 2004). In response to these difficulties, researchers have tried to investigate ways to more successfully engage students in critical thinking while using CSCL tools in higher education contexts. Previous research has highlighted a number of potential drawbacks of ADFs, such as feeling of isolation (Kalman, Ravid, Raban, & Rafaeli, 2006; Zhang & Kenny, 2010), information overload (Kalman, et al., 2006; Zhang & Kenny, 2010) and absence of immediate feedback (Herring, 1999). Furthermore, while online communication tools such as ADFs provide platforms for interaction, they do not guarantee that interaction takes place (Gray & Tatar, 2004), and if interaction does take place, there is no guarantee that it will be critical and constructive.
Interactions on ADFs and their potential to promote critical thinking can be explained in light of Vygotsky’s (1978) sociocultural theory. Sociocultural theory highlights the importance of collaborative knowledge construction as a result of interaction with instructors, peers and tools in cognitive development and learning.
According to sociocultural theory, learning is social, scaffolded, and tool-mediated. Individuals do not learn in isolation; cognitive development first takes place at a social level, scaffolded by peers and more knowledgeable others, and is then internalised at an individual level. Since, according to sociocultural theory, cognitive development is socially-situated and socially-constructed, it is affected by sociocultural factors such as cultures of learning and teaching, and the learning tools used.
Moreover, based on sociocultural theory, human cognition is mediated (Lantolf, 2000; Vygotsky, 1978). Mediating tools are either physical (e.g. hammer, computer) or symbolic (e.g. numbers, language). Vygotsky (1978) highlighted the significant mediating role of language in cognitive development. Linguistic activity (e.g. speaking and writing) plays an important role in human mental activities (e.g. rational thought, learning). Besides language, the group serves as a mediating tool for developing critical thinking, as the thinking of an individual is affected by the thinking of others in the group, a thinking process referred to as group-mediated cognition (J. Smith, 1994).
Different lines of research have investigated different aspects of teaching and learning in higher education through the lens of sociocultural theory. One implication of sociocultural theory for higher education has been the increase in implementing collaborative learning, as collaborative learning is linked to the development of critical thinking skills required of a higher education graduate (Powell & Kalina, 2009; Roberts, 2005; Stahl, et al., 2006).
One active line of research that is grounded in sociocultural theory is CSCL research. As discussed earlier, CSCL research underlines the role of computer- and Internet-mediated collaborative activity in fostering learning. The proposed potential of online communication tools, which create a platform for co-construction of knowledge, in promoting positive learning experiences and outcomes acts as an incentive for CSCL research to expand.
Gaps in the previous research
Expansion of CSCL research and the increase in the incorporation of online tools in higher education have resulted in the production of a vast body of research on the effectiveness of ADFs in general and their potential for fostering tertiary level students’ critical thinking in particular (e.g. Lee, 2009; McLoughlin & Mynard, 2009; Richardson & Ice, 2010). However, there are two key gaps in the literature that need to be addressed:
- Previous research has mainly focused on the critical thinking of the individual; the investigation of critical thinking when a group works together, that is collaborative critical thinking (CCT), has been generally overlooked. This is partly due to the conceptualisation of critical thinking as a solitary activity by cognitive and educational psychologists. CSCL, which is a pedagogical approach grounded in social theories of learning, adheres to the notion that cognitive development occurs at both the level of the group and the level of the individual. However, the main focus of CSCL research to date has been on individual critical thinking; that is on how the individual functions within the group. In these studies, the critical thinking of the group is viewed as the sum of the critical thinking of each individual within the group. This approach has been criticised as reductionist by some researchers such as Stahl, et al. (2006) who have underlined the need for analysing both the individual’s and the group’s thinking in CSCL research.
- Few studies have investigated the critical thinking processes of the individual and of the group in a single study to present a comprehensive picture of CCT processes on ADFs. In fact, Schrire’s (2004) research is one of the few, if not the only, research that is fully grounded in Vygotsky’s sociocultural theory that highlights “the complementary nature of individual and socially distributed cognition” (p. 484).
To address these gaps, I examined the demonstration of CCT on ADFs in a higher education learning context. The scope of this research was limited to a graduate blended subject over two 12-week semesters at an Australian university. I investigated the demonstration of CCT on ADFs through content-analysis of asynchronous online discussions as students discussed different topics on 20 weekly ADFs over the course of two semesters. Additionally, through semi-structured interviews with the student and instructor participants, I searched for potential factors that affected the demonstration of CCT on ADFs.
In order to aid in the future replication of the above study and in the hope of promoting more rigorous debate regarding the challenges of researching collaborative knowledge construction in CSCL contexts, I present in this paper the ontological and methodological challenges I encountered while conducting this study. The paper also discusses a number of solutions to these challenges which proved indispensible to conducting the above study.
Challenges in Conducting CSCL Research
CSCL research came into being in the late 1980s and early 1990s (Stahl, et al., 2006), and as such is a relatively new line of inquiry. Therefore, at the ontological level, CSCL research faces the issue of variations in theoretical perspectives on what collaborative learning is and how it should be conceptualised (Stahl, et al., 2006). Challenges in conceptualising collaborative learning result in challenges in measuring collaborative learning. Moreover, methodologies adopted in CSCL research are often data-driven and retrospective, resulting in myriads of context-specific practices and hypotheses that are often left unattested, and are thus difficult to replicate (Strijbos & Fischer, 2007). Difficulty in replication also stems from such studies focusing mainly on research outcomes, overlooking the importance of providing methodological specifics of how outcomes are obtained.
The above concerns with CSCL research need to be addressed. This requires the promotion of methodological debates as a collaborative scientific endeavour in the field. As a means to contribute to the debates, I discuss in this paper, the ontological and methodological challenges encountered in conducting the current CSCL research, and suggest a number of solutions to these challenges.
Ontological Challenge and Solution
At an ontological level, which is the level concerned with the nature of a social reality, interpretivists view the social reality as internal to the individual (Baxter & Jack, 2008). Interpretivists view the world as a sociocognitive construct in which there are multiple realities shaping a unified whole. This is in contrast with the positivist approach that envisions social reality as external to the individual (Bloomberg & Volpe, 2012). Therefore, from an interpretivist point of view, the social world is understood by taking into account the frame of reference of individuals acting in that social world (Baxter & Jack, 2008).
In the current study, the ontological challenge was in conceptualising and defining the social reality under examination, being CCT. The challenge stemmed from the fact that the majority of the previously proposed definitions of critical thinking have addressed critical thinking as an inherently solitary activity. In fact, the individual nature of critical thinking is considered a given in the majority of the descriptions of critical thinking. None of the definitions have conceptualised critical thinking, either explicitly or implicitly, as a collaborative activity. For instance, Geertsen (2003, p. has highlighted the individual nature of reflective thinking by referring to the “aha! experience” as a result of reflective thinking that “comes during moments of solitude when one is not pressing to find an answer due to the uncertain and elusive nature of ill-structured problems”.
Moreover, as mentioned earlier, the critical thinking taxonomies have also served as frameworks for teaching and assessing critical thinking in higher education. Therefore, it is not surprising that the conceptualisation of critical thinking as an individual activity has become normalised in higher education (M. Moore, 1993).
In order to achieve the aim of this study, which was to examine CCT demonstration on ADFs, it was crucial to reconceptualise and redefine critical thinking in a way that would correspond with the kind of critical thinking that potentially occurs in a group learning context such as on ADFs. Reconceptualising and redefining critical thinking in this study required extensive review of the literature on the social aspect of cognition.
The social aspect of cognition is highlighted by a number of scholars (e.g. Vygotsky, 1978; Yukawa, 2006), who have opposed isolationist views of thinking, and who have maintained that a conceptual transformation towards a view of critical thinking as socially distributed and outwardly directed is necessary (Thayer-Bacon, 2000). For instance, Ennis (1996) wrote that thinking critically which is considered an attribute of an individual can justifiably be attributed to group cognitive engagement and decision making. Similarly, Bailin, Case, Coombs, and Daniels (1999) described responding constructively to others in group discussions as a critical thinking ability. Facione (2000, p. 72) has argued that critical thinking is not an individual activity and “at times the complexities of good CT (critical thinking) [description added] are evident when CT is carried on by groups”.
There have also been some attempts to define critical thinking that occurs in groups (i.e. CCT). Critical thinking that occurs when groups interact with each other is interchangeably called group critical thinking (Schamber & Mahoney, 2006), co-reflection (Yukawa, 2006), collaborative critical thinking (Olivares, 2005; Yukawa, 2006) or simply described without any labels. CCT is conceptualised differently in different studies. In some studies, CCT is defined as the end product of the group’s collaborative activity (product-oriented definitions), while in other research CCT is defined as the process of the group’s collaborative activity (process-oriented definitions). Table 1 lists the CCT definitions found in the literature. In the current study, these definitions became the initial framework for the conceptualisation of CCT and for the investigation of whether and how CCT is demonstrated on ADFs.
Table 1. Definitions of CCT
As shown in Table 1, the proposed CCT definitions fall into two categories of product-oriented and process-oriented. While product-oriented definitions of CCT aid in understanding what CCT is, they overlook the process of CCT, the understanding of which has pedagogical implications for fostering CCT in higher education. Moreover, considering that the setting of the current study was higher education with its focus shifting towards both the process and the product of critical thinking (Derry, et al., 2004), I found conceptualising CCT that captured both the process and product of the phenomenon the most suitable. Therefore the process-oriented definitions of CCT were deemed most relevant to the conceptualisation of CCT in the current study.
To arrive at an operational definition of CCT that is grounded in sociocultural theory, I conceptualised CCT as a kind of collaborative reasoning activity that is mediated by language. This conceptualisation was informed by a) Ferguson’s (2009) and Mercer and Littleton’s (2007) definitions of cumulative and exploratory talk, b) Yukawa’s (2006) categories of co-reflection, and c) Jenlink and Carr’s (1996) description of interactive messages (i.e. dialog, dialectic and construction). Accordingly, the process of CCT starts with building on each individual’s knowledge and the knowledge of the group through information exchange (i.e. cumulative talk), followed by challenging ideas through argumentation, evaluating evidence, and discussing possible solutions to the problem at hand (i.e. exploratory talk) (Ferguson, 2009; Mercer & Littleton, 2007). In contrast to the conceptualisation of critical thinking in higher education as predominantly cognitivist and individualistic, in this conceptualisation, CCT is considered a social activity.
After overcoming the initial ontological challenge of conceptualising critical thinking as it occurs in a CSCL context, I postponed defining CCT, until after I analysed its demonstration by a group of graduate level students on a series of ADFs. This is because there were not sufficient frequently-researched and tested definitions in previous CSCL research to base the current study’s data analysis on. However, it was necessary to use a content-analysis model that measured the demonstration of CCT on ADFs in accordance with the aforementioned CCT conceptualisation. This led to the second challenge, which was at the methodological level.
Methodological Challenge and Solution
In line with social theories of learning, such as Vygotsky’s (1978) sociocultural theory, CSCL research seeks evidence of development “in the discourse that occurs in the collaborative environment” (Chapelle, 2001, p. 32). Hence, reliance on qualitative content analysis is prevalent in CSCL research (e.g. Henri, 1992; Mason, 1992). Content analysis is a kind of textual analysis (Hara, Bonk, & Angeli, 2000), and is “a technique to extract desired information from a body of material…by systematically and objectively identifying specified characteristics of the material” (Smith, 2000, p. 314). In other words, content analysis is a methodology to analyse and categorise qualitative data (i.e. text or different forms of data transcribed into text).
While content analysis is frequently used in CSCL research, it is not often well-explained. This is in contrast to the clear guidelines available for constructing survey questionnaires and interviews (Strijbos & Fischer, 2007). Furthermore, while using multiple coding schemes or a synthetic coding scheme can strengthen credibility of content analysis findings (De Wever, Schellens, Valcke, & Van Keer, 2006), only a few studies have used more than one or a synthesis of coding schemes for content-analysis of ADFs in search for indicators of critical thinking (e.g. Schellens, et al., 2009; Schrire, 2004). Most studies have used a single content-analysis coding scheme (e.g. Kol & Schcolnik, 2008; McLoughlin & Mynard, 2009; Richardson & Ice, 2010), or have not used any coding scheme (e.g. Lee, 2009; Sloffer, Dueber, & Duffy, 1999). Therefore, the methodological challenge in the current study was in constructing a coding scheme, (also referred to as content analysis model), to measure the demonstration of CCT on ADFs.
To address the aforementioned methodological challenge, I developed a synthetic coding scheme for analysing asynchronous online discussion postings. Specifically, I decided to create a synthetic framework to enable detecting the demonstration of CCT on ADFs more readily. For this purpose, I extensively reviewed the literature to evaluate a) the theoretical compatibility of the available coding schemes with the present study, b) the available coding schemes’ proposed critical thinking conceptualisation (i.e. individual critical thinking or collaborative critical thinking), and c) the available coding schemes’ inter-rater reliability index (Buraphadeja & Dawson, 2008).
The extensive review of the literature showed that there are a number of coding schemes available for measuring the demonstration of critical thinking in computer conferencing (e.g. Henri, 1992; Greenlaw & DeLoach, 2003; Perkins & Murphy, 2006). Three of the content analysis models for measuring the demonstration of critical thinking on ADFs have been used and modified the most (Buraphadeja & Dawson, 2008). These content analysis models are Indicators of Critical Thinking (ICC) by Newman, Webb and Cochrane (1995), Interaction Analysis Model (IAM) by Gunawardena, et al. (1997), and Practical Inquiry Model (PIM) by Garrison, et al. (2001).
Among these three most prevalently used coding schemes (i.e. ICC, IAM, & PIM), I selected IAM as the basis for the content analysis of the ADF postings in the current study. To elaborate, the majority of the indicators in ICC measure individual critical thinking. Therefore, ICC was discarded, since it does not code for CCT, a key concept in this study. However, the indicators in IAM and PIM predominantly measure CCT. IAM and PIM share a number of attributes:
- They predominantly measure the demonstration of collaborative critical thinking.
- In contrast with ICC which is product-oriented and categorical, IAM and PIM are process-oriented and hierarchical, which means that the higher phases of critical thinking are built on the lower ones.
- There is a large area of overlap among phases and indicators of IAM and PIM.
In fact, both PIM and IAM have been successfully used for measuring the demonstration of critical thinking on ADFs as they are frequently used, modified and tested by different researchers(Buraphadeja & Dawson, 2008). However, there are two key differences between PIM and IAM that resulted in the selection of the latter for use in the current study:
- While both PIM and IAM code for the five critical thinking phases of questioning, analysing, synthesising, evaluating, and decision making, IAM includes five distinct indicators for each critical thinking phase; in IAM, evaluation and decision making phases are presented as two separate phases, each with detailed indicators making it more feasible to distinguish the two skills and to code for them in ADF postings.
- Unlike PIM, which only offers broad descriptions, IAM lists more specific indicators for each critical thinking phase. Some indicators in PIM are either too broadly defined or include ambiguous words; words such as systematically, tentative, and vicarious application. What is a tentative hypothesis? What distinguishes a justified and tentative argument from a justified but not tentative argument? These ambiguities could make coding of online postings subjective while the use of less ambiguous and more specific indicators in IAM could decrease the chance of subjective coding.
Consequently, IAM was selected as the coding scheme in this study to reduce subjective coding of online discussion postings. Another reason for selecting IAM for the content analysis of ADFs in this study was IAM’s theoretical alignment with social theories of learning such as Vygotsky’s (1978) sociocultural theory, and with the aforementioned conceptualisation of CCT. As mentioned earlier, the way I conceptualised CCT was informed by the process-oriented definitions of CCT that defined different types of interactive message types.
As outlined in Table 2, IAM codes for the three interactive message types of ‘dialog’, ‘dialectic’, and ‘construction’ (Jenlink & Carr, 1996), as well as ‘cumulative talk’ and ‘exploratory talk’ (Mercer & Littleton, 2007). Another type of inquiry-based group-mediated thinking manifested in IAM is what has been called ‘challenge and explain’ (Curtis & Lawson, 2001). Deloach and Greenlaw (2005) have described the process of critical thinking in online discussions as constantly being triggered by ‘challenge and explain’ inquiries: “In electronic discussions…students are constantly challenged to improve their answers by providing relevant backing for their opinions. Simply put, there appears to be a critical thinking spillover effect” (p. 150).
More specifically, as outlined in Table 2, in IAM, at the questioning level, which is the brainstorming and problem identification level, participants engage in ‘dialog’ and ‘cumulative talk’. At the analysing level, where dissonance among participants is shared and explained, ‘challenge and explain’ is likely to occur provided participants continue clarifying and mitigating dissonance. ‘Developing dialectic conversation’ and ‘developing exploratory talk’ also occur at this level when participants support their statements of disagreement and extended statements with analytic and factual information. The last three critical thinking levels of synthesising, evaluating, and decision making are those in which participants engage in collective construction of knowledge by integrating ideas from different sources including other members’ statements. ‘Exploratory talk’ and ‘dialectic’ are also evident here when participants test the collectively proposed solutions or statements against different contexts, and decide on the applicability of the collectively proposed solutions or statements.
Table 2. CCT levels in the modified IAM and interactive message types
Moreover, Gunawardena, et al. (1997) constructed IAM in light of social theories of learning, as the researchers emphasised that IAM aims to evaluate the “learning process taking place among the group of participants, rather than to assess individual student performance” (p. 405). IAM measures the kind of critical thinking that occurs within the group and among group members through interaction. This is in line with sociocultural theory which views learning as a social activity.
Furthermore, Gunawardena, et al. (1997) have argued that lower and higher mental functions can be observed in CSCL activities, depending on the groups’ degree of critical engagement in the activity. While not labelled as such, IAM categorises CCT into higher and lower level categories with the lower levels consisting of questioning and analysing, and the higher levels consisting of synthesising, evaluating and decision making with cognitive complexity increasing at each level. In IAM, questioning is defined as raising questions, asking for clarifications and collaboratively identifying possible factors relevant to the problem. Analysing is defined as identifying and negotiating areas of disagreement among online members, and advancing arguments. Synthesising is defined as bringing together a range of relevant ideas presented on the ADF. Evaluating is defined as asking oneself and others whether the solution works, and whether it has utility in certain contexts. Decision making is defined as consensually arriving at new statements or solutions and applying them to a given task or a real-world context.
However, before using IAM in the current study, a number of modifications were applied to adapt this content analysis model to the main aim of this study (i.e. measuring CCT on ADFs). The main modifications are explained below:
- Since the aim of this study was to examine the demonstration of CCT rather than merely the interactions on the ADFs, those indicators in IAM which coded for interaction but did not code for critical thinking were modified. These indicators were modified by adding keywords from relevant indicators present in other coding schemes such as PIM. The modified indicators are as follows and the added keywords appear in italics. ‘Relevant statement of observation or opinion’ and ‘substantiated statement of agreement from one or more participants’ at the questioning level, as well as ‘identifying and stating areas of disagreement with support’ at the analysing level.
- To simplify referencing during the inter-rater reliability process and during the reporting of findings, each CCT level in the modified IAM was labelled, which corresponded to the descriptions offered in IAM for the different CCT levels. The labels for each CCT level from low to high were respectively questioning, analysing, synthesising, evaluating, and decision making.
Therefore, for the qualitative content analysis of online discussion postings in the current study, the modified IAM (see Table 3) was used to measure the demonstration of CCT on the ADFs.
Table 3. Modified IAM
During the content analysis process, I came across a number of comments on the ADFs that were not codable based on the modified IAM. This was due to these comments not containing indicators of CCT, and instead containing indicators of redundancy and off-task. Comments on the ADFs which contained redundancy were those that only paraphrased other online participants’ comments without adding to the discussion. Comments on the ADFs which contained Off-task were those that were not relevant to the online discussion topic. Table 4 presents the list of indicators for redundancy and off-task, followed by sample excerpts from the ADFs for illustration purposes.
Table 4. Indicators of redundancy and off-task on ADFs
While IAM’s conceptual and theoretical alignment with the notion of critical thinking as a collaborative activity made it a suitable tool for measuring participation and interaction, the modified IAM proved to be a suitable tool for identifying CCT indicators in text-based online discussions because all of its indicators measure demonstrations of CCT. Moreover the modified IAM had a high inter-rater reliability index. Inter-rater reliability in the first round of coding was 70.83 per cent. After negotiating discrepancies and ambiguities, the second round of coding resulted in an acceptable percentage of 83.72 inter-rater reliability.
The modified IAM can not only serve as an analytic tool for researchers and as a formative assessment tool for educators to measure the demonstration of CCT in CSCL contexts, but also as a learning tool for students to guide their CCT demonstration online. In higher education classes where CSCL tools are used for critical discussions, students can evaluate their CCT demonstration against this coding scheme and make necessary efforts to participate more collaboratively and critically. The CCT indicators in the modified IAM can provide a clearer idea as to how critically students need to approach the discussions in computer conferencing.
What needs to be noted here is that studies such as this do not analyse thought processes, rather manifestations of thought processes. It is important to realise that simply because CCT is not outwardly demonstrated, it does not mean that CCT has not taken place. As Schallart, Reed and D-Team (2003) have stated, “students learn not only by posting comments in the discussion but also by reading other students’ and their teacher’s comments” (p. 109). CCT that is not outwardly expressed is called tacit negotiation (Gunawardena, et al., 1997) or tacit co-reflection (Yukawa, 2006). Therefore, one inherent limitation of studies on critical thinking is that they can only capture the demonstrated cognitive behaviours without being able to observe internal cognitive processes (Arend, 2009; Arnold & Ducate, 2006).
However, while it is difficult to measure learners’ tacit negotiations as they are not readily accessible, through qualitative surveys, retrospective commentary, and introspective measures, researchers can arrive at an understanding of learners’ internal critical thinking before, while and after participating in computer conferencing such as on ADFs. Through qualitative surveys, researchers can also understand how the internal critical thinking processes are manifested in written communication such as those carried out on ADFs. Therefore, qualitative survey-based research can more meticulously examine whether and to what extent tacit CCT is taking place, and to investigate what factors hinder or facilitate such collective critical thinking processes.
Operational definition of CCT in CSCL Contexts
Informed by the findings of the current study, some of which are reported in Ghodrati and Gruba (2011), I propose an inductive and hierarchical yet cyclical definition of critical thinking that attempts to capture both the process and the product of CCT:
The overt and tacit interaction between two or more individuals which involves collectively questioning, analysing, synthesising, evaluating and making decisions in order to build the collective knowledge of the group and the knowledge of the individuals in the group.
To elaborate, CCT occurs both actively and tacitly (Gunawardena, et al, 1997; Yukawa, 2006). In a dialogue, the reflective self seeks feedback, shares ideas and critically addresses the ideas shared by others through explicit interaction. This is called active co-reflection (Yukawa, 2006) or overt negotiation (Gunawardena, et al., 1997). However, the process of CCT is not always active/overt. In line with Lantolf’s (2000) proposition, linguistically-mediated cognition is social even when one is acting alone. In other words, learners also engage in CCT through tacit co-reflection (Yukawa, 2006) or tacit negotiation (Gunawardena, et al., 1997) by seeking “responses to others who are brought to mind through reading, memories of previous interactions, or vicarious experience” (Yukawa, 2006, p. 207). In contrast with active co-reflection which can be observed and investigated in online postings, tacit co-reflection is not readily accessible or observed. Investigating tacit CCT requires enquiring beyond postings on ADFs. It requires eliciting information about the individual’s and the group’s thought processes before, while and after participating on ADFs.
Another key term in the proposed definition of CCT is interaction, also called active participation (Mercer & Wegerif, 1999). In contrast with participation, which in computer-mediated and online communication platforms such as on ADFs is defined as the posting of comments, interaction is defined as the posting of messages that either explicitly or implicitly respond to others’ messages (Schrire, 2004, 2006). Interaction can be instructor-centred or student-centred (i.e. online participants addressing their comments to only one online participant), both of which are considered individualistic. Interaction can also be synergistic (i.e. online participants addressing more than one online member) which is considered collaborative (Schrire, 2004). However, not all collaborative interactions in a group involve critical thinking. Collaborative interaction that involves questioning, analysing, synthesising, evaluating and decision making is the kind of interaction that demonstrates different levels of CCT (Garrison, et al., 2001; Gunawardena, et al., 1997). Such interaction is exploratory (e.g. Ferguson, 2009; Mercer & Littleton, 2007), and when triggered by cognitive conflict on the part of one or more members results in ‘challenge and explain’, provided it is shared actively/overtly with others. Such interaction is also constructive (Jenlink & Carr, 1996), in the sense that it builds the collective knowledge of the group as the group discusses and negotiates issues (i.e. critical thinking at the group level), and also builds on the already established knowledge of the individual as a result of tacit and active co-reflection (i.e. critical thinking at the individual level).
This paper presented the ontological and methodological challenges encountered in conducting a qualitative case study of the demonstration of CCT on ADFs in a blended higher education learning context. At the ontological level, I discussed the challenges in conceptualising and defining critical thinking that occurs when a group works together (i.e. CCT). At the methodological level, I presented the challenges in constructing a synthetic coding scheme to measure the demonstration of CCT on ADFs. In addition, I discussed ways to overcome these challenges. I arrived at a number of solutions to the ontological and methodological challenges encountered in the current study by a) reconceptualising critical thinking as a collaborative activity in CSCL contexts, b) constructing a synthetic coding scheme for measuring CCT on online collaborative learning platforms such as on ADFs, and c) proposing an operational definition of CCT in CSCL contexts.
It should be noted that the synthetic content analysis model and the operational definition of CCT proposed in this paper are based on the findings of research in a specific learning setting (i.e. a blended subject in higher education) using a specific online communication tool (i.e. ADF). While the proposed content analysis model for measuring the demonstration of CCT on ADFs and the proposed CCT definition were closely informed by the social conceptualisation of cognition suggested in the previous research, further research should investigate the applicability of both the definition and the content analysis model to a) other learning contexts where ADFs are used, and b) the learning contexts where other CSCL tools such as wikis, blogs and synchronous chat are used.
Amhag, L., & Jakobsson, A. (2009). Collaborative learning as a collective competence when students use the potential of meaning in asynchronous dialogues. Computers & Education, 52(3), 656-667.
Arend, B. (2009). Encouraging critical thinking in online threaded discussions. The Journal of Educators Online, 6(1), 1-23.
Arnold, N., & Ducate, L. (2006). Future foreign language teachers’ social and cognitive collaboration in an online environment. Language Learning & Technology, 10(1), 42-66.
Bailin, S., Case, R., Coombs, J. R., & Daniels, L. B. (1999). Conceptualizing critical thinking. Journal of Curriculum Studies, 31(3), 285–302.
Baxter, P., & Jack, S. (2008). Qualitative case study methodology: Study design and implementation for novice researchers. The Qualitative Report, 13(4), 544-559. Retrieved from http://www.nova.edu/ssss/QR/QR13-4/baxter.pdf
Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). Taxonomy of educational objectives: the classification of educational goals. London: Longman Group.
Bloomberg, L. D., & Volpe, M. (2012). Completing your qualitative dissertation: a road map from beginning to end (2nd ed.). Los Angeles, London, New Delhi, Singapore, Washington DC: Sage.
Buraphadeja, V., & Dawson, K. (2008). Content analysis in Computer-Mediated Communication: Analyzing models for assessing critical thinking through the lens of social constructivism. The American Journal of Distancde Education, 22, 130-145.
Chapelle, C. A. (2001). Computer applications in second language acquisition: Foundations for teaching, testing and research. Cambridge, UK: Cambridge University Press.
Curtis, D., & Lawson, M. (2001). Exploring collaborative online learning. Journal of Asynchronous Learning Networks, 5(1), 21-34.
Davidson, B. W. (1998). Comments on Dwight Atkinson’s “A Critical Approach to Critical Thinking in TESOL”: A Case for Critical Thinking in the English Classroom. TESOL Quarterly, 32(1), 119-123.
De Wever, B., Schellens, T., Valcke, M., & Van Keer, H. (2006). Content analysis schemes to analyze transcripts of online asynchronous discussion groups: A review. Computers & Education, 46(1), 6-28.
Deloach, S. B., & Greenlaw, S. A. (2005). Do electronic discussions create critical thinking spillovers? Contemporary Economic Policy, 23(1), 149-163.
Derry, S. J., Gance, S., Gance, L. L., & Schlager, M. (2000). Toward assessment of knowledge building practices in technology-mediated work group interactions. In S. P. Lajoie (Ed.), Computers as cognitive tools volume two: No more walls (pp. 3-29). Mahwah, NJ: Erlbaum.
Derry, S. J., Seymour, J., Steinkuehler, C., Lee, J., & Siegel, M. A. (2004). From ambitious vision to partially satisfying reality: An evolving socio-technical design supporting community and collaborative learning in teacher education. In S. A. Barab, R. Kling & J. H. Gray (Eds.), Designing for virtual communities in the service of learning (pp. 256-295). Cambridge: Cambridge University Press.
Dewey, J. (1998). Analysis of reflective thinking. In L. A. Hickman & T. M. Alexander (Eds.), The Essential Dewey: Volume 2 Ethics, Logic, Psychology (pp. 137-150). Bloomington, IN: Indiana University Press.
Dringus, L. P., & Ellis, T. (2010). Temporal transitions in participation flow in an asynchronous discussion forum. Computers & Education, 54, 340-349.
Ennis, R. (1987). A taxonomy of critical thinking abilities and dispositions. In J. Baron & R. Sternberg (Eds.), Teaching thinking skills: Theory and practice (pp. 9-26).
Ennis, R. (1996). Is critical thinking culturally biased? Paper presented at the 16th Annual International Conference on Critical Thinking and Educational Reform.
Facione, P. A. (2000). The disposition toward critical thinking: Its character, measurement, and relationship to critical thinking skill. Informal Logic, 20(1), 61-84.
Ferguson, R. (2009). The Construction of Shared Knowledge through Asynchronous Dialogue. The Open University.
Garrison, D., Anderson, T., & Archer, W. (2001). Critical thinking, cognitive presence, and computer conferencing in distance education. American Journal of Distance Education, 15(1), 7-23.
Geertsen, H. R. (2003). Rethinking thinking about higher-level thinking. Teaching Sociology, 31, 1-19.
Ghodrati, N., & Gruba, P. (2011). The role of asynchronous discussion forums in the development of collaborative critical thinking. In G.Williams, P. Statham, N. Brown, B. Cleland (Eds.), Changing Demands, Changing Directions. Proceedings ascilite Hobart 2011. (pp.437-451). http://www.leishman-associates.com.au/ascilite2011/downloads/papers/Ghodrati-full.pdf
Gray, J. H., & Tatar, D. (2004). Sociocultural analysis of online professional development: A case study of personal, interpersonal, community, and technical aspects. In S. A. Barab, R. Kling & J. H. Gray (Eds.), Designing for virtual communities in the servise of learning (pp. 404-435). Cambridge: Cambridge University Press.
Greenlaw, S. A., & Deloach, S. B. (2003). Teaching critical thinking with electronic discussion. The Journal of Economic Education, 34(1), 36-52.
Guiller, J., Durndell, A., Ross, A. (2008). Peer interaction and critical thinking: Face-to-face or online discussion? Learning and Instruction, 18, 187-200.
Gunawardena, C. N., Lowe, C. A., & Anderson, T. (1997). Analysis of a global online debate and the development of an interaction model for examing the social con struction of knowledge in computer conferencing. Journal of Educational Computing Research, 17(4), 397-431.
Haavind, S. (2006). Key factors of online course design and instructor facilitation that enhance collaborative dialogue among learners. Paper presented at the Annual meeting of the American Educational Research Association.
Hara, N., Bonk, C. J., & Angeli, C. (2000). Content analysis of online discussion in an applied educational psychology course. Instructional Science, 28, 115-152.
Herring, S. C. (1999). Interactional coherence in CMC. Journal of Computer-Mediated Communication Research, 4(4).
Henri, F. (1992). Computer conferencing and content analysis. In A. Kaye (Ed.), Collaborative learning through computer conferencing (pp. 117-136). Berlin: Springer-Verlag.
Jenlink, P., & Carr, A. A. (1996). Conversation as a medium for change in education. Educational Technology, 36(1), 31-41.
Kalman, Y. M., Ravid, G., Raban, D. R., & Rafaeli, S. (2006). Pauses and response latencies: A chronemic analysis of asynchronous CMC. Journal of Computer-Mediated Communication, 12, 1-23.
Kern, R. (2006). Perspectives on technology in learning and teaching languages. TESOL Quarterly, 40(1), 183-210.
Kol, S., & Schcolnik, M. (2008). Asynchronous forums in EAP: Assessment issues. Language Learning & Technology, 12(2), 49-70.
Koschmann, T., Hall, R., & Miyake, N. (Eds.). (2002). CSCL 2, carrying forward the conversation. Mahwah, N.J.: Lawrence Erlbaum.
Land, S., Choi, I., & Ge, X. (2007). Scaffolding online discussions to promote reflection and revision of understanding. International Journal of Instructional Media, 34, 409-418.
Lantolf, J. P. (Ed.). (2000). Sociocultural theory and second language learning. Oxford: Oxford University Press.
Lee, L. (2009). Scaffolding collaborative exchanges between expert and novice language teachers in threaded discussions. Foreign Language Annals, 42(2), 212-228.
Ling, L. H. (2007). Community of inquiry in an online undergraduate information technology course. Journal of Information Technology Education, 6, 153-168.
Mason, R. (1992). Evaluation methodologies for computer conferencing applications. In A. R. Kaye (Ed.), Collaborative learning through computer conferencing (pp. 105-116). Berlin: Springer Verlag.
McLoughlin, D., & Mynard, J. (2009). An analysis of higher order thinking in online discussions. Innovations in Education and Teaching International, 46(2), 147-160.
McPeck, J. (1981). Critical thinking and education. Oxford: Robertson.
Mercer, N. (2002). Developing dialogues. In G. Wells & G. Claxton (Eds.), Learning for Life in the 21st Century (pp. 141-153). Oxford: Blackwell Publishers.
Mercer, N., & Littleton, K. (2007). Dialogue and the Development of Children’s Thinking. London and New York: Routledge.
Mercer, N., & Wegerif, R. (1999). Is ‘exploratory talk’ productive talk? In P. Light & K. Littleton (Eds.), Learning with computers: Analysing productive interaction (pp. 79-101). London and New York: Routledge.
Moore, M. (1993). Theory of transactional distance. In D. Keegan (Ed.), Theoretical principles of distance education. New York: Routledge.
Moore, T. (2011). Critical thinking and language: the challenge of generic skills and disciplinary discourses. London; New York: Continuum.
Motteram, G. (2001). The role of synchronous communication in fully distance education. Australian Journal of Educational Technology, 17(2), 131-149.
Newman, D. R., Webb, B., & Cochrane, C. (1995). A content analysis method to measure critical thinking in face-to-face and computer supported group learning. Interpersonal Computing and Technology, 3(2), 56-77.
Norris, S., & Ennis, R. (1989). Evaluating Critical Thinking. Pacific Grove, CA: The Critical Thinking Press.
Olivares, O. (2005). Collaborative critical thinking: Conceptualizing and defining a new construct from known constructs. Issues in Educational Research, 15. Retrieved from http://www.iier.org.au/iier15/olivares.html
Paul, R. (1989). Critical thinking in North America: A new theory of knowledge, learning, and literacy. Argumentation, 3, 197-235.
Paul, R. (1992). Critical thinking. Santa Rosa, CA: Foundation for Critical Thinking.
Perkins, C., & Murphy, E. (2006). Indentifying and measuring individual engagement in critical thinking in online discussions: An exploratory case study. Educational Technology & Society, 9(1), 298-307.
Powell, K. C., & Kalina, C. J. (2009). Cognitive and social constructivism: Developing tools for an effective classroom. Educational Psychology, 130(2), 241-250.
Richardson, J. C., & Ice, P. (2010). Investigating students’ level of critical thinking across international strategies in online discussions. Internet and Higher Education, 13, 52-59.
Schallert, D. L., Reed, J. H., & D-Team, T. (2003). Intellectual, motivational, textual, and cultural considerations in teaching and learning with computer-mediated discussions. Journal of Research on Technology in Education, 36(2), 103-118.
Schamber, J. F., & Mahoney, S. L. (2006). Assessing and improving the quality of group critical thinking exhibited in the final projects of collaborative learning groups. The Journal of General Education, 55(2), 103-137.
Schellens, T., Van Keer, H., De Wever, B., & Valcke, M. (2009). Tagging thinking types in asynchronous discussion groups: Effects on critical thinking. Interactive Learning Environments, 17(1), 77-94.
Schrire, S. (2004). Interaction and cognition in asynchronous computer conferencing. Instructional Science, 32, 475-502.
Schrire, S. (2006). Knowledge building in asynchronous discussion groups: Going beyond quantitative analysis. Computers & Education, 46, 49-70.
Siegel, H. (1988). Educating reason: Rationality, critical thinking, and education. NY: Routledge in association with Metheun.
Sloffer, S. J., Dueber, B., & Duffy, T. M. (1999). Using asynchronous conferencing to promote critical thinking: two implementations in higher education. Paper presented at the 32nd Hawaii International Conference on System Sciences, Hawaii.
Smith, C. (2000). Content analysis and narrative analysis. In H. T. Reis & C. M. Judd (Eds.), Handbook of research methods in social and personality psychology (pp. 313–335). Cambridge: Cambridge University Press.
Smith, J. (1994). Collective intelligence in computer-based collaboration. Hillsdale, New Jersey: Lawrence Erlbaum.
Song, C. (2008). Educational games with blogs: Collaborating to motivate second language undergraduate critical thinking. Online Information Review, 32(5), 557-573.
Stacey, E. (2005). A constructivist framework for online collaborative learning: Adult learning and collaborative learning theory. In T. S. Roberts (Ed.), Computer-supported collaborative learning in higher education (pp. 140-161). London: Idea Group Publishing.
Stahl, G. (2006). Group cognition: Computer support for building collaborative knowledge. Cambridge, MA: The MIT Press.
Stahl, G., Koschmann, T., & Suthers, D. (2006). Computer-supported collaborative learning: An historical perspective. In R. K. Sawyer (Ed.), Cambridge handbook of the learning sciences (pp. 409-426). Cambridge, UK: Cambridge University Press.
Sternberg, R. (1987). Introduction. In J. Baron & R. Sternberg (Eds.), Teaching thinking skills (pp. 1-6). NY: W.H. Freeman.
Strijbos, J., & Fischer, F. (2007). Methodological challenges for collaborative learning research. Learning and Instruction, 17, 389-393.
Thayer-Bacon, B. J. (2000). Transforming critical thinking: Thinking constructively. New York: Teachers College Press.
Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press.
Warschauer, M. (1998). Researching technology in TESOL: Determinist, instrumental, and critical approaches. TESOL Quarterly, 32(4), 757-761.
Yukawa, J. (2006). Co-reflection in online learning: Collaborative critical thinking as narrative. Computer-Supported Collaborative Learning, 1, 203-228.
Zhang, Z., & Kenny, R. F. (2010). Learning in an online distance education course: Experiences of three international students. International Review of Research in Open and Distance Learning, 11(1), 17-36.
Nazanin Ghodrati PhD, is a lecturer in English for Academic Purposes at Trinity College-The University of Melbourne, Australia. Her PhD research was on the development of collaborative critical thinking in multicultural blended learning contexts in higher education. Her research interests are computer-supported collaborative learning, critical thinking and cross-cultural communication.