Program Evaluation Final Project EDTECH 505

The project for EDTECH 505 was a huge challenge and from what I understand, it strikes fear in all EDTECH students. We are expected to evaluate something real. I was no longer in the classroom and have no official affiliation with any schools. I know I was terrified about how I was going to accomplish the project.  I thought about all the resources I have and all the people I have had conversations with about things that may be appropriate for the project. A former boss, who is also like a mentor, came to mind. At the time, she was the superintendent of a Regional Occupational Program I used to work with. We had had a conversation about hybridizing some of the ROP courses. Although I have never had an opportunity to teach a hybrid course, I think they are the ultimate way education can be done. The goal of this project was to see if I created a course her teachers would actually use, how they thought about using the course. They did not use the course I wrote because it was summertime, but if they had shown interest in using it the following academic year, I would have hosted it for them.

Somewhere along the way I picked up the website URL hybridclassroom.net because I wanted to make Moodle classes for teachers. At one time I thought my first million would come from selling pre-made units or courses in a Moodle format to classroom teachers who wanted to hybridize their classes. The Certificate in Online Teaching I did at Merritt Community College taught me how to design and structure Moodle courses, and the class I took from Moodlerooms taught me how to be a Moodle administrator. I was motivated to try Dr. Fujii’s teachers out as possible clients because I really want to influence online learning in a positive way. No I did not charge them anything for what I did for them. I was earning course credit for the project so it would be wrong to accept any pay. What Dr. Fujii and I wanted to figure out is if her teachers who showed an interest in hybridizing their classes would be receptive to the course I designed for them in Moodle. Unfortunately I can’t put a link to the course because the ROP owns the materials I used. I can, however, link to the report I wrote for EDTECH 505.

Statistics Final Project

In EDTECH 562, our final project was to do a proposal where we did research that involved statistical analysis. Throughout the semester, lots of things changed for me, although I have a feeling I never fully grasped what the assignment was. Here is a link to the proposal I created for the course. I will probably also include a copy of my reflection at my Learning Log, too, because this was a phenomenal course in so many ways. I now understand chi-square!

Community of Inquiry paper

Constructivism Analyzed using the Community of Inquiry Model

This paper is being added to my learning log as I prepare for the portfolio. That is why it is out of sequence with the dates.

Melissa Getz Boise State University

12/9/2011

Abstract

The Community of Inquiry (CoI) is a paradigm established in 2000 by D. Randy Garrison, Terry Anderson, and Walter Archer that identifies how students learn in online asynchronous discussions. Over the past ten years several other researchers have joined in the scene by taking their own angles on how CoI can be measured, manipulated, applied, or just be useful. The Community of Inquiry is not unlike other learning theories where it has components that are made up of even more specific parts.  For CoI, the three main parts are called “presence” and within each presence there are variables that over the last few years have been used to measure the feasibility of the particular learning environment. First an overview of inquiry will be presented, followed by an introduction of the three presences along with some excerpts from literature.

What is Inquiry?

Inquiry is an educational practice that is based on the Constructivist learning theory. Constructivism has students create their own meaning for an assignment by having control of some of the direction or questioning involved with the lessons. Students derive meaning from the lesson instead of being told what to think or by being shown all of the answers. Hands-on learning is a constructivist approach because students manipulate objects to make meaning of the concepts or theories. Inquiry takes the hands-on approach one step further by giving students more control over the lesson, its content, or its direction. (Llewellyn, 2002) The Community of Inquiry is a manifestation of inquiry in an online written environment involving asynchronous discussions. Analysis of discussions using the Community of Inquiry model lets researchers measure the extent of inquiry that happens during said asynchronous discussions. Community of Inquiry is a learning theory because it analyzes learning that happens by means of interactive online discussions where ideas are developed over time via facilitation from each other or an instructor.

What is the Community of Inquiry?

Three presences were established in 2000 when Garrison, Anderson, and Archer published the paper, “Critical Inquiry in a Text-Based Environment: Computer Conferencing in Higher Education.”  The paper’s approach was to bring awareness to a new trend in higher education called computer-mediated communication (CMC).  They had done extensive research to delineate the components needed for learning to happen and they identified three core elements that needed to interact for a community to form where inquiry could happen.  It was in this paper that the social presence, the cognitive presence, and the teaching presence were born. Since year 2000, numerous education researchers have been wrestling with not only how to design asynchronous discussions so that all three presences can appear, but also with learning how to measure the efficacy of a discussion in the context of these three categories.

In a face to face classroom, students are given a question to explore, conduct the experiment with peers, and formulate conclusions. It is not feasible to do the same type of inquiry when students do not meet in person. However, inquiry can be done online using asynchronous discussions. For an online discussion to foster inquiry, it must have certain characteristics.  Since students make decisions about what to discuss, they formulate ideas and questions, and take ownership of what they say, inquiry is by definition involved. Students are given control over where discussions lead. If they go off topic, then other students will either point it out and steer them back on topic, or the off topic ideas will be ignored. Naturally, the participants have to be comfortable with the social setting and not be afraid to express their ideas. In addition, it helps if their ideas are received, considered, and processed by participants since having ideas validated carries a lot of weight in the online learning environment. If the social construct for the discussions is healthy, then a community of inquiry forms. Merely having a discussion forum does not guarantee it will evolve into a community where ideas are freely shared, respected, and altered.  There are other factors that need to exist. An instructor is still necessary to get the course started and to establish the safe discussion environment; however the students, when adequately engaged will create the community of inquiry because they will ask their peers questions to get others to think beyond their initial responses. Having students initiate questions that foster further discussion is a form of inquiry because the ideas and direction of the discussion are being generated by the students and not by the instructor.

Social Presence

The social presence component goes beyond netiquette to a more involved interaction with the other students in the class.  The three areas that comprise social presence are “Emotional Expression, Open Communication, and Group Cohesion” (Garrison, Anderson, & Archer, 2000, p. 89). The social presence is often the element that constitutes the human side of having a discussion. It comprises the aspects of the conversation where people relate to each other and develop a personal investment in what is transpiring online, which may keep them coming back to the discussion area. Maintaining a risk-free environment where people feel open to sharing ideas is crucial to establishing the social presence. A risk-free environment yields open communication, and the group cohesion develops as participants collaborate to produce a product from the synergy of their ideas.

Cognitive Presence

The cognitive presence is as it sounds; it deals with the thinking that happens as evidenced by the discussion. The four areas that constitute cognitive presence are, “Triggering Event, Exploration, Integration, and Resolution” (Garrison, et al., 2000, p.89).  First there is a triggering event which stimulates the participants’ curiosities. Students are engaged by a question or a dilemma they need to solve. They exchange information as they collaboratively explore possible solutions. These interactions lead to integrating ideas into a thesis or possible explanation to the question or dilemma. Finally the group applies their ideas to a new scenario or tries out their proposal to determine its validity. The last part is a resolution of their synthesis- was their collaborative premise valid?

Teaching Presence

Although at first this may seem like the role of the instructor, the instructor is involved only as a facilitator who encourages students to help lead the discussion. The three areas that comprise the teaching presence are, “Design and Organization, Facilitating Discourse, and Direct Instruction” (Garrison, et al., 2000, p.89). The teaching presence emerges when students explain things to each other (direct instruction), ask clarifying questions or encourage others to be involved with the discussion (facilitating discourse). The design and organization component may be more difficult for students to foster if the professor has already established a routine or expectations for the discussions. However, giving students an opportunity to choose partners for assignments may put some of the control for subsequent discussions in the hands of the students. Since asynchronous discussions lack face-to-face spontaneity, identifying when participants are steering the direction of the discussion may be not obvious until after you see if the discourse has gone in a unique direction.

Some research over the last ten years

There has been a great amount of research done to clarify what social presence, cognitive presence, and teaching presence mean and to identify ways these ideas can be used to measure interactions or student behaviors in online situations.  It is possible that when researchers analyze their asynchronous discussions, a statement may fall into more than one “presence” category so the articles cannot easily be separated by their presence category. Instead they are identified here according to the main idea the paper was trying to establish.

How does the instructor/teacher impact discussions?

Some researchers have aligned their research with Garrison’s theories, whereas a few other people are challenging some of the nuances of the Community of Inquiry.  Shea and Bidjerano’s 2009 paper has a lengthy introduction about theories of how online instruction works and they delve into what Garrison and his group have established with the Community of Inquiry. They also recognize other groups who are organizing theories such as Mishra and Koehler, and Larramendy-Joerns and Leinhardt. Larramendy-Joerns and Leinhardt look at the online interactions as presentational and performance-tutoring instead of having three presences like Garrison’s CoI. Mishra and Koehler are the ones who coined the term, technological pedagogical content knowledge (TPCK) model, which integrates Laramendy-Joerns and Leinhardt’s views. “Quoting Schulman (1986), Mishra and Koehler (2006), argue that the bifurcation of disciplinary knowledge and pedagogical knowledge was a major barrier to the improvement of instruction in schools” (Shea & Bidjerano, 2009, p. 544). This disconnect continues when teachers enter the classroom. They may know how to teach and what to teach, but not how to teach the content they are supposed to teach. An analogy to online asynchronous discussions is drawn. Online instructors may understand the mechanics of a discussion forum and the content that needs to be taught during that class, but do they understand how the instructor is supposed to behave so that student learning can be maximized?

Shea and Bidjerano propose that the instructor plays a significant role in the success of an online course. They administered a survey that included gathering demographic data in addition to the directed questions about student reactions to their coursework. They did not create the survey tool in isolation, but rather it was a collaborative process among several researchers, including Garrison’s group. Their findings conclude that there is a relationship between social presence and the ability of learners to extend their understanding of the content presented in the class. They assert that “teaching and social presence represent the processes needed to create paths to epistemic engagement and cognitive presence for online learners” (Shea & Bidjerano, 2009, p. 551).  What I am still not clear on, however, is if they view a teaching presence as one that is outside of the learner or if they recognize the teaching presence comes from within the learner. Some statements suggest that the instructor merely facilitates a teaching presence within the students; however, they imply instructor presence plays a role in increasing cognitive presence through the course instructor being engaged with the discussions as they happen. Students responded better when the “instructors’ judicious participation in online discussion” (Shea & Bidjerano, 2009, p. 551) focused the learners to pay attention to the relevant topics. They continue their discussion to suggest instructional designers must incorporate the role of the instructor into their courses. Although this paper did not choose to address direct instruction compared to “facilitation,” they suggest that direct instruction still has its place in the online course environment. They see this direct instruction being integrated in the ways the instructor participates in the asynchronous discussions. It looks like the creation of information still remains planted in the hands of the instructors. There continues to be a search for a way the instructors can use their content knowledge to facilitate inquiry in the students.

Does the length of the course affect how students learn?

In 2009 Akyol, Vaughan, and Garrison published their findings on how the social, teaching, and cognitive presences are affected by the length of the course. Their research was actually quite limited which should open them up to further publications as they refine their impressions on whether a four week class compared to an eight week course has an effect on what and how students learn in online classes. With the data they collected, they determined that integration, a component of the cognitive presence, is stronger for students who are in the longer course.   In addition, emotional expression, a component of the social presence, is also stronger for the longer courses. In contrast, students in the shorter course excelled more with group cohesion, a component of the social presence, and with exploration, a component of the cognitive presence. As for the teaching presence, there was no statistical difference between the two groups. This paper had so much of the research design and analysis done so poorly that I hope they redo their “experiment” with a much larger sample size and an analysis that does not make identifying student contributions for one category of a specific presence exclude other categories. For example, they decided to measure each category of the social presence based on what was said in the asynchronous discussions. This is fine, but then they added up all of the comments and decided to figure out what percentage each category made up the total number of comments. There is a bit of logic missing here because a comment that reflects on “group cohesion” should not mean that a comment was not made about “affective or emotional expression.” They are not comparable in the sense that saying something that falls in one category is a deliberate lack of saying something in another category. In addition, who is to say that there should be a specific distribution of the comments?  They grouped the data this way to give an overall impression of what students say in their discussions; however, the variables that influence what students choose to say were not exactly controlled. Regardless, more research needs to be done on the influence of course duration on how and what students learn in online classes. If nothing else, they need more data if they continue to feel compelled to use a line instead of a vertical bar graph to illustrate two data points. With at least three points they can start to establish a trend and therefore there may be some logic to doing line graphs.

Research design learning resource

Douglas Archibald is another professor who publishes papers focused on the Community of Inquiry. Although his 2010 article was very short, it set him up for future publications that look at how his Research Design Learning Resource (RDLR) is effective with preparing students for doing research in the social sciences, mainly in education.  He is using the Community of Inquiry Instrument, a survey that lets researchers analyze the components of asynchronous discussions with respect to the three presences, to evaluate the discussions students have while doing the RDLR program. The CoI instrument was developed by several other researchers, including Garrison, in 2008. With Archibald’s paper we see how the CoI framework is being extended to support specialized research that other professors are doing.  People other than those who work directly with Garrison can choose to incorporate CoI into the paradigm they are trying to promote. Like Shea & Bidjerano, Archibald seems to want to stretch the understanding of CoI into pragmatic analysis of online learning in ways Garrison’s group has not yet examined.

Collaborative virtual environments

Hamza-Lup and Stanescu work with haptic feedback devices, which are mechanical objects connected to the computer so that a person interacting with the virtual environment can have kinesthetic feedback. They have created physics interactive lessons where students do not just play with the equations for force equals mass times acceleration. The students feel how their force affects the acceleration.  The kinesthetic parameter lets a third sense join the two that are most commonly used with online discussions: vision and hearing. Now people can feel what they see or hear going on in their virtual world. Students can do an action and not only see its effect, but they can see the equations change as they alter what they are doing. It is an immediate feedback mechanism. What does this have to do with CoI?  It may not be obvious, but the CoI measures what happens when the visual and audio learning styles are used for asynchronous discussions. Hamza-Lup and Stanescu want to see how the CoI framework can measure the ways haptic feedback can “enhance the ways in which students can explore content at a distance” (2010, p. 81). They don’t intend to change how CoI is interpreted or used; instead they prefer to add another dimension that can provide context for measuring the social, cognitive, or teaching presences.

Measuring metacognitive gains using CoI

The cognitive presence may be analyzed by evaluating the variances of metacognition.  Akyol and Garrison (2011) recognize the cognitive presence component of CoI and therefore have their paper align with the CoI construct. Their paper focuses on the metacognitive part of the cognitive presence whereas other publications (Garrison & Anderson, 2003; Garrison & Vaughan, 2008) elaborate on how the cognitive part of the inquiry deals with the factual or content part of the learning. Whereas the social presence would address issues that may be somewhat akin to netiquette, the cognitive part deals with how well concepts are explained or debated in the discussions.  They found that quantity does not equal quality because clear writing can express ideas more powerfully than fluffy, unfocused statements. On the other hand, too much brevity allows for misunderstandings, which may inconsequentially lead to a more in depth discussion as more critical thinking is used to analyze what is being said, as well as for identifying what needs to be mentioned. They determine there to be three stages of metacognition:  Knowledge of Cognition (KC), Monitoring of Cognition (MC), and Regulation of Cognition (RC). They monitored and evaluated transcripts of online student discussions to evaluate where students expressed knowledge of their learning or their relationship to the information being discussed (KC), when students made evaluator comments about what other people said (MC), or made comments that affected the direction of the conversation (RC).

The metacognitive partition, Knowledge of Cognition (KC), plays a role when the students state their understanding of the topic being discussed and the process by which they are discussing it. If they recognize limitations in an online discussion then that is part of the knowledge of cognition because they are aware of how an online discussion can limit spontaneous responses that might otherwise cascade in a face to face discussion. The debate over ideas and concepts falls in the category of Monitoring of Cognition (MC) because students are determining where their ideas align with their peers and where they may contradict each other. Naturally netiquette needs to be followed when debating interpretations of content so during the debate, the social presence may be involved. When certain students dominate the conversation or tend to steer its direction, then the Regulation of Cognition (RC) is happening. Not every student will regulate the flow of the content matter and ideas it generates in every discussion. There may be times when students are elected to be the facilitators of the discussion. When students behave as strong regulators of cognition, their teaching presence is being expressed. By exerting control over the discussion and guiding its outcome, students are following a constructivist approach to learning.  They are not being passive participants. Instead they are influencing the direction of the discussion, thereby demonstrating behaviors desired when students do inquiry based learning.

References

Akyol, Z., Garrison, D.R. (2011). Assessing metacognition in an online community of inquiry. Internet and Higher Education (14), 183-190. doi: 10.1016/j.iheduc.2011.01.005

Akyol, Z., Vaughan, N., & Garrison, D.R. (2011). The impact of course duration on the development of a community of inquiry. Interactive Learning Environments, 19(3), 231-246. doi: 10.1080.10494820902809147

Arbaugh, J. B., Cleveland-Innes, M., Diaz, S.R., Garrison, D. R., Ice, P., Richardson, J.C., & Swan, K.P. (2008). Developing a community of inquiry instrument: Testing a measure of the Community of Inquiry framework using a multi-institutional sample. Internet and Higher Education, 11, 133-136. doi: 10.1016/j.iheduc.2008.06.003

Archibald, D. (2010). Fostering the development of cognitive presence: Initial findings using the community of inquiry survey instrument. Internet and Higher Education, 13, 73-74. doi: 10.1016/j.iheduc.2009.10.001

Garrison, D. R., & Anderson, T. (2003). E-learning in the 21st century : a framework for research and practice. London, New York: Routledge Falmer.

Garrison, D. R., Anderson, T. & Archer, W. (2000). Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education, 2, 87-105.

Garrison, R.D., & Vaughan, N.D. (2008). Blended Learning in Higher Education: Framework, Principles and Guidelines (1st Ed.). San Francisco: Jossey-Bass.

Hamza-Lup, F. G., & Stanescu, I.A. (2010). The haptic paradigm in education: Challenges and case studies. Internet and Higher Education, 13, 78-81. doi: 10.1016/j.iheduc.2009.12.004

Larreamendy-Joerns, J., & Leinhardt, G. (2006). Going the distance with online education. Review of Educational Research, 76(4), 567-605.

Llewellyn, D. (2002). Inquire within: Implementing inquiry-based science standards. Thousand Oaks: Corwin Press, Inc.

Mistra, P., & Koehler, M.J. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. Teacher’s College Record, 108(6), 1017-1054.

Shea, P., & Bidjerano, T. (2009). Community of inquiry as a theoretical framework to foster “epistemic engagement” and “cognitive presence” in online education. Computers & Education, 52, 543-553. doi: 10.1016/j.compedu.2008.10.007