Cycle Two Report: New Technology & New Pedagogy
From Cycle One...
Background & Supporting Data
One way to examine my own praxis is to evaluate my courses though the eyes of the students who are enrolled in them. At GCC, a staff member collects student evaluations during the last third of the term being evaluated. Students are assured that their responses will remain anonymous and that the faculty member being evaluated (i.e., me) will not see their responses until after grades are submitted. The following chart contains data (% responding "always") from five, randomly selected traditional PSY101 courses that I have taught over the past 4 years, including one short term session.
My courses begin with a seat load (i.e., max capacity) of 40, plus a 5-student wait list. At the time student evaluations are collected, the seat load is typically smaller. Consequently, the average number of students in each course used for the preceding analysis was 27.7. To make the data used for this cycle more manageable, I decided to do two things. First, I eliminated the data gathered that was not intimately connected the research questions contained herein. These included statements regarding class start time, pronunciation, and availability during office hours. Second, I compiled the five courses above into one average of student responses in my traditional psychology courses. This would provide me with a base from which to grow and expand my pedagogy via educational technology. It also provides me with an internal means of evaluating my efforts at incorporating constructivist theory into my courses.
As previously noted in my literature review and cycle one report, I fear that my (mis)use of educational technology contributed to my attrition rates, so I then transposed the student evaluations from my Fall2003 PSY101 hybrid course on the data gathered from my traditional PSY101 courses.
to see the complete form, including the statements on the image above, click here
My concerns about the impact of technology on attrition and retention seem superficial at best. The glaring reality that the data above convey is a painfully significant decrease in the students' perceptions of the course and my ability to teach it. It seems so obvious to me now: teaching is about teaching, NOT technology. I had made the error of being "technocentric" (Papert, 1987) in my design of the course, and this error was built upon over my first 5 sessions of teaching it. With each passing term, I added a layer of technological complexity for which I had little theoretical base in education. The fall '03 evaluations capture the essence of my technocentricism. My fears that I had let the technology offered to academics (i.e., WebCT, course packs, etc.) dictate my teaching seemed validated by the data above.
Worse still, was that my ignorance regarding educational technology dictated my design. I took what Smith (1998) would call the "official theory" of learning, and I used WebCT to build my course around it. The following seven statements appear to be my weakest areas in the eyes of my students, and each seems to be bound to my use of technology. Each item on the following list has been annotated with my reflections regarding my misuse of educational technology.
The instructor is well prepared for each class session 45% responded "always"
-this was obvious to both my students and me. I muddled forward with my use of technology, and I had hoped to learn as I proceeded. Unfortunately, this was poorly demonstrated in my classes.
The instructor gives clear explanations 42% responded "always"
-I did not know how to make the best use of the communicative technologies available on the internet, and I did a poor job of learning how to make my classroom learner-centered. Communication should be at the heart of any class that relies on technology for facilitation, and I simply did not know how to share this idea with my students.
The instructor's tests/assignments emphasizes the important aspects of the class 42% responded "always"
-my quizzes were designed to be weekly reading checks, and they often did not connect clearly to the ideas shared during our face to face meetings. This design flaw was driven by my own interpretation of "student contact" hours. The state outlines these hours very loosely, and I equated contact with rigorous assessment. I was wrong, and I concur wholeheartedly with my students here.
The instructor makes useful comments on the returned class assignments 17% responded "always"
This problem is my greatest weakness, and it is directly related to time. I had none because I had too much of everything! By equating rigorous assessments with student contact, I simply overworked my students and myself. This left me no time to think through responses.
After an objective exam, the instructor discusses the tests in class or makes the correct answers available 42% responded "always"
-I am not sure why my students perceived this. Each WebCT quiz provides immediate feedback, and each assessment in class was followed with a review. I should not be so self-righteous, though. It is their take that matters most on these evaluations. Another possible explanation for this response rate is that this evaluative statement was used to vent about my tests' difficulty.
The instructor makes it easy to find out my current grade in the class 42% responded "always"
-Although WebCT provides immediate feedback for its tools, It does not enter scores for assessments that occur in-class. As previously mentioned, I was over-worked and could not grade fast enough to make grades "easily" available. I was also wrestling with two grading programs: one in WebCT and the other on my desktop. My inability to effectively manage grades contributed to this, I'm sure.
The instructor is available to assist students outside of class 50% responded "always"
-There is a depersonalizing quality to the approaches to on-line instruction that I used; and I did not know how to get beyond it.
I also decided to review my students' comments on these student evaluations to look for trends that might help clarify my interpretation of the strengths and weaknesses of my pedagogy. In response to the question "What do you like best about this class?", many students cited my enthusiasm & humor, my ability to make complex ideas digestible and to make them "relatable", and my use of powerpoint. Their responses to the second question, "How can this class be improved?", were a bit more frustrating. The vast majority of students reported that the class needed no improvement. A few stated the they wished we had more time in class (i.e., f2f) together, and others said the class needed to be "easier" to make it better.
In addition to the student evaluation data, I reviewed my peer evaluations for the same time period. These evaluations were made by the college's Dean of Instruction, Social Sciences Division Chair, and the Head of the Psychology Department. These evaluations were colored with the following commendations:
These were balanced with the following recommendations:
My own synthesis of all this feedback paints a simple picture of my praxis. I am a teacher who focuses his personality on the content of the day, and I use Powerpoint to provide structure to my sharing. I share the psychology that I have seen through each day as a means of giving life to the course content. My hope is that this helps them learn to think critically, as a psychologist or social scientist might. This synthesis makes the error of my ways painfully clear: I had no way of reconciling my praxis with the hybrid course format, and I had over-worked myself in a way that disallowed me from reflecting on this error.
I set out to find technology that would build upon my strengths, and I re-discovered the profound wisdom of others in the process. Learning is about relationships and construction, and technology can facilitate this. As Riel (2000) stated, "technology is shared minds made visible". To foster this sharing of minds, I adopted wiki and blog technology as a means of building community. As discussed in my literature review:
My students were eager and active in their early use of the wiki. To assure that this would continue throughout the term, and to enable me to collect periodic assessments throughout, I expanded a critical thinking assignment used in my traditional, fact-to-face General Psychology courses. On exams in my traditional courses, I prepare a five-paragraph essay containing approximately eight errors, and I require my students to identify and correct up to five errors discovered. The assignment builds upon Sternberg and Swerling's (1996) enumeration of cognitive processes that underlay thinking and adaptive task performance. Their model is a seven-step sequence: problem recognition, process selection, representation of information, strategy formation, allocation of resources, solution mentoring, and evaluating solutions. Building upon this model, I developed an assessment called the weekly audit, which required my students to find errors or weaknesses on the wiki and outline the reasons for their selections as well as their proposed corrections/solutions.
The weekly audit assignments were also extensions of the four principles of critical literacy presented in my literature review; and the wiki provided a means for allowing the students to see other perspectives of their own ideas as the very text that they used to type their ideas changed. Students could see their thinking unfolding as a consequence of two processes: their social interaction with their community of learners, and the intrapersonal interaction between their prior knowledge and the current information. These two processes are fundamentally constructivist, and they helped me make substantial improvements upon the seven weakest areas of my course, as noted in the above student evaluation data.
A contrast of student evaluations from PSY101 Hybrid courses before (Fall '03) and after (Fall '04) the integration of wiki and blog technology demarcates astonishing improvements in virtually every aspect assessed. When contrasted with the mean of student responses in over the past 4 years in my traditional PSY101 courses, the data also show an improvement in various areas. Most importantly, the seven weaknesses highlighted previously were each improved upon during the Fall 2004 term.
The instructor is well prepared for each class session 79% responded "always", compared to 45% one year prior
The instructor gives clear explanations 79% responded "always", compared to 42% one year prior
The instructor's tests/assignments emphasizes the important aspects of the class 63% responded "always", compared to 42% one year prior
The instructor makes useful comments on the returned class assignments 68% responded "always", compared to 17% one year prior
After an objective exam, the instructor discusses the tests in class or makes the correct answers available 67% responded "always", compared to 42% one year prior
The instructor makes it easy to find out my current grade in the class 78% responded "always", compared to 42% one year prior
The instructor is available to assist students outside of class 83% responded "always", compared to 50% one year prior
Amazingly, the only evaluative statement that resulted in a decrease in score was:
The instructor clearly explained how I will be graded/evaluated in the class
-This is a fair concession, especially since grading constructivist coursework requires a level of subjectivity that is greater than in coursework that is traditionally instructionist. By subjective, I am referring to the need to assess the growth of each student individually and in the context of an ever-evolving collaborative document. The standardization of grades for this type of course design seemed nearly impossible. So, I began to examine the nature of the assessments used in the Fall 2004 term with the hope of developing a broader array of assessments for future courses.
I made several changes in the course design between the Fall 2004 and Spring 2005 terms. One, I increased the number of blogs required so that I could use them to develop my understanding of each student's growth. This was also coupled with a discussion of the importance of reflection in learning, in addition to a requirement that students read each other's blogs. Two, I decreased the number of weekly audits used in the class. Three, I replaced the final edit (of the wiki document) with community generated exam. Worth only 10% of their grade, this exam was built from questions submitted by students and it was intended to capture the content mastery that occurred via the interactions among students. Four, I also developed an oral exam to serve as part their final. I had tried it during the Fall 2004 term, but my design seemed weak. Still, many students excelled using this format:
These interviews were followed with a reflective assignment, designed to encourage critical self inquiry.
In the Spring 2005 term, I used this same reflective assignment, and I made it 14.5% of their course grade. My use of questions during the interview also changed, and I tried to engage the students in a Socratic Dialogue about the nature of science as it relates to psychology.
Ultimately, these four changes in assessment lead to a significant shift in my grade distribution, which I believe addressed my initial concerns regarding compromises of academic integrity for the sake of retention.
So, did I discover which aspects of my own praxis are best suited for teaching in a hybrid course? And did I develop a successful means of incorporating constructivist principles into my course via wiki technology?
Yes and yes.
The "theater-like" environment that I create in my traditional classes took years to develop. As a student, I recall being awestruck by teachers who could captivate and inspire me. As a teacher I took this fascination with me and honed my own craft so that I too could try to captivate and inspire. This style of teaching draws on the relationships that I build with my students, much like a performer develops a relationship with the audience. This is the basis for my praxis, relationships. Constructivist pedagogy is defined by relationships- those between the new information and the prior knowledge, as well as those between people who share ideas.
I tried to develop my abilities as a constructivist teacher by focusing on the relationships in my class. I realized that the three primary relationships I needed to nourish were between each student and me, between each of the students, and between the students' prior knowledge and the new information that they were to learn. With these three in mind, I set out to develop a community of learners because I believed that this is the best way to maintain these three relationships. The technology used to maintain each relationship and its significance are described below.
Relationship 1: Each Student and I
Communication is critical in any relationship, and I used email and instant messenger technologies to assure timely and meaningful responses to my students. I had been comfortable using email in the past, so this technology was manageable. Instant messenger technology, however, was relatively new to me, but not to most of my students. Once they knew my screen name, I was contacted early and often. Amazingly most of the communication along these lines was not formally connected to course content. It was a casual way of connecting with me about many topics, such as music, politics, academic counseling, etc.. Students would ping me well after midnight and throughout the day, and this told me that they were comfortable enough with me to broaden the scope of our relationships. I believe that trust developed from this, and it did so in both directions. This cannot be overstated. Trust works as a remarkable form of intrinsic motivation, and this countered much of what many of my students were familiar with in their histories as students. This motivation drove many to visit the community (i.e., f2f or in the wiki) often and with more excitement that I had previously experienced in any class. I find it difficult to think of a greater lesson beyond trust as means of motivation to maintain community.
Relationship 2: Student to Student
To develop the relationships between students, I tried to initiate connections and then back away. These relationships, after all, belonged to them. I asked the students to set up blogs on the first day as a means of introducing themselves, and I required that each respond to the blogs of at least 3 other students. Relationships flowered in a way that I had never witnessed. It was beautiful and critical to the classes success for it also helped develop trust and intrinsic motivation. Students saw beyond the course content as a means of connecting. They developed relationships based on their high schools, music preference, age, or family life. The course content seemed secondary in the early weeks of the course, and this took delicate processing for me to accept. I eventually realized that this did more to for the course than I had anticipated. As we progressed, the students' relationships gave them a more complex framework for discussing course content. They knew each other, and this fostered better communication in most cases, and it made examples shared in class more meaningful.
Relationship 3: Prior Knowledge and New Information
The best way to appreciate constructivism is to see it happen. The wiki did this. Ideas were developed as part of the community's knowledge, and then they were changed as new information was acquired. Witnessing such change required a great deal of selflessness, which was another lesson that emerged as the term progressed. The wiki hosts a community document that does not track individual authorship, and it develops as ideas are shared and pruned. Initially, this was a difficult task for both the students and me. I had to let go of a fair amount of control over my students work. I could not comment to individuals who were working on the wiki, even though I knew who and when edits occurred. Doing so would undermine the constructivist spirit of the wiki's design. So, I learned to watch learning as knowledge was constructed, rather than as ideas that I transmitted. As someone who has primarily experienced traditional schooling, this was amazingly difficult to do. But I saw a greater good in my relinquishing of control for the sake of community.
The wiki also encouraged active participation because it served as the community's focal point. It was our shared history of learning. This was build upon the community's mutual engagement, joint enterprise, and shared repertoire. As a teacher who spent the year learning to place himself in community with his students, I saw no greater joy than in their development and in the growth of their shared history. The question that remained was:
How could I share this growth with the college community around me?
This concern will frame my actions in the third cycle of my action research project.
Papert, S. (1987). Computer criticism vs. technocentric thinking. Educational Researcher, 16 (I), 1-17.
Smith, F. (1998). The Book of Learning and Forgetting . Teachers College Press: New York.
Wiggins, G., & McTighe, J. (1998). Understanding by Design. Association for Supervision and Curriculum Development; Alexandria, VA.
Site created on August 10, 2004