Jim Kenney

Pepperdine University
Omet Cadre 7

 
 
 
 
 


 

A Synchronous Solution for Amgen
Action Research Overview

Introduction
Literature Review
Research Cycles

 

Introduction

The purpose of my research was to improve global learning by introducing synchronous online learning into my organization. My research work began in Fall of 2004 and continued into Summer of 2005. The document provides an overview of the research conducted, including a summary of my literature review and the results of three separate research cycles, and finally a reflection of the research and its impact upon me personally and as a education professional.

The setting for my research.
 I work for Amgen Inc., a biotechnology company with headquarters in Thousand Oaks, California. As a curriculum designer, I work in the curriculum design group within a department called POQ Training and development. We provide instructional materials for those involved in manufacturing medications, and for all related service groups. This includes supporting staff in Process Development, Operations, and Quality organizations (POQ). Most of the training we support is highly technical in nature.

Since we are a global department, we develop instructional materials for Amgen sites located through the United States and Puerto Rico. For this research, I introduced a synchronous curriculum for manufacturing sites located in California, Colorado, Puerto Rico, Rhode Island and Washington.

Currently, I manage the development of training materials for our site in Puerto Rico. I manage a team of nine instructional designers, all developing materials for the project. In addition, I also take on other global projects as needed. One of those projects included developing a set of materials for troubleshooting autoclaves, a complex piece of equipment common at all manufacturing sites. It was the autoclave class and its materials I decided to use to introduce synchronous instruction.

The community of practice.
Completing this research required a team of experts from a variety of backgrounds. First, we assembled a team of professional trainers representing sites in California, Rhode Island and Puerto Rico. Since our topic related to engineering, mechanical maintenance and instrumentation disciplines, we used trainers who already instructed in these areas.

The team also included a subject matter expert (SME), a master mechanic who had extensive experience in working with autoclaves, out chosen topic. One of my senior instructional designers assisted in modifying course content for online delivery. Finally, I served as the project lead and provided experience as both an instructor and instructional designer.

Although not directly involved in development or delivery, an important area of support came from managers in both training and engineering organizations. It was management that approved the initiative, provided updates to senior management, and provided resources that were invaluable to the success of the project.

Finally, it was the students themselves that contributed to the community. Mechanics, Instrumentation Technicians, System Owners, and Operators participated and shared knowledge from California, Colorado, Puerto Rico, Rhode Island and Washington. Skill levels ranged from novice to experts.

Statement of problem.
I chose to introduce synchronous instruction as a means to solve a global learning problem. The issue was a lack of adequate technical knowledge across all Amgen sites and groups. The lack of knowledge distribution resulted in performance problems that impacted the entire corporation.

To overcome this problem, sites would request site-based classroom training as a solution. However, because of a shortage of qualified instructors, the resources were often unavailable to provide courses. In the past, having instructors travel had been a viable option. However, cost reductions combined with a demand for just-in-time training required a new solution.

Research Questions.
In order to define and focus my area of study, I developed research questions prior to beginning my actions. The question related to my overall project was: If I use synchronous web-based learning to deliver my department's course materials to students at multiple sites, will it improve the distribution of knowledge compared to traditional site-based classroom instruction?

For my first research cycle, I asked the following: If instructors and subject matter experts learn the methods and strategies for online synchronous learning, will they feel both prepared and motivated to instruct a online synchronous class?

For cycle two, I wanted to know: If I use synchronous web-based learning to deliver my department's course materials to students at multiple sites, will it improve the distribution of knowledge compared to traditional site-based classroom instruction?

Finally, for cycle three I asked: If I make improvements to my synchronous web-based course, including changing instructors, adjusting course length, providing additional instructor support, and adding learner centered activities, will it improve knowledge transfer and learner satisfaction compared to the previous synchronous session and traditional site-based classroom instruction?

Literature Review

In Fall of 2004, I began reviewing literature relating to all facets of online learning. The review would ultimately help me anticipate and prepare for actions conducted for my research cycles.

For my review, I wanted to learn if online learning had been effective in other settings; including synchronous and asynchronous delivery, as compared to traditional classroom instruction. I was interested in the ability of online learning to distribute knowledge over long distances. Finally, I wanted to examine the role the instructor plays in online learning, including instructor preparation and course design.

There were several themes I discovered in this review. First, most of the research supported that online learning was just as effective as traditional classroom instruction. This supported my assertion that online learning was a viable alternative to existing means of delivery. There were also examples of successful use of online instruction for distance learning.

I learned that there are advantages to both synchronous and asynchronous online instruction. As a result, I had planned to use both synchronous and asynchronous methods in my 3rd cycle. However, because of current limitations in my organization, I postponed adding asynchronous delivery until future cycles. My further review revealed that online instruction benefits from a learner-centered approach. Online Instructors should be both qualified and work toward delivering interactive sessions.

As for online application to my work, there appeared to be a direct benefit to Biotechnology. "Biotech companies that want to excel in the face of breakneck change and demands to do business faster and smarter will embrace e-learning. Through the wonder of the internet, it offers a practical and affordable way to foster the kind of learning culture required of successful knowledge-based companies in the 21st century" (Gregory, 2002, p.2).

In summary, my review uncovered that online learning has numerous advantages of traditional instruction. In additional, there seemed to be a tremendous potential for online instruction for a variety of purposes, including improved distance learning.

You can see my entire review by visiting my Literature Review.

Research Cycles

Cycle 1 summary.
For my first cycle, I facilitated an online instructor preparation class in December 2004. This included teaching both instructors and project team members on how to effectively deliver online synchronous training. It allowed participants to explore this learning strategy and to practice online instructional techniques.

The cycle yielded some pleasant surprises. For one, the participants seemed to thoroughly enjoy the synchronous tool we used, WebEx. This technology was very easy to use, but it also appeared to make the learning enjoyable. It was in cycle 1 that I learned the appeal of synchronous learning. I designed much of the course to allow for hands-on learning for the students. Each student had an opportunity to present using on of the WebEx tools. This learner-centered approach seemed to enable quick learning of WebEx. Even though one instructor did request additional practice time, for the most part, everyone felt ready to take the next step and instruct an actual course. Since one of my primary concerns was gaining support and buy in for this method of instruction, the positive feedback and continued desire from all students to proceed with further instruction gave me confidence to proceed with cycle 2.

Cycle 2 summary.
For my second cycle, I organized an online synchronous course pilot for February. This included modifying a current version of our now successful "Troubleshooting Autoclaves" course and adapting for synchronous delivery. Because of the positive feedback regarding WebEx, I decided to continue with the same technology for this cycle.

Beside the obvious newness of presenting the course online, we made some other major instructional changes. For one, we decided to instruct the course over six days. Second, we limited each session to one hour. Another challenge was the location of the instructor. The instructor was in Rhode Island, while a Subject Matter Expert and myself were located in California.

We were pleasantly surprised by the results for out first attempt at online instruction. The instructor presented the instruction each day as scheduled. Feedback from the students showed they liked synchronous instruction, and most even expressed they preferred online over traditional classroom instruction. The primary reason for this positive feedback was the convenience of taking the instruction at their own desk. However, it also seemed that the technology itself helped make the learning appealing.

When comparing the test results of the online course versus past classroom sessions, the online version actually scored slightly higher. This seemed to affirm my Literature Review finding that online learning was at least as effective as traditional instruction.

I had several questions as a result of cycle two. One, would a change of instructors result in improved results? Although my instructor was knowledgeable, he seemed to lack the energy and enthusiasm for dynamic online instruction. Another question related to instructor support. If the instructor was in the same room as the subject matter expert and myself, would the additional support result in a smoother running session and free-up the instructor to focus more on instruction and interaction?

I also wondered if lengthening each session would help the content seem less rushed. Finally, I wanted to add activities that were more learner centered and encouraged greater involvement from all students.

Cycle 3 summary.
Based on my analysis of cycle 2, I decided I would focus on making improvements to the same course as cycle two. My first change was lengthening each session. I increased each session from one to an hour-and-a-half. I also reduced the course from 6 to 5 days.

I also changed instructors. The new instructor, although not as knowledgeable in the subject matter, had a more dynamic personality. I spent a month mentoring the instructor on online instruction, hoping he would be better prepared and comfortable with WebEx.

Another big difference was the make-up of the students. In cycle 2, we had students from California, Rhode Island and Washington. For this cycle, we added students from Puerto Rico and Colorado. My objective was to make the class smaller. Having 16 in cycle two seemed a little large. In the end, we had 12 for cycle 3. This was far more manageable.

A final major change was the instructional approach for our troubleshooting activity held the last week of class. Normally, we read through a series of scenarios and then take turns suggesting a course of action. For this class, we took a learner-centered approach. We assigned several scenarios to each site and told them to work together to come-up with their own solution. We then set-aside class time for each team to present their scenario and who they would respond. The control of the WebEx tool was given over to the chosen team presenter, and they presented to the class their scenario. This activity worked well. Each team did a great job in presenting and seemed to learn tremendously from the activity. The best result was that participation rose dramatically.

The end results for cycle 3 were not as I expected. The test results were lower than both cycle 2 and previous classroom sessions. However, the difference was not that significant, and each student still showed an improvement in learning. In addition, schedule changes and added instructor support seemed to be positive changes and should have resulted in improved results.

The reason for the lower results were most likely due to several factors. Because of several urgent site emergencies, many of the students missed sessions. Also, more of the students reported having little or no time for self-study and adequate preparation for sessions. Finally, similar to session two, a number of the students experienced noisy learning environments with frequent interruptions. The combination of all of the resulted in significant learning challenges for some students.

A final area that still needs improvement is the instruction. Although the instructor received high marks from the students, he still struggled at times with the content and the technology. This became very evident a day when the Subject Matter Expert was unable to attend. It is clear that the challenging nature of this delivery requires an instructor that is both a subject expert and has an engaging personality.

To review detailed reports, see cycle 1, cycle 2, or cycle 3 reports.

Analysis

The data collected from all three cycles indicate both students and instructors alike fully accept and appreciate learning in an online environment. Even a drop in test scores in cycle three had little impact on how students felt about synchronous learning. Much of this success relates directly to the technology. WebEx worked almost flawlessly for all three cycles. The few students who did have technical issues were able to resolve them with little impact to learning. In fact, Webex was so easy and dependable that the students and instructors seemed to feel fully comfortable with technology after a few short hours. Another pleasant surprise was the fun students had with the tools. For example, being able to use functions like drawing on the whiteboard, using the chat feature or answering a poll seemed to naturally encourage interaction and fun. Despite the challenges faced with the curriculum and instruction, it was largely the technology which created an environment rich for learning.

What about the overall results for the cycles? Even though the technology was received favorable, did learning occur? For this, I looked at the results of each cycle in relation to the research questions.

My research question for cycle one was: If instructors and subject matter experts learn the methods and strategies for online synchronous learning, will they feel both prepared and motivated to instruct a online synchronous class? Overall, I believe the answer to this was yes. The training session I delivered in cycle one resulted in the instructors learning the technology and instructional approach. Each instructor responded that they wanted to instruct a future course. Was it adequate preparation? Cycle one showed the instructor was able to effectively deliver a course. In fact, all the feedback received from the students regarding the instructor was positive. However, there was feedback that the pace of the course lagged at times. This resulted from the instructor still learning the technology during instruction. This was a clear indication that additional practice was needed prior to the first class. This same need for additional practice would also have benefited the instructor for cycle 2.

What about the results for cycle two? My cycle two question stated: If I use synchronous web-based learning to deliver my department's course materials to students at multiple sites, will it improve the distribution of knowledge compared to traditional site-based classroom instruction? The results of the pre and post-test scores indicate a yes to this question. Students showed a 32% improvement in their scores when comparing post-test to pre-test results (previous classroom session had a 31% improvement). So why the positive results? Although there were many factors, a few stood out. As already mentioned, the technology was conducive to learning. The technology, in addition to being enjoyable for students, also had the enormous benefit of convenience.

The course materials were also a factor. I already knew from previous classroom sessions, that the participant materials were very effective and perceived as useful to nearly all students. For the online course, we asked the students to read content in the participant materials prior to attending the live online sessions. Obviously, this "self-study" approach aided in knowledge transfer.

For the online sessions, we adjusted the curriculum to encourage participation and dialog. This resulted in discussions that reached across sites and disciplines. The WebEx polling feature, chat function, whiteboard activities and instructor questions all aided in increasing interaction and improving knowledge sharing.

Finally, a major "plus" involved having a Subject Matter Expert available for all cycles. The students benefited from asking our expert a variety of challenging questions. Our expert shared much of his experience, allowing the learner to acquire knowledge not available in learning materials or equipment manuals. He also helped relate the learning to "real life" examples. I consider much of this valuable knowledge that is normally hidden from inexperienced staff.

What factors might have limited the "distribution of knowledge?" For both cycles 2 and 3, certain students were limited by noise and interruptions in their learning environment. Since our synchronous sessions included audio, and taking the course from their own desks, the environment had added implications and hindered learning for some. Despite building activities and discussion into the sessions, there was still too much lecture and too little learner-driven instruction. Because of the nature of our topic, some lecture is required. However, with the aid of the materials and other resources, learners could have constructed more of their own learning.

In cycle three I asked: If I make improvements to my synchronous web-based course, including changing instructors, adjusting course length, providing additional instructor support, and adding learner centered activities, will it improve knowledge transfer and learner satisfaction compared to the previous synchronous session and traditional site-based classroom instruction? As I noted earlier, the results for cycle 3 were not an improvement over cycle 2. What were the main reasons? The problem of noise and interruptions for learners was probably the biggest factor for some. The fact that the attendance was low for some session also impacted learning. In addition, a lack of sufficient self-study by most of the students resulted in lower results. Finally, my instructor lacked sufficient knowledge of the topic. Although the Subject Matter Expert provided assistance, there was one day he was unable to attend. Since that was critical day when highly technical subject matter was discussed, there were many concepts the instructor had difficulty explaining.

Overall, what was the impact of introducing online instruction at Amgen? Returning to my primary research question: If I use synchronous web-based learning to deliver my department's course materials to students at multiple sites, will it improve the distribution of knowledge compared to traditional site-based classroom instruction?

The answer to my question was yes. The distribution of knowledge improved by using online synchronous learning. Synchronous instruction allowed knowledge to be shared across 5 sites. Each student had live access to an instructor, students from other sites, and a Subject Matter Expert. This delivery method was convenient and well liked by students.


Reflection

Online synchronous learning holds tremendous potential for highly technical instruction. For today's mobile and diverse workforce, there is an opportunity to share knowledge in an live online environment, from basically any computer, and from anywhere in the world. I find that amazing. However, harnessing all the benefits of synchronous instruction is tricky. There seems to be no shortcuts. From my experience, a good instructor can make a poorly designed class successful. In the same way, a well designed class can succeed to a point, even if the instructor lacks experience. This does not seem to be the case for online instruction. Having everything in place, including a well balanced and skilled instructor seems a requirement. Finding an instructor with some technical aptitude, a dynamic personality, is extremely organized, and has a deep understanding of the subject matter is difficult to find. In the future, I would carefully search for these instructor qualities early in the online development process. If that ideal instructor does not surface, then I would focus more on developing skills prior to the class.

What was surprising about synchronous instruction was the appeal and effectiveness of the technology. The technology almost sold itself. The challenge then is the curriculum, instruction, environment and as I already mentioned, the qualifications of the learner. It is the technology that attracts the attention of the learner, but it is the use of that technology that truly impacts the end result - the transfer and construction of knowledge.

I learned this lesson during my last cycles. Although the technology was functioning very well, I felt the level of involvement was still lacking. Since we were online, I was unable to "see" how attentive the students were. I did have clues from the amount of activity resulting from polls, chat and dialog, but I still felt the need for more involvement. How do I increase involvement? One method is to develop a constructivist strategy which encourages involvement and allows student to control and construct their own knowledge. Adding our troubleshooting activity during the end of the cycle 3 course was my attempt to implement this strategy. As I mentioned earlier, the activity did stimulate the class and was a wonderful example of turning control over to the learner for a very technical topic. This was all positive, but I felt a little late into the learning process to be very effective. More control of the learning should have been granted to the students in week one, when the foundational concepts were being learned. I did struggle with this broader leaner-centered activity because some of the initial concepts are very technical. However, in hindsight that is the reason constructivism would have worked so well. Because of the complexity of the topics, students would have benefited from learning about autoclaves in their own way. In addition, this topic would have worked well since the support structure was already in place with a subject matter expert and training materials readily available. So what would I do differently? I would completely redesign much of the course taking a more learner-centered approach.

Of course, an important part of the course design already included self-study of the participant materials. This was an important learning strategy, but we failed to get adequate commitment of the learners and their managers to set aside time for self-study. I find this very typical in our environment. So what is needed to motivate students to self-study? One idea is to write up a contract/agreement in the beginning that a supervisor and learner would agree to. Having both supervisor and student sign would create a certain level of accountability. Another motivator is having student presentations or projects that are based on the content reviewed in the self-study. For future session, I believe both strategies would improve this challenge. Ultimately, it must be the learner who sees a value in learning, and that is something that must be considered by every learner before the learning begins.

A final area of change must come in relation to the learner's environment. The very selling point that makes this approach so appealing, the convenience of taking this course from a learners own desk and computer, is also one of the greatest challenges. How can a learner isolate themselves enough from area noise and interruptions so that full attention can be applied to the instruction. Although using headphones and warning neighbors of class commitments works in some cases, more is needed for some students. Is it feasible to have students take the course in a conference room or in a computer lab? Both of these have there challenges, including availability and access to computers and phone lines. Ultimately, much of this would be solved if all students had laptops and the freedom of wireless technology. However, for now that is not an option for Amgen. The best strategy for now is a detailed list of options reviewed with the learner and his or her supervisor. As with self-study, it is important for all parties to commit to providing an adequate learning environment.

This brings me to final thought. How did I develop personally through this research effort and what would I do differently? I believe my greatest challenge was overcoming a resistance to using technology as a solution to learning problems. I knew learning technology was needed to meet the demands of our increasingly dynamic workforce, but I was uncertain how to take the first step. This project proved to be the stepping stone I needed. However, my growth was more than a simple application of using technology. It was about risking enough to try an unproven technology to solve a real need. Bringing together a group of learners online, from sites around the country, and in real time, would have seemed impossible only years ago. However, the impossible becomes possible through vision, leadership and action. It was a combination of those attributes I developed through this effort. As I have told my instructors several times, we are the pioneers. We moved into uncharted territory, with the power of community and technology pushing us forward.

What would I change next time? I would be more assertive in moving the program forward in the organization. For example, I should have pushed harder to get an asynchronous tool in place. Learning how to make learning technology a priority within the organization takes time. Although there are areas I would do over again, I learned some learning requires time. There is no replace for experience.