Chapter 3

Cycle 1: Identification of Technology Skills
     

The first step in redesigning the technology professional development program is to determine the basic technology skill level and knowledge of technology resources available at the school. This process will involve a systematic approach and servs as a springboard to future cycles. I feel it is important to have an accurate picture of the current environment so that the effects of my future actions can be more easily understood. By surveying my current faculty and staff, I will also be able to evaluate the effectiveness of such a survey tool in evaluating the technology skills of employees. Results from this survey can be compared to trends in calls to the help desk. This will allow me to informally assess the validity of this type of self report.

CYCLE 1: Identification of technology skills and knowledge of current employees at Greenhill School. This information will be collected through a comprehensive survey.

Research Questions for Cycle 1:

  • How will understanding the technology competencies of the current employees of the school help me develop a professional development technology program that better serves their needs?
  • How will the use of this survey instrument with current employees help me create a similar assessment tool for new employees?
  • How will my attention to the needs of the new employees affect the way they relate to me and other teachers at the school?

Actions Taken During Cycle 1

The first step in redesigning the technology professional development program was to ascertain the basic technology skill level and knowledge of technology resources at the school. This was accomplished with a self-assessment survey. Looking at various survey methods, I chose a model that would allow me to group questions into categories and reduce the number of actual pages necessary to deliver the instrument. I believed that the structure of the survey was important and that grouping would increase the yield of survey responses.

I also reviewed helpdesk logs and discussed common training issues with the technology staff that serves as technology support for the school. This became a basis for developing most of the questions. We also met as a department to discuss the basic skills necessary for employees to efficiently use technology in their jobs.

The Survey Instrument

The survey was organized into 11 categories (see Table 1) with most questions requiring a Likert scale response: Never heard of it, Heard of it but don’t know how, Limited: just learning, Competent: can complete satisfactorily and Expert: can teach others.

TABLE 1

Survey categories

Section

Topic

Discussion

1

Basic Skills

Basic computer skills related to working in a windows environment and network

2-5

MS. Office

Competencies with the MS office Suite: Word, Excel, PowerPoint, Outlook

6

Internet

Knowledge of the world wide web and use of internet browsers

7

Software

Knowledge of additional software resources available a the school

8

Hardware

Knowledge of additional hardware resources available at the school

9

Future Training

An opportunity for faculty and staff to express their desire for additional technology training and topics for future training

10

Training Methodologies

Respondents were asked to respond to options for training schedules and methodologies

11

Demographics

Respondents supplied demographic data while not supplying information for individual identification

 

Implementation of the Survey Instrument

The survey questions were all drafted and then entered into Survey Monkey. This was then released on a limited basis for review by a group of critical friends. Data and comments were collected related to this limited release and adjustments were made to the questions and survey format. All test data was purged from the system and then the survey was released to the entire employee community of the school.

A letter explaining the purpose of the survey was sent out via school e-mail to all employees and a link was provided to the survey. The survey was sent to 380 employees. This list includes full-time faculty and staff, part-time faculty and staff, adjunct coaches and part-time aftercare personnel. There was a 29% response rate to the survey with 111 employees responding. Adjusting the expected response to focus on the core staff, the response rate was closer to 45%. This also represents the employee base that would normally be involved in this type of training. Table 2 below illustrates the response rates of the core staff based on primary assignment at the school. For the purposes of this study, the maintenance staff and instructional support were included in the staff numbers. Instructional support includes Nurses, Learning Specialists, Psychologists and Librarians. The structure of the survey allowed for the isolation of maintenance staff from other respondents. Faculty was defined as teaching at 100% of assignment.

 

TABLE 2

The response rates for core employees at the school.

Core Staff

Core Staff Numbers

Number Responding

% Responding

Faculty

138

69

50

Staff

73

21

29

Fellow

10

1

10

Administration Team

13

13

100

Maintenance

14

1

.07

Unknown

NA

7

 

Totals

248

112

 

 

Initial Analysis of Results

The survey responses are extensive, however, for this research cycle the scope of analysis will be narrowed to focus on several key areas: skill sets related to core software available on all campus computers and skill sets related to software which, for the purpose of this study, has been categorized as potentially facilitating curriculum integration. Other areas of focus for this cycle include: perceived need for additional training, desire for additional training and preferred training models, methods and times.

Analysis of Core software Skills

The school has standardized on the Microsoft platform and Microsoft Office Suite is installed on all computers on campus. I have classified the following applications as basic literacy applications and where employees would need a working knowledge to effectively fulfill their responsibilities: Word, Excel, Outlook, PowerPoint and Internet Explorer or other Internet browser.

Each question set for these applications had a Likert scale response of:

  • Never heard of it
  • Heard of it but don’t know how
  • Limited: just learning
  • Competent: can complete satisfactorily
  • Expert: can teach others

 

Assigning a value of 1 to 5 for these responses, the mean value for competency level for each application was calculated for each skill surveyed. Table 3 illustrates these values and the ranking of proficiency in these applications related to surveyed skills.

TABLE 3

Rank and mean values for all skills surveyed for each application. All mean values are based on a 5 point scale with 5 demonstrating the greatest proficiency.

Rank

Application

Mean

N

1

Word

4.33

111

2

Basic Skills

4.13

111

3

Outlook

3.51

109

4

WWW

3.39

109

5

Excel

2.90

107

6

PowerPoint

2.88

108

 

MS Word

Microsoft Word is used by all employees on campus as the main word processing tool. It was one of the first applications available to the staff when the network was established in 1996. Extensive training on this application had been done in the past. The results of the self-assessment related to proficiency in Word were not surprising. Most respondents (61%) rated themselves as expert: can teach others. This was by far the highest rated application on the survey as it related to specific application skills. However, it is interesting to note that when asked in a later part of the survey to rate their over-all skill and comfort level (not related to specific skills), 49% rated themselves as Comfortable with the software- Intermediate and only 45% of the respondents rated themselves as advanced- can teach others. This is in contrast to the 61% that rated themselves as expert in Word. This would seem to imply a realization of additional skills beyond their knowledge. Yet, when asked about future training in Word, 66% declined future training. It is possible that they do not need additional training beyond their current knowledge to effectively perform their job. However, calls to the help desk lead me to believe that there is a sizable population that is working ineffectively with limited or incomplete knowledge of the program.

MS Outlook

The school uses MS Outlook with Exchange for e-mail. These applications have been in use at the school since the introduction of the network. E-mail was new to most employees 8 years ago. Every employee has an email address and e-mail has become the main form of communication on campus. Outlook received a median aggregate score of 3.53 for the skills surveyed ranking it 3 out of 6 (see table 3) for the basic applications surveyed. 60% of the respondents identified themselves as intermediate users-comfortable with the software. Comparing this with the calls into the help desk, this would seem to be an accurate reading. There have been few requests in the past for additional training by the teachers and staff. However, there have been some requests from administrators for training of their teachers and staff. Specific training requests have been in the area of calendar features as they desire to make better use of this feature with their staff. These requests have been by division with most of the requests coming from the lower school. There have also been some requests by the administrators of the upper school for additional training to make use of additional functionality with their staff.

Looking at responses to individual questions, it is clear that there are gaps of knowledge in key areas of functionality and resources provided in MS Outlook. The following table highlights some of the more prominent skill sets where employees selected the second option- Heard of it but don’t know how. These skills were selected because it is believed that employees with these skills will be more efficient and effective using this tool to communicate with the community. While all of these skills would add to their efficiency at the job, some of them could also have a positive impact on their use of technology in the classroom and enhance the ability for classroom communities to communicate.


TABLE 4

The percentage of respondents lacking skill in the use of these function of MS Outlook.

Question

Percent responding they had heard of it but did not know how to perform task or skill.

Subscribe to listservs*

31%

Create a meeting invitation to an appointment*

29%

Create rules to handle e-mail

28%

Create a discussion group in public folders*

52%

Copy appointments from Greenhill master calendar (public folders) to personal calendar

50%

Setup a read receipt*

28%

Create voting buttons*

34%

* Knowledge of this skill should have a positive impact on a teacher’s ability to communicate with the class community and or increase their ability to integrate this technology into their teaching.

Efforts that focus on increasing these skills should have a positive impact on the efficiency of the community to use Outlook as a tool for communication within the organization as well as with the classroom community.

Internet

The responses to the questions related to the World Wide Web (WWW) pointed to some discrepancies related to perceived competency levels. The school maintains a dedicated T1 line for Internet access. The Internet is one of the most prolific tools at the school. While employees may not have many of the other applications at home, most do have Internet access. When the Internet goes down for any reason during school hours, the calls immediately arrive at the help desk. It would be safe to say that everyone at the school uses the Internet sometime each day. With this type of usage, the responses to this part of the survey were surprising. 80% of the respondents rated their over-all skills with MS Internet Explorer and the WWW as competent or higher. The mean score for the skills measured related to the WWW was 3.39 placing it 4 th out of 6 (see Table 3) in the basic software competencies. This places competency related to surveyed skills related to the WWW somewhere between “limited: just learning” and “competent: can complete satisfactorily”. This discrepancy in the results might be related to the mix of answers from faculty and staff. Further analysis will need to be completed to determine the trends as they relate to additional training in this area as 47% of the employees surveyed do not want additional training with using the WWW as an educational resource. This number does include non teaching staff which would affect the results of this part of the survey.

The skill sets that ranked highest related to the WWW were the basic skills of simple searches, navigation, creating and organizing bookmarks, copying text from a web page and saving images. Skills that rated lower were searches using advanced Boolean logic, evaluation of websites for accuracy and relevancy, adjusting preferences in a web browser, citation of electronic sources (a skill needed by educators), skills related to authoring and posting a website, and skills related to downloading and installing software from the Internet.

The results of this part of the survey seem to point to a population that is well versed in the basic skills of using the Internet with little desire to grow beyond these skills. However, based on calls to the help desk and my personal interaction with employees at the school, there does seem to be a need for additional training as users are making requests that show their need to function beyond these basic skills. Further analysis and study in this area is needed to better determine the needs of specific constituents of the school community. It is not unlikely that the faculty would have very different skill requirements than the support staff and as such, training should be focused on differential skills of the various functional positions of the community.

MS PowerPoint

The results from the question set related to knowledge of MS PowerPoint were a bit surprising given the prevalence and potential of this tool. PowerPoint ranked lowest (mean score of 2.88 on a 5 point scale) in the core software related to specific employee knowledge and skill. This application has been available on campus since the introduction of the network. This tool has value as a teacher presentation tool as well as use as a tool for students’ project presentations. When asked about creating a new presentation, the responses seemed to be equally divided between the last four categories ranging from heard of it but don’t know how to Expert: can teach. Beyond that, the majority of the responses fell into the “heard of it but don’t know how” category. Yet, this application received a mean rating of 3.30 on a 5 point scale for over-all competency. While the results would suggest that teachers are not utilizing PowerPoint in their classrooms, the lack of use might also suggest a lack of engagement with technology in their teaching. Further study will need to be done here to determine the effect additional training would have on the use of this tool in the classroom. However, over-all lack of knowledge in this application suggests an area that must be explored for future development.

MS Excel

The final application being considered in this survey as a core application is MS Excel. This application scored second lowest of the core applications weighing in with a mean score of 2.90 (on a 5 point scale) on the specific skills being studied. However, when the study group was asked about their over-all competency in this tool the mean response was 3.63 (on a 5 point scale). This placed Excel above PowerPoint and almost on par with Internet Explorer. Most of the respondents felt they had a basic working knowledge of Excel but would not be able to perform tasks beyond simple spreadsheets.

This is a tool that has implications for productivity as well as integration into curriculum. There are several seasonal activities that are performed annually by many members of the community that require them to interact with existing spreadsheets. The help desk receives an elevated level of calls during these times for assistance on simple manipulation of these spreadsheets. In some cases, the work is passed on to others to perform as the individual responsible does not have the knowledge or skills to work at even the most basic of levels with this application. There have been extensive training opportunities in the past with full workshops given in this application. There have also been opportunities for development in one-on-one and small group environments. The calls logged at the help desk support the findings of this survey and additional training is necessary in this area. Yet, 33% of the respondents stated that they have no desire for additional training in this area. The remainder of respondents had expressed some desire for additional training in Excel.

Reactions to the structure and timing of professional development

Like most schools, time for training is an issue at this school. Teachers and staff have full schedules and setting time aside for training means giving something else up or getting a substitute to cover during the training time. In the past two years, the school has completed several studies by survey, consultant and committee that have pointed to time being a major issue.

One of my current training models emphasizes one-on-one training with standing schedules. Training is tailored to the specific needs of the individual and delivered with an emphasis on authenticity. However, with an employee base of over 250, this method can only reach a few that can set the time aside. I have also offered short workshops and classes on specific applications. These have been met with varying degrees of success. I have created a small collection of training tips and video clips but there has been no formal delivery method or training program related to these tips and clips. The subject of these tips and clips has been drawn from the logged calls to the help desk and interaction of the technology staff and myself with the employees.

The survey asked the respondents to select the best times and desired training methods for future training development. It was not a surprise to see that one-on-one training was preferred by almost 70% of the respondents. However, there was also a lot of interest in other training delivery methods and times. Chart 1 illustrates the preferred methods and times as reported by the respondents in the survey.

CHART 1

Training- Time and Delivery: Choice of structure and time of desired technology training.

1

One on one training scheduled at my convenience

2

One hour stand-alone workshop offered 4:00-5:00

3

One hour stand-alone workshop during the school day

4

Self-service training through tech clips and tech tips

5

Multi-day workshop offered during the summer

6

CBT on specific technology subjects

7

2 hour stand-alone workshop offered 4:00-6:00

8

Multi-hour workshop offered after school in 2 hour increments

9

2 hour stand-alone workshop offered 6:00-8:00

 

The one hour stand-alone workshop offered either during the day or immediately after school (4:00-5:00) was selected by about 50% of the respondents. This would provide many opportunities to attend a short focused workshop as it would be repeated on different days of the week at different times of day. The attendance at any one of the workshops would be expected to be low but the aggregate attendance might be quite high. This has never been done at the school and would be a new method of offering technology training. I believe that this type of training may also help address the issue of time as there would be multiple opportunities for any training session offered and the variety of times would allow faculty and staff to pick a day and time that works with their schedule.

Over a quarter of the respondents indicated a preference for the 2-hour after school workshop when the time was 4:00 to 6:00 but less than 10% were interested when the time is 6:00 to 8:00. Since this was one of the lower ranked forms of professional develop, 2 hour after school workshops will not be further explored at this time.

The interest in self-service training through tips and clips and the delivery of training through computer based training (CBT) were quite high. While there has been no formal deployment of these methods, there has been some experimentation with tips and clips. They were well received by some yet others still wanted only the written instructions. The method of delivery was by e-mail and there was not formal archive of these training tips and clips that was readily accessible. It seems that this might be an area were I can make some real progress can in training and development of current staff at the school.

General Reflection from Cycle One

The purpose of this survey was to determine through self report the level of technology competence of the current faculty and staff. This is a stepping stone to the goal of developing a technology training program for new faculty and staff in the style of an induction program as well as a way to better serve the current employees. It was important to determine the training needs of the current faculty and staff as some of them may play a role in a new training program. It was also important to determine the competencies of the current group of employees as this would help me determine necessary changes in my training methodologies.

This cycle was very involved and the analysis of the data will continue for some time. However, this first analysis of the data has answered some questions, affirmed some suspected trends and has raised further questions.

How effective was this survey as a form of self report to measure the technology competencies of the existing faculty and staff? Most of the results from this survey were inline with what might be expected based on anecdotal evidence from my personal interactions and those of the technology staff as we interact with the faculty and staff on a daily basis.

This survey seems to have been an effective instrument for assessing the level of technology literacy at the school. As a result, I feel this type of survey would work well as an instrument to gather similar knowledge of new employees prior to their arrival at the school. Some of the survey will need to be adjusted for this group as they will not be aware of the resources available at the school and the issue of “survey intimidation” will also need to be addressed. However, while altering the survey to better match the needs of a new employee, a portion could query them on prior experience with some of the technology resources available at the school thereby creating an awareness of the technology that would be available to them when they arrive at the school.

The data collected from the new employee survey will help me facilitate the ability to tailor a training program to the specific needs of these new employees. New users with stronger skills will be identified early. Users that might have difficulty with the technology at the school could be placed in relationships with more advanced users with the intent to share knowledge. This type of pairing could be facilitated by information gathered through a survey.

Another advantage of gathering this information prior to the arrival of the new employees is the opportunity for me to customize the new employee technology orientation. In the past, technology orientation has always been rather general as I had no prior knowledge of the technology abilities of the new employees. With the knowledge gained from the survey, the orientation can be tailored to the new employee group.