A study of personal productivity software skills and desired internet features in a web-enhanced introductory technology skills course Thomas N. Janicki Information Systems Department - University of North Carolina - Wilmington Wilmington, NC 28403 Geoffrey Steinberg Information Systems Department - Kent State University Kent, OH 44242 Abstract A challenge for any introductory Management Information Systems course is to provide an increasing level of computer literacy for the office productivity software packages. It is important to understand the level of proficiency in the 'office' suite software prior to the students beginning an introductory Management Information Systems course. This paper summarizes annual surveys conducted over a four year period that detail students' perceived skills in office productivity software as they entered an introductory technology skills course. A trend of increasing knowledge of office productivity software prior to enrolling in an introductory technology skills course is shown. Another goal of the survey was to detail what features are desired by students in the areas of communications and learning assistance in web-enhanced and full web-delivery courses. Students reported that practice quizzes, personalized feedback on assignments, on-line grade books and on-line handouts as the most utilized features and best learning features that should be incorporated into web-based materials. Keywords: , Personal Productivity Software, Computer Mediated Communications (CMC), Internet Usage, Instructional Technology 1. INTRODUCTION The Internet and the World Wide Web is changing the way universities and colleges are offering their course work. According to the National Center for Educational Statistics more then 78% of public four year institutions offered distance education courses in the 1998-1999 academic year (Lewis et. al., 1999). Distance education courses come in many varieties forms, from all lesson content and tests are on the Internet (with no traditional classes), to courses that use the Internet as an additional communications and learning vehicle to enhance the traditional classroom learning experience. In addition a hybrid model is also developing where a mixture of traditional course material is presented along with independent web based learning content. Commercial developer tools such as WebCT, Blackboard, and TopClass have been implemented at many universities to assist instructors in the presentation of course materials and the tracking of student progress. These tools provide the instructor the capability to test students on-line, provide status reports, bulletin boards and chat rooms as part of their basic knowledge. In addition many instructors build their own web-sites comprised of various options to enhance the learning and communications between student and instructor (Janicki and Liegle, 2001). Many educators, students and employers intuitively feel that the integration of the computer with its interactive capabilities into a classroom or learning experience will enhance learning and the student's ability to apply knowledge and skills to future problem solving situations (Alavi, 1994). Despite this belief, Alavi states that this new technology has not integrated sound pedagogical practices into the development of new learning modules either as a stand-alone lesson or combined with a classroom setting. Alavi continues that the effective use of computers as a learning tool in a classroom requires a departure from traditional instruction modes so that the technological mediated communication in the classroom becomes pedagogically superior to the alternative modes of instruction. She calls for additional research to provide information and insight into the design and implementation of this emerging information technology in the learning process. An additional challenge for any instructor of a technology skills course it the growing experience with personal productivity software of the learners prior to enrollment in the course. This evolution of skills requires the instructor to provide challenging new assignments with personal productivity software as well as introductory materials for those students who might not have been exposed to the software prior to the course. We define the personal productivity software to encompass word processing, spreadsheet, database and presentation software applications. This paper summarizes student responses to surveys completed annually for four years in an Introduction to Management Information System (MIS) course offered at two public universities. The survey had questions that covered three basic areas. They were: 1. What are the office productivity software skills students already possess before entering an introductory MIS course? In addition are these skills increased as a result of completing the introductory Information Systems course? 2. What are the features that should be incorporated to the electronic side of a web-based course or web-enhanced course? Specifically what are the best features to enhance learning and communications between the instructor and student in web-enhanced courses? For purposes of this paper we define a 'web-enhanced' course as one in which the instruction is delivered in a traditional classroom with the Internet supplementing the traditional classroom delivery 3. An overview of the today's student in terms of hours worked as well as uses of the Internet. 2. SURVEY DESIGN AND DEMOGRAPHICS OF RESPONDENTS As part of the Introduction to Management Information Systems course at two different public universities a survey of attitudes and perceptions of computer related issues was offered to students. This survey was administered at the beginning and at the end of the semester in each course over a four year period. The survey was offered in an on-line fashion. As an incentive to complete the survey students were awarded extra credit points in the class. The survey averaged between 35 and 45 questions over the four year period. Twenty-five questions were constant in all survey years, while some questions were added or subtracted over the year. Unless otherwise indicated the results shown in this paper are for all four years of study (Identical questions were present in all four years of the survey). Note that some percentages will not add to 100% as some students did not respond to all questions. In addition most survey results are reported for the end of the semester. Four Year Averages Number of Respondents 418 Sex Female 49.4% Male 50.4% Class Year Freshman 29.6% Sophomore 42.6% Junior 19.2% Senior 6.9% Prior Information Systems courses 0 84.3% 1 11.32% Grade Expected (results from survey at beginning of the semester) A 83.7% B 14.3% C 0.4% Table 1: Four Year Survey Average Results Table 1 details demographics for the four year average in which the trend did not change over the four year period, while Table 2 details those demographics where some trends are notable. The typical respondent was in their first Information Systems course, a freshman or sophomore and during the survey period and expected an "A" in the course. The survey averaged between 35 and 45 questions over the four year period. Twenty-five questions were constant in all survey years, while some questions were added or subtracted over the year. Unless otherwise indicated the results shown in this paper are for all four years of study (Identical questions were present in all four years of the survey). Note that some percentages will not add to 100% as some students did not respond to all questions. In addition most survey results are reported for the end of the semester. Table 1 details demographics for the four year average in which the trend did not change over the four year period, while Table 2 details those demographics where some trends are notable. The typical respondent was in their first Information Systems course, a freshman or sophomore and during the survey period and expected an "A" in the course. Table 2 details beginning trend information. A most interesting trend is the decreasing number of Information Systems majors at both universities. It was above 15% in 1999 and 2000, but dropped to the 10% range in 2001 and 2002. This could clearly be a result of the decreased opportunities in the Information Systems field following the collapse of the dotcom's in 2000 and 2001. Also of note is the percentage of students working full or part time. Years 2000 and 2001 were approximately 70% while the number of students employed in 2002 dropped to 59%. The number of hours worked was highest in the 11 to 20 hours per week range. A new question asked for the first time in 2002, was the number of all web-based courses students had completed prior to this course. Over 25% either had completed one or two web based courses. This will be an interesting question and trend to watch into the future. 3. OFFICE PRODUCTIVITY LITERACY SKILLS A goal of the survey was to determine if basic computer literacy skills in the office productivity software applications of the respondents prior to the course were increasing. High schools and even elementary schools are offering computer skills in word processing and spreadsheets, which increases the skills of incoming students. Figure 1 details the comfort level with personal computers of students upon entering the class. It is interesting to note that respondents' level in the high and above comfort levels has been dropping over the three years the question was asked. A possible explanation is that as more and more features and software is added to machines (DVD's CD-Writers, Cameras, and Photo Editing) the student felt less and less comfortable and less knowledgeable with all the new features and functions. High or above comfort level went from a high of 61% in 2000 to 41% in 2002. Figure 1: Comfort level with Personal Computers Office Productivity Software Skill Levels Prior to the Introductory Course To detail their skills before coming into the MIS course, each student was asked to rate their knowledge of the office suite of products. Specifically they were asked if they knew: the basics of each software package; had minor knowledge; or had knowledge of the special features of the package. Shown in Figures 2 and 3 are the results of the survey. It can be noted that the highest level of skills was shown in word processing and decreases for the spreadsheet, database and finally presentation graphics packages. It is interesting to note that over 95% of the students had some exposure to word processing software before their first MIS course, and over 80% had exposure to spreadsheet software. Database exposure went from 70% to 88% and Presentation exposure went from 56% to 77% in just two years. It is also noted that the level of skills in all software applications acquired prior to the introductory course has also increased.. As a footnote a post course survey was completed to gauge their increase in basic and advanced skills gained. As would be expected (since skills training was part of the course, spreadsheet skills increased an average of 18%, with database, 9%, presentation 11% and HTML skills up by 26% 4. COMMUNICATIONS AND PEDAGOGY FEATURES IN A WEB-ENHANCED COURSE The web may be used to increase communications between the instructor and student. This is especially true in larger class environments where the student may feel isolated. A University of California at Santa Barbara (1995) study details problems found in classes. The UCSB study offers several recommendations particular for large sized classes: 1. personalize, personalize, personalize 2. ask students for feedback 3. give feedback to students early and often. Perrin and Rueter (1998) report that the Portland State University is facing the same dilemmas as classes increase in size. They are turning to technology to assist in communication and learning pedagogy in the forms of multimedia presentations, the Internet, electronic study guides and computer-based tutorials, simulation, email and threaded on-line discussion groups. The question becomes how to implement the above suggestions. Communication options The web may be used to increase communication and personalize communication through an on-line grade book or 'status report', as well as personalized calendars. WebCT and BlackBoard as well as other popular course management tools offer bulletin boards, threaded discussion groups and 'live' chat rooms. In addition customized emails, progress reports and immediate feedback on assignments and tests increase the level of personalization and communication (Janicki and Steinberg, 1999). Students indicated that an on-line grade book is the most desired feature in terms on communications (Table 3). In a subjective question students indicated that the on-line grade book provided them the information of where 'they stood' in the class. Interesting many of the course content applications packages (WebCT, BlackBoard) stress the use of bulletin boards and live chat rooms. However in separates question pertaining to usage of communications features, 85.7% indicated they never used the live chat feature of course, and 59% never used the bulletin board feature. Best Communication Feature (%) 2nd Best Communication Feature (%) (On-line Grade Book 74.6 21.5 Handouts On-Line 18..5 60.6 Bulletin Board 3.6 11.0 Live Chat Room 0.6 1.9 Table 3: Survey Question: What is the best communication feature of the web portion of the course? Pedagogy support Web enhanced course may offer additional learning opportunities to students. Included are tutorial learning modules, practice quizzes, study guides, on-line exams and systems that provide immediate grading and feedback on assignments and quizzes. Table 4 details which features were rated high to the question of which feature enhanced your learning opportunity. Students liked the capability to take practice quizzes on-line and felt they learned the most from them. Rated as the next best learning tool was enhanced feedback on assignments of what was incorrect on electronically submitted and graded homework assignments. This supports the idea of personalization detailed previously. Additional learning tools included study guides for future exams to help the students focus on the key points from the learning material. They also indicated that detailed feedback on questions missed on quizzes enhanced learning. Best Learning Tool 2nd Best Learning Tool Practice Quizzes 51.3 22.7 Immediate feedback on electronically submitted homework 16.7 11.6 Study Guides 13.4 25.7 Answers given to on-line quizzes for questions missed 9.25 17.8 On-Line quizzes 9.0- 17.9 Table 4: Survey question: What is the best featured that increased your learning from the web-enhanced portion of the course 5. INTERNET USAGE STATISTICS A third goal of the research was to notice any trends in Internet usage by today's college students. Usage of the Internet for research peaked approximately 80% when combining the 1 to 5 hours per week columns as shown in Figure 4. 97% of students do indicate they use the Internet for school research on a weekly basis. No significant change in hours above 5 for research was noted. Personal usage of the Internet has not changed significantly over the three year period. 96% of the students access the Internet for non-academic reasons, with the majority of the students in the range of non academic time on the Internet in the 1 to 5 hours a week. Currently the survey indicates that students spend more time for school research projects then non research projects on the Internet (excluding email activities). The typical uses of the Internet for non-academic reasons are detailed in Table 5 and Table 6. As can be seen emailing and school research are the main activities and have comprised over 50% of the activities while connected to the Internet. Showing strong increases over the three year period are both the downloading of music files (MP3's) and Live and Instant Messaging Chat. The playing of games on-line was also questioned and maintained about a 2% usage rate over the course of the surveys. 2000 2001 2002 3 Year Average Email 47.6 42.7 38.2 42.7 Research 17.8 19.1 13.6 16.9 Downloading Music 9.8 10.1 15.1 11.6 Live Chat 6.5 16.1 15.1 11.4 General Surfing 12.3 9.3 10.9 10.9 Table 5: Main Activity while on the Internet (% Time Spent) 2000 2001 2002 3 Year Average Research 30.3 21.2 25.8 26.0 Email 23.3 25.4 26.2 25.0 General Surfing 15.3 14.9 14.2 14.8 Downloading Music 12.5 17.3 13.8 14.4 Live Chat 9.3 12.1 10.7 10.6 Table 6: Second Activity while on the Internet (% of Time Spent) 6. FUTURE RESEARCH Future research will continue to report any continuation of trends or any new trends. In addition new questions were added in 2002 to the survey. An interesting area is the computer literacy skills possessed by the students. There is a need to objectivity assess the skill levels of incoming students versus the current basis of asking their 'knowledge' of skills. The fall 2002 survey has been modified to ask questions of specific features in the software applications versus letting the survey respondent to determine what advanced versus basic features are in the applications. An example would be: Can you create a Macro in Excel without using the Record Macro Wizard? An additional area for future research revolves around questions added to the survey in 2001. There is evolving different means to deliver courses, from a totally web-based course, to a web-enhanced course (where a significant amount of communications and information is exchanged) to a hybrid of both where courses might meet periodically in a traditional class environment while the web is used for significant learning materials. 7. SUMMARY The survey set out to determine knowledge, usage and trends in three basic areas over a three to four year period. The level of productivity software application skills that students possess did increase over the period of the study (2000-2002). The survey indicated more then 90% of the students had basic to high levels of word processing skills prior to taking an introductory MIS course. In addition, previously learned spreadsheet skills advanced from 82% to 93% of students over the three years. Students had less exposure to database, presentation, and HTML software prior to the class. Database knowledge grew from 71% to 89% in the three year period, while Presentation software started at 42% and grew to 78% over the same time period. It is evident that incoming students have more knowledge of typical office productivity software in just the three to four years of this study. The communications features most desired by students in web-enhanced and totally web-delivered courses were an on-line grade book and the availa-bility of handouts for downloading. Show-ing only minimal desires were bulletin boards and live chat areas for communications between instructors and students. The two pedagogical features that enhanced learning were on-line practice quizzes and personalized feedback to homework assignments. The hours per week that those students spent on the Internet remained in the 1 to 2 hours/week for 35% of the respondents and 3 to 5 hours/week for an additional 40% of students. Recreational use of the Internet averaged 1-2 hours/week for 35%, 3-5 hours/week for 25% and 6 to 10 hours/week for 23% of the respondents. The number one activity on the Internet was emailing, with research, downloading music and live and instant messaging chat following in terms of popularity. 8. REFERENCES Alavi, M. (1994). Computer mediated collaborative learning: an empirical evaluation. MIS Quarterly, 18 (2), 159-174. Janicki T. and Jens Liegle (2001). "More than class notes? A features review of current web-based course management tools and their adherence to accepted learning pedagogy", Allied Academies of Education Leadership Journal. 4(2), 36-51. Janicki T., and G. Steinberg. (1999). "Improving student satisfaction in large-sized classes through a computer-mediated communications system", Journal of Information Systems Education. 10(2), 8-13. Lewis, L.; K. Snow, E. Farris, D. Levin (1999). Distance Education at Postsecondary Education Institutions. U.S. Department of Education Office of Educational Research and Improvement. (NCES 2000-013). Perrin, J. and J. Rueter.(1998). "Curriculum Revision with Educational Technology: Improving Student Outcomes in Large Courses", US Department of Education: Post Secondary Education, June http://edtech.clas.pdx.edu/ University of Santa Barbara, School of Education http://id-www.uscb.edu/IC/Services/IN/IN_F95.htm