Alumni Assessment of the 1990s MIS Curriculum at a Liberal Arts University

Debbie B. Tesch
Gerald F. Braun
Elaine A. Crable
Department of  Information Systems, Xavier University
Cincinnati, OH, 45207-5161, USA
E-mail: tesch@xu.edu


Abstract

Information systems curricula at schools of higher education are constantly under pressure to update curricula to reflect current industry trends. Alumni from the 1990s of a Midwest liberal arts institution were asked to evaluate the business core, information systems core, and elective courses. The comparative value was analyzed. The perceived content value, in terms of quantity, for the information systems core was analyzed. 

Keywords:	IS curriculum, alumni evaluation, business core, information systems core 


1.	BACKGROUND

Information systems curricula at schools of higher education are constantly under pressure to update curricula to reflect current industry trends. This pressure comes from knowledge of industry expectations, requirements of accrediting bodies to provide evidence that graduates are provided with necessary knowledge and skills opportunities, and self-imposed expectations of IS faculty to prepare graduates for future employment.

One mechanism for curriculum evaluation is the periodic assessment of knowledge and skills requirements for business students in an information systems environment. Periodic assessment is a must if business schools are to be efficiently reactive to industry requirements when making changes to their curricula. Frequent assessments from relevant respondent groups can contribute to the efficient response of academia to industry requirements. Respondent groups frequently used in curriculum research include students about to graduate, recent alumni, supervisors of graduates, IS faculty, non-IS faculty, and program directors/coordinators (Ramakrishna, 2000).

Emphasizing the need for continuous curriculum assessment, relevant research describes the gap between IS expectations and academic preparation (Trauth et al, 1993). Trauth, et al (1993) identified two main areas of difference as the need for integration of fast moving technologies along with the need to manage career education. In addition, research describes methodologies for skills/knowledge assessment in a frequent, timely manner from relevant respondent groups (Chrysler and Van Auken, 1999; Hanchey, 1996;  Ramakrishna, 2000). Ramakrishna (2000) examined the similarities among different respondent groups in assessing information technology skills and information technology knowledge. ANOVA results indicated that when knowledge and skills categories were combined, recent alumni, supervisors of graduates, IS faculty, and program directors perceptions of knowledge/skills requirements were the same. Thus, researchers could avoid overusing respondent groups for periodic surveys and attempt to target groups most likely to provide the higher response rates.

Asking alumni to evaluate course content of required and elective courses, Chrysler and Van Auken (1999)  analyzed the comparative value of courses and correlated the value of each course to the length of time since the student graduated. The methodology they presented allowed MIS faculty to periodically determine which courses were highly valued and which courses needed to be redesigned or replaced. The goal was to assure that an MIS program offered courses students evaluated as having a strong content value. Alumni are important evaluators of program effectiveness. Measuring alumni perspectives can provide an opportunity for detached objectivity from previous participants in a program. Recent alumni can often differentiate between skills/knowledge that were a result of the educational experience and those skills promoted in the employment environment. In addition, alumni can provide an appropriate context for establishing long-term measurement of objectives (Hanchey, 1996).

In addition to establishing alumni as the respondent group and providing for an evaluation of required and elective course content, Hanchey (1996) suggested that survey instruments should consider questions of demographics, first job experiences, current job experiences, cooperative educational experiences, and questions that elicit responses about the strengths and weaknesses of an MIS program. Questions for analysis might be organized into categories of skills and knowledge such as technology, business administration, and interpersonal relations. Students should report the most useful courses/skills in both major and non-major courses, technical skills considered most important and software and programming languages currently used.

In an effort to identify skills needed now and in the near future for IT professionals, FedEx working with the University of Memphis and a host of vendors, outlined a list of recommendations for curricula which represented needed business skills (McGee, 1998). The template suggests course offerings for three levels of study. Table 1 summarizes this suggested list of baseline requirements (McGee, 1998). 

TABLE 1
Suggested Course Offerings by Level
General Education
Information Systems baseline
Internet certification

Accounting
Computer concepts and architectures
Building Web applications
Business management
Database concepts including SQL
Developing business strategies for Web applications
Project management
Data warehousing

Resource and time management
Relational technology

Organizational theory
Overview of C++ and Java






In a more recent study, Martz and Landof (2000) compared student and recruiter perceptions about what skills were considered important for an information systems career now, in three years, and for career advancement. Results indicated that students are failing to recognize the value of problem solving and analytical and conceptual skills, while tending to overvalue the areas of general legal knowledge and forecasting/predicting. Again in this study, characteristics were divided into a body of general skills and IT skills. The top five skills in each area 
perceived to be necessary for advancement were identified.

Consistent with the need to react to industry expectations of information systems graduates, faculty at a Midwest liberal arts university are developing a mechanism for ongoing feedback from recent graduates. More than half of the required hours for graduation come from liberal arts core courses at this particular liberal arts institution. A dedication to the mission to prepare students ?intellectually, morally, and spiritually? to take their place in a rapidly changing global society places severe restrictions on the number of available hours for coursework in the management information systems major. The Information Systems major consists of 18 hours of information systems courses. This purpose of this study was to determine the critical courses needed to teach in the limited hours of the major.

2. PURPOSE OF THE STUDY

The purpose of this research was to develop a methodology for evaluating the effectiveness of an information systems (IS) program in terms of both business and information systems courses and the quantity of instruction over a specified IS skills set. Specifically, the study was designed to (1) develop and test the evaluation instrument, (2) describe the employment environment of 1990s IS graduates and (3) survey 1990s graduates of an IS program to ascertain the value of business and information systems courses. Results and data analysis from this sample will provide the foundation and guidelines for an ongoing web-based implementation of this assessment instrument. This study describes the curriculum experience and evaluation of information systems alumni of the 1990s.

3.	METHODOLOGY

In the fall of 1999, a survey of 1990s graduates of an MIS program from an AACSB-accredited Midwest liberal arts university was administered. Based on an examination of the literature, the survey attempted to capture relevant demographics, information about first full-time employment, information about current employment, and a likert-scale evaluation of business courses, information systems courses, and a similar evaluation of quantity of instruction of IS skills. One hundred thirty-one (131) surveys were distributed and 41 returned, a 31% response rate. In addition to the task of analyzing student responses, developers of the survey were concerned with improving response rates and a mechanism for continuous assessment. 

Of major significance to the study was the development of a reference frame that would, for the most part, remain appropriate over time. To this end, the information systems skills statements were phrased to focus on course content. The list of information systems courses considered electives implemented in the late 1990s that would remain appropriate alternatives into the 21st century (i.e., E-commerce, Web Fundamentals). 

The scoring issue for business and information systems courses was addressed using a five-point scale allowing students to rate a course from very high value to ?of no value.? The five-point scale, however, did not force students to respond in one direction or another. For preparation in ?amount? of instruction, in terms of quantity, not quality, scoring of information skills instruction used a six-point scale. The even-numbered scale was used to force student responses in one direction or the other as follows:

Business problem solving
No coverage
Less coverage than necessary
Limited coverage
Some coverage
Sufficient coverage
More coverage than necessary

In preparation for an ongoing evaluation of the program, database tables were designed using Microsoft Access 2000. Access was chosen as the application that would be used to capture the data online. The database was populated with the survey records from data of completed paper forms. Refinements to the data format will be implemented in the online survey. As a means for data extraction and statistical analysis, tables were exported to Excel.  This step in the process requires little to no additional effort in terms of data entry, as the data is already organized into a spreadsheet format.  Again, this provides for easy manipulation and extraction of the collected data.
        
4. FINDINGS

Employment Environment of 1990s IS Graduates

Table 2 provides summary statistics for the respondents. Of the 41 responses, 56% were males and 66% in the twenty to twenty-nine age ranges. Forty-six percent had a GPA in the major of 3.5 or greater, whereas, only 29% had an overall grade point average of 3.5 or greater. Of the responses received, 88% graduated within 5 years of the survey.

Respondents were employed primarily in Ohio (78%) and bordering states of Kentucky and Indiana (93% in 3 states). Jobs were located in major cities (72% of provided responses). Of the 33 respondendents 3 states). Jobs were located in major cities (72% of provided responses). Of the 33 respondents indicating employment in an information systems related area, 36% worked in computer services and consulting. Nineteen of 39 responses (49%) have held the same job since graduation; 79% have held two or fewer jobs.

Respondents were asked to indicate whether knowledge of a specific type of technology tool was a requirement for employment and whether or not they used the specified technology. The technology tool areas indicated were: programming languages, PC productivity tools, web authoring tools, and ERP system development tools. Table 3 presents a summary of these findings. 

Required Business Course Content Evaluations

The next analysis performed was a determination of the average score of perceived values of business course content. The business courses compared were:
Financial Accounting, Managerial Accounting, Managing Information Technology, Microeconomics, Macroeconomics, Legal Environment of Business, Managerial Behavior, Business Finance, International Trade and Business, Principles of Marketing, Quality and Production, Business Statistics, and Managerial Communications.

Table 4 provides results of this analysis. As noted in this table, IS majors rated the IS related business core course, Managing Information Technology, most valuable to their career.

Information Systems Course Evaluations
Table 5 indicates the results of the alumni evaluation of the required Information Systems required courses for the major.  The courses perceived to be the most valuable were the senior capstone courses of Systems Analysis & Design and then the implementation of that design.  These two courses are taken consecutively during their senior year and involve analyzing and implementing an information system for a local service agency. This 30 week hands-on client experience was perceived to be very valuable.

TABLE 2
Summary Statistics for 1990s IS Graduates


Personal Data
Occupational Experience





Males
23

Employment
State:

Females
18

Ohio
Kentucky
Indiana
Maryland
Virginia
32
4
2
1
1





Age:


City Size:

20-24
25-29
30-34
35-39
40-44
>=50
12
15
6
5
2
1

>= 500,000
100,000-499,999
50,000-99,000

26
6
4




Graduation Year:

Business
Classification:

1999
1998
1997
1996
1995
1994
1993
1992
1991
1990
8
4
5
3
7
9
1
1
1
2

IT services
Consulting
Manuf. ? other
Insurance
Other
Banking
Retail
Finance
Government
Manuf.-computers
Utilities
6
6
5
4
4
2
2
1
1
1
1





GPA
Major
Overall
# of Companies
since Graduation:

>= 3.5
3.0-3.49
2.5-2.99
19
19
3
12
25
4
1
2
3
4
19
12
7
1

Table 6  provides the alumni results for their evaluation of the IS Elective courses.  The co-op experience was rated as the most valuable elective followed by the courses involving business via the World Wide Web.  

Table 7 indicates that the skill perceived to be the most valuable during the major was the team interaction occurring during the courses and then the database design and modeling skills. These particular skills were used extensively during the year-long senior project course.



TABLE 3
Knowledge Requirements/Utilization of Technology Tools

Programming
PC Productivity
Web Authoring
ERP
Required:
21
33
2
4
Used:
16
34
11
8




TABLE 4
Mean Scores for Business Courses
Course Title
Mean
Standard
Deviation
N
Rank
Financial Accounting
3.40
1.08
40
4
Managerial Accounting
3.55
1.03
38
3
Managing Information Technology
4.26
0.94
39
1
Microeconomics
2.63
1.04
41
12
Macroeconomics
2.60
1.08
40
13
Legal Environment of Business
3.18
0.90
40
8
Managerial Behavior
3.33
0.93
39
7
Business Finance
3.36
0.99
39
5
International Trade & Business
2.86
1.03
37
11
Principles of Marketing
3.00
0.95
41
10
Quality & Production
3.35
1.16
37
6
Business Statistics
3.10
1.10
40
9
Managerial Communications
3.92
0.85
38
2
TABLE 5
Mean Scores for Information Systems Required Courses

Course Title
Mean
Standard
Deviation
N
Rank
Structured Programming
3.75
1.05
36
5
Programming Applications
3.79
1.02
33
4
Data Modeling & Management
4.16
0.97
38
3
Systems Analysis & Design
4.29
0.87
38
2
System Development Project
4.38
0.83
37
1






5.	DISCUSSION

This study evaluated the perceived effectiveness of an Information Systems major at a Midwest liberal arts university.  The IS alumni respondents to the survey indicated the most important business core course to be the Management of Technology course, followed by the Managerial Communications and Managerial Accounting courses. The perceived key Information System courses were the senior project courses of Systems Analysis & Design and the System Development Project. Of the IS electives taken, the survey participants felt that the co-op course was the most valuable followed by e-business courses. When evaluating the importance of the key IS skills taught within the Information Systems courses, alumni indicated that the team interaction skills were the most 
valued followed by the database management and design skills. 
	Trying to identify and then teach the most valued technological skills is a moving target in the Information Systems field.  It is important that the field of Information Systems prepares their graduates with important fundamentals.  Surveying the alumni, as this study did, should be an ongoing process in order to prepare future members for this dynamic field.




6.	REFERENCES

Chrysler, E. and S. Van Auken, ?A Methodology for Alumni Assessment of an MIS Program.? Journal of Computer Information Systems, 39(4), (Summer 1999), 33-39.

Hanchey, C.M, ?An Assessment Model for Surveying Graduates.? Journal of Computer Information Systems, 36(2), (Winter 1995-1996), 67-75.

Martz, Jr., B.M., and G. Landof,  ?Information Systems Careers: A Comparison of Expectations.? Journal of Computer Information Systems, 40(2), (Winter 1999-2000), 41-45.

McGee, M. K. ?School daze.? InformationWeek, (February 2, 1998), 44-52.

Ramakrishna, H.V. ?Assessing Critical IS Knowledge and Skills Requirements for MBAs: How Similar Are Different Respondent Groups?? Journal of Computer Information Systems, 40(3), (Spring 2000), 32-37.

Trauth, E.M., D.W. Farwell, and D. Lee, ?The IS Expectation Gap: Industry Expectations Versus Academic Preparation.? MIS Quarterly, (September 1993), 293-307.






TABLE 6
Mean Scores for Information Systems Elective Courses

Course Title
Mean
Standard
Deviation
N
Rank
Data Communications/Networking
3.86
1.03
35
5
Hardware/Software Overview
3.93
0.90
28
4
Small Computers for Business
3.65
0.85
26
7
World Wide Web Fundamentals
4.00
0.84
21
3
Electronic Commerce
4.35
0.79
17
2
Contemporary Issues in IS
3.73
0.98
22
6
AI & Expert Systems
3.40
1.18
15
8
Co-op Education
4.48
0.81
21
1









TABLE 7
Information Systems Skills Instruction
Course Title
Mean
Standard
Deviation
N
Rank
Business problem solving
3.75
1.32
36
12
Programming:




   Interface design
3.25
1.48
36
17
   Programming logic
3.95
1.31
38
10
   Application development
3.97
1.26
37
9
   File/Database processing
4.00
1.32
38
7
Business application software:




   Spreadsheets
4.05
1.15
39
6
   Database management
4.00
1.21
38
7
   Graphics/presentation
3.45
1.29
38
15
System security
2.56
1.32
36
23
Computer ethics
2.95
1.29
38
20
Operating systems
3.21
1.23
38
18
Hardware specifications
2.97
1.26
37
19
Web site development
2.68
1.63
34
22
Logical design specifications
3.47
1.52
36
14
Physical design specifications
3.43
1.48
35
16
Project development
3.80
1.34
40
11
Project management
3.59
1.37
39
13
Network design
2.47
1.21
36
24
Network administration
2.25
1.16
36
25
Network construction
2.18
1.14
34
26
Database design
4.25
1.08
40
2
Data modeling & management
4.15
1.24
41
3
Normalization
4.14
1.31
36
4
E-R Diagrams
4.09
1.38
33
5
SQL
2.85
1.46
34
21
Team interaction
4.72
0.92
39
1