To Automate Or Not To Automate, That Is the Question:
A Research Framework for the Integration of
Computer-Based Training in IS Curricula

WooYoung Chung
Department of Information Systems, Case Western Reserve University
Cleveland, OH  44106


Abstract

Due to highly publicized school rankings, an increased number of corporate universities, and the proliferation of distance learning options, IS educators are under considerate pressure to improve their levels of efficiency. Automated learning systems such as Computer-Based Training (CBT) could significantly enhance the efficiency of IS education. However, many researchers and educators are skeptical of the pedagogical effectiveness of automated instruction, claiming that computers are no substitute for human instructors. Because of this skepticism, the use of automated learning systems in IS education still remains controversial. To promote systematic research on the effectiveness of automated learning systems, I address in this paper this skepticism about automated instruction. In particular, I present a contingency framework for investigating the effectiveness of automated learning systems. The basic premise is that automated learning systems do not need to be effective in all situations to be deemed useful. If these systems are used for those situations in which they are most effective, the gains in efficiencies, especially in terms of instructors' time, could be redirected to other teaching and research activities. Hence, I argue that systematic research should be conducted to identify the contingencies for which automated learning systems could be effectively used.

Keywords: Education, information systems curriculum, instructional technology, automation of education, training



1. INTRODUCTION

Undoubtedly, educational institutions are under significant pressure to improve their overall efficiency (Alavi, 1994; Arge, 1999; Couger et al. 1995; Lee, Trauth, and Farwell, 1995; Leidner and Jarvenpaa, 1995; Mowday, 1997). Highly publicized school rankings, a rapidly increasing number of corporate universities, and the proliferation of distance learning have accelerated competition among universities and colleges. In order to compete effectively, these educational institutions must enhance their quality of instruction and offer more educational content than ever before while simultaneously increasing their research output. 

IS degree programs are no exception. Indeed, rapid advances in information technologies have significantly expanded what needs to be taught in IS curricula-both technical skills and business functional knowledge. These technological advances also increased the number of different career options for IS graduates (Couger et al. 1995; Lee et al. 1995). Many IS programs and educators are faced with the challenging task of providing flexible curricula in order to accommodate the increasingly diverse career aspirations of their students. 

Incorporating the use of automated learning systems such as computer-based training (CBT) or computer-aided instruction (CAI) could dramatically improve the efficiency of IS education. Indeed, similar types of automation have been widely used in many industries and have revolutionized efficiencies (Frank and Cook, 1995; Hamer and Champy, 1993). For example, Internet banking and ATMs perform many functions that used to be carried out exclusively by human tellers. These new technologies have significantly improved the efficiency of many banks, thus enabling them to provide convenient services to consumers at a low cost (Brynjolfsson and Hitt, 1998). In addition, advances in broadcast and consumer electronics technologies have allowed movie production companies to achieve extraordinary levels of economies-of-scale (Frank and Cook, 1995). Thanks to wide acceptance of these technologies, movie production companies can afford to invest as much as $200 million to produce a single movie, yet charge customers only a fraction of this cost to view or own a copy of the production.

These examples demonstrate how IS education might benefit from automated learning systems. For instance, if one could, in the manner of film production companies, capture "best" teaching practices, lectures, and other forms of educational contents in computerized interactive learning systems, IS programs could effectively provide instructions on a wide array of topics, which their student could learn at their convenience and at their own pace. In addition, the instructors' time that is freed up when these automated learning systems are used could be directed to other teaching and research activities to strengthen the program. 

On the other hand, despite these potential benefits, the use of automated learning systems still remains controversial among IS educators and researchers. In particular, some educators and researchers question the pedagogical effectiveness of these automated learning systems, claiming that automated instruction is no substitutes for human direction (Leidner and Jarvenpaa, 1995). In addition, diverse political and practical issues regarding the implementation of these automated learning systems, such as intellectual property rights of the systems and the autonomy of faculty have not been adequately addressed in the academic community (Arge, 1999). Consequently, while collaborative learning, distance learning and other instructional technologies have received much attention from educators and researchers, automated learning systems have been, by comparison, rather neglected in IS education (Alavi, 1994; Alavi, Wheeler, and Valacich, 1995; Leidner and Jarvenpaa, 1993, 1995). 

The central objective of this paper is to promote pedagogical research on the use of automated learning systems in IS education. Toward that end, in this paper, I address the skepticism surrounding the effectiveness of automated instruction. In particular, a growing body of evidence supports the contingency view that the relative effectiveness of each teaching approach varies widely depending on the circumstances in which the method is employed (Lengnick-Hall and Sanders, 1997; Lim et al. 1997; McGrath, 2001; Morrison, 1993; Schloss, Wisniewski, and Cartwright, 1988; Schultz, 2001). Hence, in this paper, I present a research framework that IS researchers could use to conceptualize the contingencies for the effective use of automated learning systems. The basic premise is that, by identifying these contingencies, IS educators would be able to incorporate these learning systems into the IS curriculum and redirect these efficiency gains from this automation to other teaching and research activities. 

In this paper, I first discuss general criticisms about the pedagogical effectiveness of automated learning systems. I then present a research framework that facilitates and promotes the systematic investigation of contingencies in which automated learning systems would be effective in IS education. 

2. CONTROVERSY ON EFFECTIVENESS OF AUTOMATED INSTRUCTION

Some researchers and educators question the effectiveness of automated instruction for several reasons. First, it is assumed that automated learning systems primarily present a codified fixed body of knowledge in rather linear sequences. Hence, students using these systems tend to learn in a rather passive mode. However, IS education should not focus on simply conveying a fixed body of skills and knowledge to students. To become a competent IS professional, students must learn how to think independently and critically in order to provide innovative solutions to complex and novel business problems (Lang and Dittrich, 1982; McGrath, 2001). Received wisdom suggests that a human instructor would be more capable of providing flexible learning environment than an automated system could and could also accommodate the students' independent and creative thinking process. Specifically, a human instructor can help students creatively explore new ideas and assimilate knowledge and skills on their own terms-a process often referred to as constructive learning (O'Loughlin, 1992; Lang and Dittrich, 1982; Leidner and Jarvenpaa, 1995). 

Indeed, even the most sophisticated information systems could not carry on a substantive conversation with a person. It would be quite difficult to program computer systems to help students think critically and creatively. Consequently, some argue that expanding the use of automated learning systems in IS education could over-emphasize the importance of passive learning and the assimilation of a fixed body of knowledge in an IS curriculum. 

Second, educators not only teach knowledge and skills, but they also motivate and inspire students to learn. Studies demonstrate that such students' motivation and engagement is essential to a successful educational outcome (Alavi, Wheeler, and Valacich, 1995; Arthur and Airman-Smith, 2001; Edmonson, 1999; Mathieu, Tannenbaum, and Salas, 1992; Martocchio and Webster, 1992; Senge, 1994; Webster and Hackley, 1997). Some researchers and educators question whether or not an impersonal computer system could effectively motivate and inspire students. 

Third, automated learning systems are primarily used in an isolated setting where a student interacts solely with a computer. Many studies on collaborative learning support the idea that students significantly benefit from interacting with other students (Alavi, 1994; Alavi et al. 1995; Bagley and Hunter, 1992; Flynn, 1992; Lim et al. 1997). For instance Lim, Ward, and Benbasat (1997) suggests that students who learn how to use information systems in groups of other students learn more than those who work alone because they must interact with one another and articulate what they are learning. Alavi and her colleagues (1995, 1997) also stress the importance of knowledge sharing among students during the learning process. Expanding the use of automated learning systems in an IS curriculum could reduce the interaction among students, limiting the opportunities for collaborative learning. 

Fourth, a human instructor would be much better at helping students to assume a professional identity than any computer system could. Studies have demonstrated that professional identity plays an important role in career development (Biddle, Bank, and Slavings, 1987; Brown and Starkey, 2000; Dutton, Dukerich, and Harquail, 1994; Ibarra, 1999; Morrison, 1993). In particular, Ibarra (1999) suggests that people become professionals or executives by observing and internalizing informal social norms, attitudes and routines of other professionals and executives. It would be quite difficult to facilitate such a socialization process using impersonal computer systems. 

Due to all these potential problems, some researchers and educators claim that automated learning systems are no substitute for human instructors. However, one should carefully examine the potential deficiencies of automated learning systems for several reasons. 

First, to some extent, automated learning systems could provide a flexible learning environment in order to facilitate constructive learning and the creative exploration of knowledge and skills. For instance, automated learning systems do not need to present materials in a relatively linear sequence. Automated learning systems include hypertext and diverse forms of search capabilities. Using such systems, students could acquire knowledge and skills at their own pace and at the most appropriate time. 

In particular, one should keep in mind that automated learning systems could provide a significant level of economies-of-scale. Once designed and implemented, a large number of students could use the system just as easily as one. If these systems are to be widely used in an IS curriculum, significant resources could be used to produce a complex automated learning system that could effectively anticipate the diverse "paths" of learning that students may follow. 

Second, automated learning systems do not need to be impersonal. The idea that computer systems cannot motivate, inspire, or socialize students into a professional community is partly based the assumption that computer systems are quite impersonal and that they are to be used to convey a fixed body of sterile and codified knowledge. However, studies demonstrate that such general assumptions about computer systems should be carefully examined (Chung and Henderson, 2001; Markus, 1994). Depending on methods of presentation, contents, and the situations in which the system is used, students may perceive automated learning systems as a rich medium of communication rather than a cold storage of information. 

For example, when combined with multimedia technologies, automated learning systems could provide instructions by showing a video presentation of an instructor. This could be perceived as more personal than a textual presentation. If the instructors also discuss their own personal professional experience during the video presentation, students may perceive such discussion as even more personal in nature. In addition, if students get the chance to interact with the instructor in the video face-to-face they may perceive the automated learning system as an extension of their personal interaction with the instructor. 

Third, automated learning systems could provide a constant and reliable quality of instruction. There is no doubt that in an ideal situation-a competent instructor spending as much time as needed with each student-a human instructor would outperform any automated learning system. On the other hand, in reality, instructors' levels of knowledge and skills vary widely; additionally, they can only spend a limited amount of time with each student individually-much interaction between instructors and students is completed in classroom settings. With these practical constraints, the quality of instruction that human instructors provide could significantly fluctuate. 

For similar reasons, the quality of collaborative learning can also vary widely. Many IS programs have students with diverse backgrounds, skills, and motivation levels. In some situations, this diversity can be quite beneficial-students learn from one another, complementing each others' skills and sharing their experiences. However, as much as skill and motivation levels could synergistically enhance learning effective-ness, they can also diminish the quality of learning. Automated learning systems, on the other hand, eliminate any variation in learning effectiveness that could be attributed to these "human" factors. In short, these systems provide a constant standardized quality of instruction. 

Finally and mostly importantly, automated learning systems do not need to be effective in all aspects of the educational process to be useful. Even if the use of these systems is not appropriate for some learning objectives, such as constructive learning and creative exploration of knowledge and skills, these systems could be used for other purposes. For example, even if automated learning systems could effectively present factual information only, such as knowledge and skills that are commonly taught in a lecture format, by using such automated learning systems outside a conventional in-class lecture, instructors and students can spend more time together focusing on constructive learning or the creative exploration of new ideas. 
In addition, one should also consider the synergy between the use of automated learning systems and other learning approaches. For example, the use of automated learning systems outside traditional classroom lessons could enhance in-class interactions among students and instructors. 

For all these reasons, despite the skepticism and criticism surrounding the expanded use of automated learning systems, I argue that IS educators and researchers should carefully consider the effectiveness of automated learning systems. In particular, as the first step, systematic research should be conducted to identify the contingencies in which automated learning systems could be most effective. In the following section, I present a contingency model for investigating these contingent situations (see figure 1).

3. CONTINGENCY MODEL OF AUTOMATED INSTRUCTION

One of the common ways to classify educational topics in an IS curriculum is to separate technical topics from business functional topics (Couger et al. 1995; Lee et al. 1995). Information Systems is an interdisciplinary field that encompasses diverse specialization in business and computer engineering. A competent IS professional should have both functional business knowledge and technical skills (Couger et al. 1995; Lee et al. 1995). It would be helpful in designing and planning a curriculum to be able to determine if automated learning systems would be more effective in either or both  functional business knowledge and technical skills. 

Both technical and business-focused skills could be taught at varying "depths" of learning (Davis and Bostrom, 1993; Lang and Dittrich, 1982; Santhanam and Sein, 1994; Lim et al. 1997; Vandenbosch and Higgins, 1996). These depths could be mapped onto a continuum ranging from problem solving skills for highly structured problems to highly unstructured problems (Gorry and Scott Morton, 1971; Simon, 1977). In this paper, the problems I refer to as highly structured are in fact the relatively simply problems to which standard and clear-cut solutions could be easily applied. Highly unstructured problems are complex problems for which no standard solutions exist; multiple sets of knowledge and skills must be identified and creatively combined in order to devise appropriate solutions. Semi-structured problems fall in the middle of this continuum For these problems, a few standard solutions may exist, yet the application of these solutions requires significant a mental effort. I argue that these intended levels of understanding could significantly impact the effective-ness of automated learning systems (see Table 1). In the following, I present three levels in which automated learning systems could be used: 1) retention; 2) application; and 3) synthesis. 

First, the retention-level teaching focuses on the presentation of factual information. The objective at this level is to familiarize students with diverse technologies, methodologies, and business frameworks. This would be the end objective in many introductory survey courses that focus on the breath of topics. In other courses, this retention of knowledge and skills would serve as a foundation or a starting step in actually learning these skills which can later be applied in a complex problem solving situation. 

Although achieving this level of understanding should not be the desired outcome in many situations, identifying the effective methods of presentation that enable the efficient retention of factual information has become quite important in IS education. In particular, the proliferation of new advanced technologies and their applications in business organizations-such as diverse object oriented development platforms, N-Tiered architecture, new networking standards, and ERP solutions-significantly expanded what IS graduates should be at least aware of, if not proficient. This includes technical knowledge and skills such as the use of "point-and-click" graphic user interfaces of object oriented systems development platforms and program-ming language syntax. Examples of business knowledge and skills at this level are business models for e-commerce and case studies that examine the implementation of diverse technologies. In addition, every year many new hypotheses and theories are tested and reported in IS journals. This ever increasing cumulative body of knowledge in the IS field needs to be presented to the students. For these reasons, although this level of learning should not be the final educational outcome, increasing the efficiencies of retention-level teaching could significantly enhance the overall quality of instruction. 

Conventionally, textbooks and lectures are widely used for this teaching purpose (McKeachie, 1990). On the other hand, advances and the wide acceptance of hypertext and multimedia raises interesting questions about how effectively students could learn by watching recorded lectures rather than by attending live classroom lectures and whether or not students would learn more by using multimedia hypertext learning systems rather than conventional textbooks (Ives, 1994).

Second, the application-level learning focuses on the creative application of a given set of knowledge and skills. By knowing and retaining factual information only, students would be able to solve only highly structured simple problems. In order to creatively apply what they have learned to relatively complex problem solving situations, these students would need a deeper understanding of their knowledge and skills, including a grasp of the underlying conceptual model (Lim et al., 1997; Santhanam and Sein, 1994; Vandenbosch and Higgins, 1996). In addition, they would also need to practice the application of their knowledge and skills in





FIGURE 1 Contingency Model of Automated Learning Systems


Automated Learning Systems	             FIT		Teaching Effectiveness
Interactive videos, hypertext or 
searchable knowledge database,
simulations, "drill-and-practice" 

				Contingencies 
				Topics: 	Technical skills
Functional business knowledge

				Depth of Learning: Retensio-level, 
					application-level, synthesis-level



TABLE 1. Teaching Objectives and the Features of Automated Learning Systems

Teaching Objective	Conventional Teaching Methods	Automated Learning Systems

Retention Level	Textbooks, Lectures	Interactive videos could be used to present
 		     	recorded lectures.

		Printed textbooks could be converted to 
			computerized interactive knowledge 
			base using multimedia, hypertext, and 
			diverse search engine technologies. 

Application Level	Lectures, Exercises, 	Computerized exercises and diverse forms of 
	Assignments, Small Case Studies		simulations can provide feedback 
			including video demonstrations 
			and commentaries.

Synthesis Level	Case Discussions, Term Projects	Interactive video could be used to present 
	Internship Projects		recorded (and scripted) case discussions.
		
		Multimedia and hypertext technologies could be 
			used to enhance and computerize 
			conventional case studies.

		Interactive video and simulations could be used 
			to provide a complex unstructured problem
			solving situations.



order to solve semi-structured problems-problems where the necessary knowledge and skills are clearly defined, yet the application of each requires a significant mental effort. 

Complex algorithm design and relational data modeling would be good examples of this application-level of learning. Simply memorizing computational theories, diagramming notations, and formula would be of little use in actually solving complex problems of these kinds. In addition, students could read business cases on business reengineering and memorize related reengineering guidelines. Yet, this level of understanding would not enable these students to analyze, redesign, and optimize business processes. 

In terms of application-level teaching, diverse forms of exercise, case studies, assignments, and class projects are widely used. These exercises are constrained enough that students can readily identify the necessary knowledge and skills. Instructors provide feedback on students' work and they also demonstrate how they themselves would complete these exercises. 

It would be difficult to program an automated learning system to provide individualized feedback on student's work. However, "drill and practice" types of automated learning systems could include an interactive video demonstration of how the given set of knowledge and skills could be applied. Simulations can also provide similar feedback-interactively providing whether or not the student correctly applies the given set of knowledge and skills (e.g., Leidner and Jarvenpaa, 1993). 

The teaching objective for the third level, synthesis, is to enable students to deal with highly unstructured complex problems. Unstructured problems could be interpreted via multiple perspectives; therefore, diverse sets of knowledge and skills could be applied to the situation. In addition, personal experience, intuition, and knowledge and skills acquired from work experience and prior education could be all useful in dealing with unstructured problems. 

Hence, at this level of learning, students must learn to understand diverse perspectives and creatively integrate all knowledge and skills. In particular, students should also learn how to articulate their reasoning and work with those who have different views and opinions. 

Case study discussions, often based on the Socratic approach, have been widely used in capstone or IT strategy courses to facilitate this level of learning (Lang and Dittrich, 1982). In these discussions, instructor and students present their own interpretation of case studies, share their personal experiences and opinions, and provide feedback to one another. Through these interactions with others, students (and the instructors) learn how to articulate their reasoning and integrate diverse perspectives. In addition, term projects and internship projects based on a real-life projects could offer an opportunity for students to practice their aptitude for solving unstructured problems. 

It would be quite difficult for automated learning systems to provide such dynamic interactive learning opportunities for this level of learning. In particular, computer systems could not "understand" a student's work enough to provide any meaningful feedback, nor could a computer system come up with its own creative solutions. 

On the other hand, automated learning systems could be used to provide recorded case discussions to students. Students would be able to observe and, to some extent, participate in, discussions in a quite limited fashion. In particular, these recorded case discussions could be scripted based on real in-class case discussions, presenting many diverse perspectives and ideas. 
In addition, multimedia case studies could be also used to provide unstructured problems on which students could practice. This type of automated learning system can present in an interactive fashion a large amount of relevant case information that allows the student to experience the complexity of real-life problems. Although the system may not be able to provide substantive feedback on students' work, it could include various commentaries of instructors and practitioners that could guide students. Business simulation can also provide similar dynamic and complex unstructured problem solving situations where students may practice integrating their knowledge and skills. 

It is important to note here that these levels of teaching do not necessarily represent a prescribed sequence in which a student should learn. For example, the synthesis level-teaching could proceed the other two levels so that students are exposed to the situations in which diverse sets of knowledge and skills are needed. In addition, the synthesis-level teaching might be done at the end of a curriculum as a final capstone course or internship project so that students can apply to a complex unstructured problem the knowledge and skills they have learned in the curriculum. 

Nonetheless, however these levels of teaching are sequenced, efficiency gains at any level could benefit all levels of teaching. For instance, if automated learning systems could effectively facilitate, in part or as a whole, the retention-level instructions, instructors could spend more time in-class focusing on the application- or synthesis-level teaching. 

In addition, it is worth noting here some of major difficulties in designing a study that measures the effectiveness of automated learning systems. In particular, one can easily envision an experimental study where the same teaching materials are presented to students by an instructor and by an automated learning system. This type of direct comparison between human instructor and automated learning systems should be interpreted with caution for the following reasons. 

First, in practice, automated learning systems and human instructors are not likely to teach the same materials. Indeed, one of the advantages of using automated learning systems is that, thanks to the economies-of-scale, automated learning systems could be prepared with more time and effort than an instructor could personally do. Second, the cost of having a human instructor and the cost of using an automated learning system would be significantly different. Third, students could use automated learning systems at their convenience and at their own pace while interaction with an instructor would be limited by practical constraints. For all these reasons, direct comparison between human instructors and automated instruction should be interpreted with caution. 

4. CONCLUSION

In this paper, I argued that automated learning systems hold an important place for the future of IS education. Much like similar types of automation have revolutionized the efficiency of our economy, automated learning systems could significantly enhance the efficiency of IS education. 

To promote systematic research in this area, I presented a contingency model for investigating diverse situations in which automated learning systems may be most effective. Future research efforts should be directed toward an empirical examination of the effectiveness of automated learning systems in these contingencies along with other contingencies that may affect the effectiveness of these systems. 

In addition, we need to spur active discussions in the IS academic community in order to examine how these automated learning systems should be developed and shared. Undoubtedly, collaborative development and sharing among many universities could be beneficial to everyone. However, all stakeholders-IS educators, university administration, accreditation agencies, and even textbook publishers-need to participate in the discussion of how such collaborative development and sharing should be facilitated. 


5. REFERENCES

Alavi, M. (1994), "Computer-Mediated Collaborative    Learning: An Empirical Evaluation," MIS Quarterly, 18(2), 159-174.

Alavi, M., Wheeler, B. C., and Valacich, J. S. (1995), "Using IT to Reengineer Business Education: An Exploratory Investigation of Collaborative Telelearning," MIS Quarterly, 19(3), 293-312.

Alavi, M., Yoo, Y., and Vogel, D. R. (1997), "Using Informaiton Technology to Add Value to Management Education," Academy of Management Journal, 40(6), 1310-1333.

Arge, P. E. (1999), "The Distances of Education," Academe, 85(5), 37-41.

Arthur, J. B. and Airman-Smith, L. (2001), "Gainsharing and Organizational Learning: An Analysis of Employee Suggestions Over Time," Academy of Management Journal, 44(4), 737-754.

Bagley, C. and Hunter, B. (1992), "Restructuring, Constructivism, and Technology: Forging a New Relationship," Educational Technology(July), 22-27.

Biddle, B. J., Bank, B. J., and Slavings, R. L. (1987), "Norms, Preferences, Identities and Retention Decisions," Social Psychology Quarterly, 50(4), 322-337.

Brown, A. D. and Starkey, K. (2000), "Organizational Identity and Learning: A Psychodynamic Perspective," Academy of Management Review, 25(1), 102-120.

Brynjolfsson, E. and Hitt, L. M. (1998), "Beyond the Productivity Paradox," Communications of the ACM, 41(8), 49-55.

Chung, W. and Henderson, J. C. (2001), "Americans on the Internet: Utilitarian and Social Participation Perspectives," Sprouts: Working Papers on Information Environments, Systems, and Organizations, 1(1).

Couger, J. D., Davis, G. B., Dologite, D. G., and Feinstein, D. L. (1995), "IS '95: Guideline for Undergraduate IS Curriculum," MIS Quarterly, 19(3), 341-359.

Davenport, T. H. and Short, J. E. (1990), "The New Industrial Engineering: Information Technology and Business Process Redesign," Sloan Management Review(Summer), 11-27.

Davis, S. A. and Bostrom, R. P. (1993), "Training End Users: An Experimental Investigation of the Roles of the Computer Interface and Training Methods," MIS Quarterly, 17(1), 61-85.

Dutton, J. E., Dukerich, J. M., and Harquail, C. V. (1994), "Organizational Images and Member Identification," Administrative Science Quarterly, 39, 239-263.

Edmondson, A. (1999), "Psychological Safety and Learning Behavior in Work Teams," Administrative Science Quarterly, 44, 350-383.

Frank, R. H. and Cook, P. J. (1995). The Winner-Take-All Society: Why the Few at the Top Get So Much More Than the Rest of Us. New York, NY: The Free Press.

Gorry, G. A. and Scott Morton, M. S. (1971), "A Framework for Management Information Systems," Sloan Management Review(Fall).

Hammer, M. and Champy, J. (1993). Reengineering the Corporation: A Manifesto for Business Revolution. New York, NY: HarperBusiness.

Ibarra, H. (1999), "Provisional Selves: Experimenting with Image and Identity in Professional Adaptation," Administrative Science Quarterly, 44, 764-791.

Ives, B. (1994), "Transforming the Learning Industry," MIS Quarterly, 18(1), v-vii.

Lang, J. R. and Dittrich, J. E. (1982), "Information, Skill Building, and the Development of Competence: An Education Framework for Teaching Business Policy," Academy of Management Review, 7(2), 269-279.

Lee, D. M. S., Trauth, E. M., and Farwell, D. (1995), "Critical Skills and Knowledge Requirements of IS Professionals: A Joint Academic/Industry Investigation," MIS Quarterly, 19(3), 313-340.

Leidner, D. E. and Jarvenpaa, S. L. (1993), "The Information Age Confronts Education: Case Studies on Electronic Classrooms," Information Systems Research, 4(1), 24-54.

Leidner, D. E. and Jarvenpaa, S. L. (1995), "The Use of Information Technology to Enhance Management School Education: A Theoretical View," MIS Quarterly, 19(3), 265-291.

Lengnick-Hall, C. A. and Sanders, M. M. (1997), "Designing Effective Learning Systems for Management Education: Student Roles, Requisite Variety, and Practicing What We Teach," Academy of Management Journal, 40(6), 1334-1368.

Lim, K. H., Ward, L. M., and Benbasat, I. (1997), "An Empirical Study of Computer System Learning: Comparison of Co-Discovery and Self-Discovery Methods," Information Systems Research, 8(3), 254-272.

Markus, M. L. (1994), "Electronic Mail as the Medium of Managerial Choice," Organization Science, 5(4), 502-527.

Martocchio, J. J. and Webster, J. (1992), "Effects of Feedback and Cognitive Playfulness on Performance in Microcomputer Software Training," Personnel Psychology, 45(3), 553-578.

Mathieu, J. E., Tannebaum, S. I., and Salas, E. (1992), "Influences of Individual and Situational Characteristics on Measures of Training Effectiveness," Academy of Management Journal, 35(4), 828-847.

McGrath, R. G. (2001), "Exploratory Learning, Innovative Capacity, and Managerial Oversight," Academy of Management Journal, 44(1), 118-131.

Morrison, E. W. (1993), "Newcomer Information Seeking: Exploring Types, Modes, Source," Academy of Management Journal, 36(3), 557-589.

Mowday, R. T. (1997), "Reaffirming Our Scholarly Values," Academy of Management Review, 22(2), 335-345.

O'Loughlin, M. (1992), "Rethinking Science Education: Beyond Piagetian Constructivism Toward a Sociocultural Model of Teaching and Learning," Journal of Research in Science Teaching, 29(8), 791-820.

Santhanam, R. and Sein, M. K. (1994), "Improving End-User Proficiency: Effects of Conceptual Training and Nature of Interaction," Information Systems Research, 5(4), 378-399.

Schloss, P., Wisniewski, L., and Cartwright, G. P. (1988), "The Differential Effect of Learner Control and Feedback in College Students' Performance on CAI Modules," Journal of Educational Computing Research, 4(2), 141-150.

Schultz, M. (2001), "The Uncertain Relevance of Newness: Organizational Learning and Knowledge Flows," Academy of Management Journal, 44(4), 661-681.

Simon, H. (1977). The New Science of Management Decision. Englewood Cliffs, NJ: Prentice Hall.

Vandenbosch, B. and Higgins, C. (1996), "Information Acquisition and Mental Models: An Investigation into the Relationship between Behavior and Learning," Information Systems Research, 7(2), 198-214.

Webster, J. and Hackley, P. (1997), "Teaching Effectiveness in Technology-Mediated Distance Learning," Academy of Management Journal, 40(6), 1282-1309.