Threaded Live Case Study Lessons Learned Les Waguespack, Ph.D.1 Computer Information Systems Department, Bentley College 175 Forest St., Waltham Massachusetts 02154-4705, USA Abstract The case study (a.k.a. "Harvard Case") is a widely respected and utilized pedagogical instrument in management education. Although commonly used in upper level IS courses relating to strategic planning and policy, it is not so commonly used in IS courses relating to technology capability and application. This paper presents a discussion of the case study concept as it has been used for the past four years across segments of an IS curriculum. It differs from the Harvard case in two ways: first it is live, engaging students in IS practice and second it is threaded, intertwining a series of IS courses covering various curriculum aspects. This paper presents results of this approach and survey results from four years of students participating in the pedagogy. The concept is interpreted in the BSCIS program at Bentley College. KEYWORDS: Case study, pedagogy, IS curriculum, IS practicum 1. INTRODUCTION In an earlier work I introduced the concept of the threaded live case study (Waguespack1 , 1997). This pedagogical device is intended to address the challenge of providing Information Systems, IS, Computer Information Systems, CIS, and Management Information System, MIS, students with the opportunity to apply IS theory and practice in a rich and realistic application development exercise. That earlier work introduced the issues as follows. One of the characteristics that typically distinguish IS and CIS from MIS academic programs is a commitment that graduates possess high quality and mature application development skills. One of the characteristics that distinguish IS and CIS from CS academic programs is that graduates possess application domain knowledge grounded in the business disciplines (e.g. management, accountancy, marketing and finance). The business application domain enfolds the study of systems, analysis, modeling and design. The challenge for IS and CIS programs is to steward the limited program credit hour resource available in most business colleges and to optimally engage students in practical experiences. And by doing so we strive to expose students to the realities of applying best of breed IS practices in real world business situations. The computing industry, professional organizations and institutions of higher education are committed to preparing high quality IS professionals. Their commitment is evidenced by their investment of time, people and energy in the Information Systems Curriculum Guidelines, IS '97 (earlier drafts of which were called IS '95) (Couger et al 1997). IS '97 witnesses to the importance of information system professionals to the domestic and global economies. It addresses higher education's responsibility to prepare future generations of IS professionals grounded in business domain knowledge. The sheer breadth and depth of knowledge in business and information technology described in IS '97 pose a daunting challenge to business schools in general and IS curriculum designers in particular. The threaded live case study is a vehicle for integrating domain knowledge in the IS curriculum. Case study based pedagogy is not a new concept (Christenson et al 1991). It is widely respected and utilized in management education. And although commonly used in upper level IS courses relating to strategic planning and policy, it is not so commonly used in IS courses relating to technology capability and application. For the past several years the threaded live case study has been used in a suite of IS courses in the CIS curriculum at Bentley College. In this paper I discuss one faculty member's experience with the pedagogy and report on a survey of student attitudes to the pedagogy at various points in their experience. (I also include a brief discussion of the statistical treatments used.) I hope that the discussion presented here will encourage new development of the case concept in the IS education community. I describe a pedagogical device, I call the threaded live case study, and describe its application in an interpretation of IS '97, the BSCIS'96 (Waguespack et al 1996) curriculum at Bentley College. I present a description of two cases used and the three courses through which they have been threaded. Finally, I consider possible implications and improvements to the pedagogy based on our experience. 2. A PEDAGOGICAL DEVICE As discussed in the antecedent work (Waguespack et al 1996) , business curricula carry a heavy burden in trying to balance business domain and information systems domain education within the strictures of limited academic credit hours. The primary goal of the threaded live case study is to maximize the educational value of their application development exercises by integrating theory and practice learning and minimizing the "ramp up time" required to "get into a problem." The following excerpt from (Waguespack 1997) describes the pedagogical device. LIVE CASE STUDY Experiential learning (also called "learning-by-doing") is the most effective pedagogy in preparing IS professionals. Experiential learning of advanced IS methodologies and skills is a real challenge for student and teacher. The primary difficulty lies in the fact that "industrial strength" professional practice works best on "industrial strength" problems. Professional practice experiences with C.A.S.E. tools, project management, problem decomposition techniques, data modeling methods and tools, version control systems, documentation management disciplines, group ware systems, etc. - none of these "scale down" effectively. Using these professional methods or tools on undersized problems leaves students (and sometimes teachers) confused and often frustrated. The details of the methods combine with the intricacies of the application problem and domain. Students often fail in their attempts to grasp professional practice not because of their ineptness with the technology or concepts, but rather because of an ignorance of the essence of the problem and its domain. A typical approach to providing students with practical experience in professional practice is the live case study. A live case study is an exercise in which the students participate (e.g. analysis, design or development, etc.) designed to illuminate particular theories or issues of professional IS practice. The task may be individual or team oriented. From the teacher's perspective the more authentic the experience the more expensive the task of preparing the student to understand the application problem itself, let alone the technology to be applied. The aspect of the problem that will be most directly manipulated, that part which is directly in contact with the technology being applied is usually not particularly complex. Rather, it is the contextual motivation found in the application domain for seeking a particular result, for applying the technology in the first place that is lost without the student's understanding of the problem's domain. Herein lies the teacher's dilemma. Does the teacher make a large time investment (in and out of the class) to prepare a realistic application domain for the student to understand and then spend the time to teach the student about the domain? Time spent on the application domain itself is not spent on the technology application experience. Or does the teacher prepare a more simplistic experience of applying the technology that may not convey the professional perspective that the student needs to have? If the technology experience does not engender confidence in the students, the students are unlikely to develop confidence in the technology or in their own use of it. Both of which are primary goals in the development of any professional! (Technology transfer research shows that methods and tools in which professionals have little faith are seldom, if ever, applied. And when they are applied they are done so halfheartedly (Raghavan 1989) . THREADED LIVE CASE STUDY One possible pedagogical vehicle for providing "just in time" application domain knowledge is the threaded live case study . A threaded live case study defines an application domain (Waguespack 1994) enclosing several application problems. It is designed to inform the student about the context surrounding the application problems. Each problem is a platform upon which specific IS methodologies, technologies or analyses may be applied to achieve specific learning objectives. Working each problem imparts knowledge and experience with particular principles, concepts, and skills of professional practice. A threaded case is intended to be incrementally presented to the student. Each increment unfolds new aspects of the application domain which in turn present a new opportunity to explore IS practice. Experiencing multiple aspects of professional practice within the same application domain allows the students to explore the interplay that exists among theories, methodologies, and tools - reflecting a system of professional behavior rather than collection of loosely related technical ideas. 3. COURSES STITCHED WITH THE THREAD Over the past four years I have employed the threaded live case study in a suite of three upper level CIS courses at Bentley College. Their catalog descriptions follow are found in appendix A. CS360 is a course in systems analysis and design. CS460 is a course in applied information systems project management. CS450 is a course in object technology emphasizing object modeling and object oriented software engineering. CS360 is a required junior level course for the CIS and the Accounting Information Systems (AIS) majors. CS460 is a senior level course required for the CIS major. CS450 is a senior level CISelective usually taken after CS460 although some students may take these simultaneously. Two cases have been applied as threads through these courses over the past three years. One or the other of two cases has been threaded through CS360, CS460 and CS450 over the past four years. The first is called the degree requirement summary, DRS, project. The second is the referee committee project. DRS Case Thread The DRS is an information tool used by every Bentley student. It includes a listing of each required course, elective option, experience or distribution experience required to complete a specific degree. In its document form it is the focus of advising, academic credit transfer, course substitution and major change for every student. The DRS's roots in the course catalog and student transcripts presents a rich multi-layered requirement domain. The DRS is used in information and process modeling in CS360. Each student was required to work alone. In particular, entity relationship (ER) modeling something that the students perceive as a document, instead as a collaborating system of relational tables is a difficult task. In CS460 students are asked to work in assigned development teams to devise an information aid embodying the functionality of the DRS in a more "user friendly" manner. This might be described as a "personal curriculum assistant." It may take the form of a "Palm Pilot" application or a web based kiosk. In CS450 students are asked to revisit their modeling experience of CS360 using the DRS, but this time using the object oriented paradigm to model the domain in groups. This serves as a dramatic backdrop within which to contrast process and data modeling against object modeling. Referee Committee Case Thread The Massachusetts state referee committee project (called MSRC) derives from the organization that oversees soccer referees throughout the United States. In each of the fifty states there is a state referee committee under the auspices of the United States Soccer Federation. These state committees are responsible for recruiting, training, registration, assessment, upgrade and (to some degree) assignment of soccer officials to competition ranging from youths under 8 years of age to the matches of Major League Soccer. In Massachusetts there are 4000+ registered referees at various levels of experience and professional competency. Closely connected with referees is the process of referee assignment to league and tournament matches throughout the year. In CS360 students are asked individually to model the information resources suitable for managing this large pool of independent referee contractors. Issues of information dissemination, correspondence, coordination, and data collection are tedious. In CS460 students assigned to teams are asked to provide registration and assignment services. The latter involves all the intricacies of time tabling and conflict resolution. In CS450 as with the DRS project, students are asked to revisit the basic information architecture of the MSRC to determine if object modeling might elicit more or different information patterns from a domain perspective. Each of these cases is documented by web sites providing the basic background descriptions and references to related materials available in most cases on the WWW. The sites were prepared once and used by each class working on that particular case. Student project products (models, prototypes, and documentation) were submitted to the instructor for evaluation and grading. 4. SURVEYING STUDENT PERCEPTIONS In the spring of 2000 I had experienced three full tracks of students through the threaded live case studies as I had been able to teach succeeding sections of CS360, CS460 and CS450. I did not teach all sections of these courses, however. Therefore there were students who experienced one of the cases in one, two or three of the courses. Their first experience with one of the threaded cases could have been in any one of the three courses as well. I devised a survey to be administered to students near the completion of their course. It solicits their perception of difficulty, challenge, helpfulness and satisfaction regarding the particular threaded case they experienced. The survey included questions to determine their longitudinal perceptions as well. That is, they were asked to indicate their attitude toward the case in their first, second and/or third encounter with a case study (depending on how many they had had). Each student answered the survey once. The survey is found in the appendix B. (Students who had not yet taken either or both of CS460 or CS450 were asked to answer questions 17-20 as predictions.) The survey data was compiled and analyzed using StatView 5.0.1. (StatView 1999) Statistical Outcomes The experience reported here includes input from 105 students - of those 64 took CS360, 67 took CS460 and 53 took CS450. Of those 48 took CS360 and CS460, and 43 took CS460 and CS450. Finally, 28 took all three courses. In each of these cases if the student took more than one course, the same case study was experienced. There was no significant difference found in the means of responses from students who worked with the DRS or MSRC cases. Students' overall attitude toward the case studies were very positive, CS360: 89%, CS460: 96% and CS450: 93% responded that the case helped their learning of the respective course's learning objectives while 98%, 91% and 88% found the case challenging. To the question of whether the same case would be boring they responded 23% in the affirmative overall. However, the expectation / experience of boredom decreased as the same case was experienced two or more times, once- 31%, twice- 20%, thrice- 14%. Regardless of their boredom perception, 68% felt that the same problem helped and 58% agreed that the same case should be used for all three courses. Because students received the survey after CS360 or CS460 or CS450 there is some indication of an evolution in their perception of the cases over time. In particular, students reporting on their experience with the DRS case after working with it a third time more frequently indicated that they did not understand it in CS360 than those who had seen it only once in CS360. This may indicate that experience with the same case does indeed lead to a deeper perception of the intricacies of the problem domain; a perception that they were unable to achieve with only one involvement. (As a contributing factor, the difference in project requirements from one course to the next tends to address problem characteristics that are more subtle or complex than earlier exercises.) Did repetition of involvement in a particular problem domain engendered in the case improve the students' understanding of the case? Of those who took two of the three courses using the same case 26% indicated that their understanding improved. Of those who took all three courses using the same case 32% indicated that their understanding improved. Nearly a third of all students experiencing the same case two or more times reported improved understanding. As a final positive indicator for the pedagogy, 42% of those experiencing it once, 80% of those experiencing it twice and 93% of those experiencing it three times responded that the case helped their learning in the respective courses. (Some words of reservation are in order. Although the aggregate numbers of responses are non-trivial, the fine partitioning of responses into cells often leaves too few to make assertions of significance. And therefore, from a statistical perspective these results can only be used to encourage further study.) Subjective Outcomes The preliminary statistics indicate that the threaded live case study as a pedagogical device yields some expected benefits. Student understanding improves with repeated exposure to the same problem domain. Students perceived that their understanding of the problem domain improved and that their learning of the IS theory and practice were helped by the cases. Improved understanding by a substantial segment of the students revisiting a case would seem to outweigh any expectation or experience of boredom with the same case. The amortized cost of preparing a thorough set of materials for the students to reference is a real benefit to the instructor. This savings allows both more complex problem domains and more extensive reference materials to be prepared as the instructor sees fit. And although these cases were used only by one instructor they are available to more instructors of the same course or other courses with little of no augmentation. In the introduction to the threaded live case study I predicted three benefits (Waguespack 1997): "First, students receive an enriched set of individual application problem experiences. The experiences are richer because their understanding of the enclosing application domain is more complete than could be otherwise expected. Second, students develop a better overall understanding of the professional practice. This is because solving several problems (even with different technologies) within the same domain makes the interrelationships within the professional practice more easily observable. Third, the natural integrating effect on the courses threaded by the case study strengthens the students' perception of the cohesiveness of the curriculum. A sense of 'wholeness' about the professional practice makes students more confident in the technology and in their overall grasp of their discipline." The survey results presented here address to some extent the first two expected benefits. Student responses indicate positive evidence of both. Examining the third benefit requires more study, but seems well suited for a similar survey approach in the future. The limitations in this study (single instructor, two specimen case studies, and three threaded courses) must be addressed by an expanded treatment of threaded live case studies if statistically significant conclusions are to be achieved. The preliminary results reported here do not prove its value, but I am strongly encouraged to continue pursuing the pedagogy with additional monitoring. The next hurdle will be to introduce this pedagogy into other CIS courses and eventually in courses across business departments. Each of these initiatives raises the stakes both in terms of coordination cost and potential benefits across a curriculum. 5. CONCLUSION Delivering a practical experience in application development in the academic setting is very challenging. The challenge is compounded by the speed with which the technology "tools" emerge, evolve and fade from professional practice. The potential of delivering more learning in the same course package may be more than a fond desire, it may be the difference between average and "best of breed" CIS education. The threaded live case study demonstrates real promise as a tool for increasing the learning density in our CIS courses. More study is needed. My experience so far leads me to believe the investment is warranted. 6. REFERENCES Christenson, C.R., D.A. Garvin & A. Sweet Eds. 1991, Education for judgment: the artistry of discussion leadership, Harvard Business School Press. Couger, J.D., G.G. Davis, J.T. Gorgone, D.L. Feinstein, and H.E. Longenecker, Jr., 1997, IS'97, Model Curriculum and Guidelines for Undergraduate Degree Programs in Information Systems, School of CIS, University of South Alabama, Mobile, AL 36688, Spring. Raghavan, S.A. & D.R. Chand, 1989, "Diffusion of Software Engineering Technology," IEEE Software, July, pp. 81-92. Rossi, P. H., J.D. Wright, & A.B. Anderson, 1999, Handbook of Survey Research, Academic Press, New York, NY, 1983.StatView Reference, SAS Institute Incorporated, August. Waguespack, L.J., 1994, "Domain Analysis is an Essential Skill of the OO Analyst," Proceedings of ISECON '94, October 28-30, Louisville, KY, pp. 186-193. Waguespack, L.J., D. R. Chand, & J.D. Robertson, 1996, "Crafting A Business Bachelor of Science Program in Information Systems Following on IS'95," Proceedings of ISECON'96, October, St. Louis, MO. Waguespack, L.J., 1997, Threaded Live Case Study, Weaving Practice Through IS'97, Proceedings of ISECON '97, October, Orlando, FL. Appendix A Descriptions of Courses Using the Threaded Case Studies CS360 Analysis, Modeling and Design (three credits) Prerequisite: Procedural and Data Abstractions This course begins with business functional analysis and ends with information systems design. Students are introduced to tools and techniques enabling effective analysis, design and documentation of an information system. The student learns formal methodologies that form the basis of modern information systems development practices. Models that focus on the articulation of business functions, integrating process, data and behavioral abstractions form the core of formal methods in systems development and computer assisted systems engineering. CS460 Applied Software Project Management (three credits) Prerequisite: CS360 Students learn and experience the process of information systems development through managing team dynamics and performing software engineering project management. Specific topics discussed include the value of different software development life cycles, project management tools and techniques, software process management practices and software quality management practices. This course fuses the student's prior IT and business education preparing them to launch their professional IT careers. CS450 Object Oriented Technology (three credits) Prerequisite: CS360 This course teaches the object oriented paradigm which encompasses object oriented analysis, design, programming and database. By focusing on object oriented modeling the student is prepared to assimilate the widest variety of methodologies and tools applicable to object oriented information system building. Appendix B Student Survey Instrument All questions except for yes/no are 5 point Likert (Rossi et al 1999) ranging from strongly disagree to strongly agree with don't know as the median. 1. Did you take CS360 with this professor? Yes No if you answered NO to this question skip to quest. 6 2. I did not understand the DRS problem in CS360. 3. The DRS problem in CS360 helped me learn to create DFD's. 4. The DRS problem in CS360 helped me learn to create ERD's. 5. The DRS problem in CS360 was challenging for me. 6. Did you take CS460 with this professor? Yes No if you answered NO to this question answer only 11, 15, 20 below. 7. I did not understand the DRS problem in CS460. 8. The DRS problem in CS460 helped me learn applied project management concepts. 9. I was proud of the project my team completed for CS460. 10. The DRS problem in CS460 was challenging to me. 11. I did not understand the DRS problem in CS450. 12. The DRS problem in CS450 helped me learn OO concepts. 13. The DRS problem in CS450 helped me learn OO modeling. 14. I am proud of the final project my team completed for CS450. 15. The DRS problem in CS450 was challenging for me. 16. Using the same problem in CS360, CS460 and CS450 made learning in those courses easier for me. 17. Using the same problem in CS360, CS460 and CS450 was boring. 18. The problems in CS360, CS460 and CS450 should all be different. 19. Using the same case project in CS360, CS460 and CS450 made studying the concepts in those courses easier for me. 20. The same case problem should be used in CS360, CS460 and CS450. Questions were adjusted to account for which case study was relevant to the student group. 1 LWaguespack@Bentley.edu